Is ZeroGPT Accurate? We Tested It on 30 Short Emails (2026)

Mikhail Konkov · Founder, Limato

Non-native English speaker living in Spain. Wrote the 15 non-AI emails in this study, then ran all 30 through five AI detectors — including his own.

Is ZeroGPT accurate on short email? On this corpus it errs cautious rather than trigger-happy: ZeroGPT scored every one of the 15 emails in our non-AI set at 0% AI, and it missed 2 of the 15 AI-written emails at the default ≥50% line. It was also the only detector in the test whose two sets did not overlap at all — 0% on all 15 non-AI emails, 26.2% or higher on all 15 AI ones — and it did that on 64–86-word emails, every one of them under the 150-word minimum its own FAQ asks for. Limato ran all 30 emails through five detectors — ZeroGPT, GPTZero, QuillBot, Grammarly and Limato's own AI Detector — on 2026-08-15, at two thresholds, with confidence intervals. The result that should worry you is not any single tool's error rate: it is that the tools disagree with each other so hard that 4 of the 30 emails were scored 0% by one detector and 100% by another.

This page also carries a correction. An earlier version of this study, run the day before, found that detectors flagged 63–100% of non-native business emails as AI. That finding was wrong, it has been withdrawn, and the section explaining why is below rather than buried. Every number on this page comes from the replacement run.

We keep measuring this because Limato ships a free AI Detector of its own and it faces the identical risk: scoring a non-native writer's own words as machine-generated. On the new data it fails that test worse than any tool we compared it against at the ≥50% line this post applies to everyone — and better than that count suggests at the ≥70 boundary where the product actually prints the word AI. Both numbers, and the one that is worse for us than either, are in that section.

Method

30 short business emails, five detectors, one run per detector per item, run date 2026-08-15. 15 items are AI-written; 15 are not.

• Non-AI set (hw-01–hw-15, n=15) — written by the author of this post, a non-native English speaker with a single first language. Five of them (hw-01–hw-05, labelled hand) he typed in English from scratch. The other ten (hw-06–hw-15, labelled model-draft-hand-edited) started from drafts a model generated against a pattern extracted from those five, and he edited every one of them. Both labels are recorded per item in corpus-hw.json, and the results are reported split by them below as well as pooled.
• AI set (ai-*, n=15) — business emails written by a model, unedited. Nobody touched them. They are the 15 longest kind: "ai" items from the same corpus as above, already published in esl-email-2026-08/corpus.json, so they are not duplicated in this study's data directory.

Lengths, in words:

Why short matters, and where that puts us relative to each tool's own floor. A real business email is short. Several detectors state a minimum input length that this entire corpus is under — which is the point of the study rather than a defect in it, but it has to be on the record before any result is read.

A stated minimum is not always an enforced one: in a length probe outside this sample, both GPTZero and QuillBot returned a real score for a 5-word, 21-character input.

Two thresholds. Every score is evaluated at the common default of ≥50% AI and at a stricter ≥20% AI cutoff. The strict cut is not a straw man — it is how a suspicious reader actually behaves, treating any non-zero flag as a mark against you. Reporting both shows how much of a detector's reputation rests on threshold choice alone.

One run per text per detector. No retries, no best-of-N. Detector scores are known to be non-deterministic; a single run understates that variance, and that is listed as a limitation below.

One wrinkle in that run date. The 15 non-AI items were scored on 2026-08-15. The 15 AI items were scored on 2026-08-14 and carried over unchanged when the human half of the corpus was replaced — they were never re-run, apart from one Grammarly cell re-collected on 2026-08-15. results.csv stamps the rebuild date, 2026-08-15, on all 120 of its rows; the raw *.jsonl collector logs carry the true per-call dates, and anyone checking the numbers should use those.

The result we had to throw away

The first version of this study used a different human set: 30 non-native business email drafts from esl-email-2026-08/corpus.json, the same corpus behind our earlier rewrite measurement. Run on 2026-08-14, it produced a striking result.

Those numbers mean nothing, and we are publishing them anyway so the record is complete.

The 30 drafts were model-generated. They were built to imitate non-native business email, but a model wrote every word of them and no person was involved in producing them at any point — not writing, not editing. So when ZeroGPT flagged 19 of them, ZeroGPT was correct. When GPTZero flagged all 30, GPTZero was correct. A false-positive rate computed over machine-written text is not a false-positive rate; it is a detection rate with the label flipped. The number that looked like an indictment of the detectors was, read properly, the detectors doing their job.

The consequence is worth stating plainly, because it cuts against a claim this site has repeated and that circulates widely: "AI detectors punish non-native writers" did not reproduce once the non-AI half of the corpus was written by a person rather than by a model. On the replacement set the same detectors flag 0 to 2 of the 15 non-AI items at ≥50%. If there is a penalty on non-native English at this length, this study did not find it — on 15 emails from one author with one first language, which is thin enough that "did not find it" is the strongest form the claim can take.

Two caveats on the withdrawn table itself. Those 30 items are no longer in results.csv — the numbers above were recovered from the raw per-call collector logs, which still hold one line per item per detector, all dated 2026-08-14. For 9 of the 150 item-by-detector cells (8 ZeroGPT, 1 GPTZero) only a repeat-run reading survives; the repeat reading was used rather than dropping the item. The recovered ZeroGPT, QuillBot and Grammarly counts — 19/30, 23/30 and 24/30 — reproduce the figures the first version of the study reported exactly, which is the main reason to trust the reconstruction.

And the comparison is not clean. The old set and the new one differ in length (63–126 words, median 69.5, versus 64–86, median 74), in how they were constructed, and in whether a person was involved in producing them at all. This is a documented before-and-after of our own methodology. It is not a controlled A/B on identical text, and nobody should read the swing as an experiment about corpus construction.

False positives on the replacement set

A false positive here means a detector scored an item above the threshold when it was not AI-written. All 15 non-AI items count, at both thresholds.

Read the confidence intervals before the point estimates. At n=15 a 0/15 result still admits a true rate as high as 20.4%, and 2/15 spans 3.7% to 37.9%. Nothing in this table separates the third-party tools from each other at any useful resolution. What it does support is a negative: none of them produced the mass flagging the withdrawn corpus appeared to show.

Does it matter how the email was produced?

Five of the 15 non-AI emails were typed in English from scratch; the other ten started from a model draft the author then edited. Splitting the same measurement along that line is the obvious robustness check, so here it is.

Threshold ≥50%

Threshold ≥20%

On ZeroGPT and QuillBot the two groups are identical: 0 flags in either group, at either threshold. Grammarly flags exactly one item in each group at both thresholds — hw-01, typed from scratch, and hw-12, from a model draft. The two groups separate only on GPTZero and Limato, and they separate the same way: both flag the model-drafted-then-edited items more often (GPTZero 2/10 against 0/5 at ≥50%, Limato 8/10 against 2/5 at ≥20%). With 5 and 10 items the intervals overlap almost completely, so that is a direction, not a difference this data can size.

False negatives on AI-written email

The 15 AI items were model-written and not edited. A detector scoring one below the threshold is a false negative — the opposite failure, and the one that matters if your goal is catching machine text rather than not accusing a person.

Unedited model output at email length is easy to catch: worst case here is 2 of 15 missed. The interesting failure at this length is not missing AI text. It is what happens on everything that is not obviously AI text.

ZeroGPT specifically

ZeroGPT is the tool most people mean when they search "is ZeroGPT accurate", so it gets its own read beyond the tables.

It flagged nothing on the non-AI set. 0 of 15 at ≥50%, 0 of 15 at ≥20%. And it did not merely stay under the line: it returned exactly 0% on every single one of the 15 items. Not 3%, not 11% — zero, fifteen times.

It missed 2 of 15 AI-written emails at ≥50% (13.3%, 95% CI 3.7–37.9%), the joint-worst false-negative rate in the set alongside QuillBot. At the strict ≥20% cut it missed none.

And all 15 non-AI items are below its own stated floor. ZeroGPT's own FAQ answers “What text length works best?” with “At least 150–200 words”; the longest item in the non-AI set is 86. This is the part worth thinking about. A tool operating below its documented minimum could behave in either direction — it could guess wildly, or it could refuse to commit. ZeroGPT does neither, and the result is the cleanest number in the study: exactly 0% on all 15 non-AI items, and 26.2% or higher on all 15 AI ones. The two sets do not overlap at all. It is the only detector here that separates them — GPTZero, QuillBot, Grammarly and our own each score at least one non-AI item above their own lowest AI item.

Read carefully what is being separated, though. The line it draws is between unedited generic business prose and short email in a non-native register — and it draws it the same way whether the email was typed from scratch or edited out of a model draft, returning a flat 0% on all 15 either way. So a 0% from ZeroGPT is not a statement about who wrote the words. It is ZeroGPT saying the text does not read like unedited model output at this length.

The detectors do not agree with each other

Every one of the 30 items has a score from all five detectors, which makes disagreement directly measurable. It is large.

• 9 of 30 items (30.0%) have at least two detectors scoring 50 percentage points or more apart.
• 4 of 30 items (13.3%) were scored 0% by one detector and 100% by another. Same text, same day, opposite verdicts at full confidence.
• Across the four third-party tools, the per-item spread (highest score minus lowest) has a median of just 5 percentage points — but a maximum of 100 and a minimum of 0. Most of the time they cluster; when they break, they break completely.

At the ≥50% line, where the score collapses to a yes/no, pairwise agreement across all 30 items runs from 80.0% to 93.3%:

80% agreement between two tools sounds high until you notice that a corpus which is half AI-written by construction makes agreement cheap: both tools flagging the same obvious machine text costs them nothing. The disagreement concentrates exactly where a person's reputation would be at stake.

Three worked examples

Unedited detector output, one run each, each item labelled with how it was produced. Scores are each tool's own 0–100% AI likelihood.

Example 1 — hw-01, typed in English from scratch, 78 words

This is the clearest false positive in the study. The email reads as a non-native speaker's own English — "should be arrive", "in the documentation absent actual invoice", "during current day" are the author's first-language syntax wearing English words — and it was typed out from scratch by the person whose first language is doing the interference. Three detectors read it correctly. Grammarly returns 100%, its maximum. Our own detector returns 52% — over this post's ≥50% line, though our own interface labels that score mixed rather than AI.

Example 2 — hw-12, edited from a model draft, 72 words

Look at the row as a whole: 0, 42.2, 0, 100, 28 for one email. This is one of the 4 items scored 0 by one detector and 100 by another. Two tools see nothing, one sees a coin flip, one is certain. Whatever these numbers are, they are not five estimates of the same quantity. The typo in the first word ("afternoom") survived the hand edit; whether any of the five keyed on it is not something one run per text can answer.

Example 3 — hw-07, edited from a model draft, 83 words

Another 0-to-100 split, running the other way from Example 2: GPTZero is maximally certain this is machine-written while ZeroGPT, QuillBot and Grammarly all return a flat 0%. GPTZero's 100% and Limato's 78% are two of the false positives counted in the tables above. Five tools, one email, and no majority in either direction: at least three of the five are wrong here, and one run per text cannot tell you which three.

Our own detector, reported against itself

Limato's AI Detector was the worst false-positive offender in this study. On the 15-item non-AI set it flagged 4 at ≥50% (26.7%, 95% CI 10.9–52.0%) — twice the rate of GPTZero and Grammarly, against 0 of 15 for ZeroGPT and QuillBot. At ≥20% it flagged 10 of 15 (66.7%, 95% CI 41.7–84.8%), while ZeroGPT and QuillBot still flagged none. One of the four it flagged at ≥50% is hw-01 in Example 1 above.

The mirror image is also true, and we are not publishing one without the other. On the withdrawn 30-item model-generated corpus, Limato's detector was the only one of the five that flagged nothing at all: 0 of 30 at ≥50%, where the four third-party tools flagged between 19 and 30. A day earlier that was written up as evidence our detector avoided a failure mode the others had. It was nothing of the kind. On text a model actually wrote, our detector missed everything the others caught. At ≥20% on that same corpus it flagged 19 of 30 (63.3%) — so its apparent perfect record lived entirely inside one threshold convention.

Put the two together and the shape is consistent: our detector is the most willing to flag on the current corpus and was the least willing on the old one, which mostly says its scores sit in a narrow middle band that different thresholds cut in different places. It did get the AI side right on the current data — 0 of 15 missed at both thresholds — but so did GPTZero, and every tool here catches unedited model output easily. That is the cheap half of the problem.

One number this page owes you, because it is the one we ship. The ≥50% and ≥20% cuts used throughout this post are analyst conventions, applied identically to all five tools so the columns compare to each other. They are not what our product prints. Limato's detector returns a raw 0–100 score and a verdict, and the verdict boundary is fixed in our API: at 70 and above it says AI, from 30 to 69 mixed, below 30 human. Two of the four non-AI emails counted against us at ≥50% scored 52 and 65 — our own interface calls those mixed. Counted by the label a user actually sees, rather than by a threshold we do not use:

That is a better result and we are not going to pretend it is not. It is also not the exoneration it looks like, for two reasons. First, we cannot extend the same courtesy to the other four: we do not have their shipped verdict boundaries in a form we could apply consistently, so every cross-tool number on this page stays at ≥50% and ours is quoted there too. Comparing our shipped boundary against their re-thresholded scores would be the exact trick this post exists to document. Second, and worse for us: no threshold separates our two sets at all. Our highest-scoring non-AI email scored 78; our lowest-scoring AI email scored 72. The classes overlap, so 1-of-15 is a cut sitting inside that overlap rather than a clean boundary — move it up to 79 and we would clear the false positive at the cost of missing 11 of the 15 AI emails. ZeroGPT, on this corpus, has no overlap to cut into. That is the comparison that matters, and we lose it.

If you use Limato's AI Detector, carry both numbers: on this data it flagged roughly a quarter of non-AI short emails at the ≥50% threshold this post applies to everyone, and 1 of 15 at the ≥70 boundary where it actually prints the word AI. Every caveat we would apply to a competitor's score applies to ours.

What this data does not show

• Anything at scale. 30 items total, 15 per side. Every confidence interval in this post is wide enough to swallow most of the differences between tools.
• A false-positive rate on more than 15 texts. The non-AI set is 15 emails from one author, ten of which began as a model draft he edited. That is a small sample produced through two pipelines, not a cross-section of how people write email.
• Anything about other language backgrounds. Single author, single first language. Whether detectors treat Spanish, Mandarin, Hindi or Arabic L1 English the same way is not measured here and must not be inferred from it.
• The full range of business email. The corpus was planned across 8 intents and collapsed to 4: request (4 items), follow-up (4), apology (4) and cold intro (3). Status updates, declines, scheduling and thanks are absent entirely.
• Detector consistency. One run per text per tool. Detector scores are non-deterministic between runs; a rerun could land differently, and this study cannot say by how much.
• Institutional detectors. Turnitin — the tool most students and academics actually face — has no public-facing UI or API for a study like this, so it is not included.
• Anything past this corpus, these five tools' versions on 2026-08-15, and this run date.

Reproduce it

Every file behind this page is public, in limato-app/limato-public under data/detector-study-2026-08/: sample.json (the 30 item ids with word counts, the detector list and both thresholds), corpus-hw.json (the 15 non-AI items verbatim, each carrying a provenance label recording how it was produced), results.csv (one row per third-party detector per item — 120 rows), limato.jsonl (our own detector's raw score and the verdict it printed for each item — the verdict-boundary table above is a straight count of that verdict field, not a re-thresholding of the scores), and score.js, which recomputes the false-positive, false-negative and coverage tables from those files and carries its own Wilson confidence-interval implementation plus a --selftest flag. Download the four data files into one directory and node score.js reprints every table on this page offline, no dependencies and no network. Only the 15 non-AI items are stored there verbatim; the 15 AI items live in esl-email-2026-08/corpus.json in the same repository, published with the earlier study, and are referenced by id rather than copied.

Two things to know before you run it. score.js does not split on provenance: it prints the n=15 rows only, and those match this page's headline table exactly — the hand and model-draft-hand-edited columns in the robustness check were computed over those two subsets with the same Wilson formula. And limato.jsonl still carries the 30 superseded esl-* rows, deliberately, because this page cites them; they are not in sample.json, so the script prints a run of WARNING: row for unknown item_id lines before the first table rather than scoring rows it was not asked about. The rest of the withdrawn corpus's scores are recoverable the same way, from the esl-* lines in the per-detector zerogpt.jsonl, gptzero.jsonl, quillbot.jsonl and grammarly.jsonl logs in that directory. The collectors that produced those logs are not published; the logs are their verbatim output, and every number on this page is derived from them by the script above.

Reuse. The data files in data/detector-study-2026-08/ are released under CC BY 4.0. Republish the numbers, the tables or the corpus anywhere, including commercially, as long as you credit Limato and link back to this page.

Related measurements from the same corpus family: what a rewrite model actually cuts from non-native English email — measured on the same 30-item corpus withdrawn above — and our comparison with QuillBot as a writing tool rather than a detector. Limato also ships a Humanizer for restructuring AI-sounding drafts; it is a separate tool and was not part of this measurement.