What AI Text Detectors Can and Cannot Tell You

Teachers, editors and hiring managers now meet AI text detectors every week, often built into tools they already use. The tools report a percentage or a label, and it is tempting to read that number as a finding. This page explains what the number is, summarizes what independent research has found about how reliable it is, and sets out a fair way to use a result. It does not rate any product; the principles apply to all of them.

A detector score estimates how the prose reads. It says nothing certain about who wrote it.

How AI text detectors work

Most detectors are classifiers: statistical models trained on large collections of text labelled as human-written or machine-generated. Given a new passage, they estimate how closely it resembles each collection. Many rely on the observation that language models tend to choose likely words, so machine text is often more predictable, word by word, than human text, and more uniform from sentence to sentence. Some newer research systems instead look for a watermark: a hidden statistical pattern that a model deliberately builds into its word choices, which a matching detector can later test for. Watermarks only exist if the model that produced the text was built to add one.

What independent research has found

The published evidence is consistent on three points. First, detectors misclassify human writing, and they do so unevenly: a Stanford study found that essays by non-native English writers were flagged as machine-written far more often than essays by native speakers, largely because simpler vocabulary and structure look predictable. Second, paraphrasing defeats them: researchers showed that running generated text through a rewording step sharply lowers detection rates, and argued that reliable detection becomes harder as models improve. Third, people are not much better: in one widely reported test, scientists reading abstracts could not reliably tell generated ones from real ones.

None of this means a detector result is meaningless. It means a result is one weak signal, with known biases, that needs other evidence before anyone acts on it.

Why a reading record makes the score unnecessary

A detector score is a probability about style, produced after the fact by a tool that never saw the work being done. A reading record is the opposite kind of evidence: it is made while the work happens, by the person doing it, and every part of it can be opened and checked. The record has four parts, and none of them needs special software.

Put the two side by side and the difference is plain. A score says that a passage resembles other passages; it cannot say who typed it, when, or from what reading. A record says which pages were read, which words were quoted, when each draft existed and how it changed. A teacher who has the record does not need the score, and a teacher who has only the score does not have enough to act. The hub on highlights as evidence of reading and sourcing collects the methods, and keeping dated drafts with the revisions highlighted is the one to start with.

The record also settles the case where a detector has flagged honest work, because it can be examined without anyone having to trust a probability. Students who keep one from the first week of a course never have to assemble it in a hurry afterwards. The habits cost little: highlight as you read, date your notes, write in one place, and keep a link to each source.

Are AI detectors accurate?

Not accurate enough to decide who wrote a text on their own. Do AI detectors work? They catch a good share of unedited machine text (74% accuracy on it in the Weber-Wulff study below), but published tests find two weaknesses that matter most: they flag some human writing, and they miss much AI text once it has been edited or paraphrased. How accurate AI detectors are depends on who measured them, on what text, and which error was counted.

Read the rows by the error they count. A false positive rate tells you how often honest work gets flagged; an accuracy or detection rate tells you how often machine text gets caught. Vendor figures are measured by the vendor on its own data, and Turnitin noted its metrics may change. The independent rows also test edited and paraphrased text, which is closer to what teachers and editors actually receive, and their numbers are much lower.

Length matters as well: Turnitin will not score fewer than 300 words, and the developer of one retired 2023 classifier called it very unreliable below 1,000 characters. Checking a single paragraph for AI writing covers what a reader can do when the text is too short for a fair score. When a result is already in hand, a fair process for reviewing suspected AI writing sets out the evidence to gather before anyone acts on it.

Detectors are also built into writing tools. Walter Writes AI, an AI humanizer that supports over 80 languages, includes a built-in AI detector on every plan. It has the same limit as every tool in the table: no detector, Walter's included, can guarantee 100% accuracy, so read its result alongside drafts and sources, and expect other detectors to disagree with it at times.

Do AI detectors work? It depends on the job you give them

The short answer is: for some jobs, sometimes, and never for the job people most want. A detector can be asked to do three different things, and the evidence above, plus the tool-by-tool pages on this site, gives a different answer for each.

The swing for single tools is the part most people miss. In the Weber-Wulff study, GPTZero's results on 9 of 18 human-written documents would have led to a false accusation, yet the much larger RAID benchmark found it flagged almost no human texts; the evidence on GPTZero sets the studies side by side. ZeroGPT showed the opposite pattern in the same 2023 study: no false accusations on the 18 human documents, but 13 of the 36 AI documents missed, as the review of ZeroGPT's accuracy explains. And QuillBot's AI detector found no false positives in one small preprint and flagged 10 of 30 student essays in another.

So the same tool can look excellent or unusable depending on the texts in the test. That is not a flaw in any one study; it is what a probability about style looks like when the writing changes. Short texts make it worse, which is why minimum lengths exist.

Read that way, a detector works as a first filter on long, unedited text, and as a reason to look more closely. It does not work as evidence of authorship on its own, and it works least well on exactly the writing teachers worry about most: drafts that have been edited, and writing by people working in a second language. When a result arrives, the steps in the next section, and a reading record, carry the weight the score cannot.

Using a detector result fairly

1. Treat the score as a reason to look, not a finding

   A high score justifies reading the work more closely. It does not justify a conclusion, a grade penalty or a rejection on its own.

2. Read the passage yourself

   Mark the features that drew the score, using the approach in close reading for machine-writing habits, and look just as hard for specific detail only the writer could know.

3. Ask for the process

   Drafts, notes, version history and sources settle most questions quickly. The page on reviewing authorship through drafts and sources describes the conversation.

4. Record what you relied on

   If a decision follows, write down the evidence other than the score. If the only evidence is the score, there is no decision to make.

If a detector flags your own work

Stay calm and bring your process: dated drafts, the sources you read with your highlights, your notes, and your document's version history. Ask which passages were flagged and explain how you wrote them. Point out, politely, that published research documents false positives, especially for non-native writers. A clear record of your process is the strongest answer to any score.

Expert tips for anyone handed a score

• Ask what the tool measured. A percentage with no explanation of the method is a number without a meaning. If the vendor's documentation does not say what the score represents, treat it as unexplained.
• Ask for the record before the conversation. Drafts, marked sources and notes can be read in ten minutes and answer most questions; a conversation that starts from a score alone starts from nothing.
• Never act on a single passage. Formulaic prose is common in introductions, summaries and any writing by someone working in a second language. Look at the whole piece and at the sources behind it.
• Write down the evidence other than the score. If a decision follows, the file should show what it rested on. If the only entry is the score, the decision does not stand.
• Keep your own record too. If you write for publication, keep marked sources and dated drafts of every piece, so that a flag, whenever it arrives, is answered by opening a folder rather than by argument. Exporting highlights and notes so they can be handed over explains what the folder should contain.

Where this fits on the site

This site builds highlighting tools, not detectors, and the reading skills here are the part of the problem a detector cannot do for you. The tools and extensions section covers the highlighters and annotation apps that make close reading and process records practical, including annotation layers for shared reading and highlight links you can send as evidence.

For the detectors students meet most often, the guide to Turnitin's AI writing indicator sets out its documented limits, and reading an AI detection score explains why no percentage is a safe line. Watermarking is a different approach to the same question; whether ChatGPT text carries a watermark covers what has been announced. Teachers choosing a tool can start from AI detectors ranked for classroom use. Tool by tool, the evidence is uneven: read GPTZero's claims against the studies, ZeroGPT's published results and Copyleaks' own testing method before you rely on one score.