Here we go again...bringing the maths to the people

A note before we begin

I've been banging this drum for a long time. Long enough that I'd rather not still be doing it.

In my day job, I work with machine learning and statistics, and from the first wave of AI detectors it was clear to me that the maths didn't support the way these tools were being used, and continue to be used.

Being right about that gives me no pleasure, because the damage is now well documented. There is no joy in saying "I told you so".

Freelance writers have lost work after detectors flagged writing they'd done themselves, and reporting has confirmed cases where a writer was let go because of a score.

Students have been given zeros and referred for disciplinary action over essays they wrote, and one family had to show that the very tool used against their son flagged famous published works as AI-generated to clear his name. Autistic, neurodivergent and non-native English writers report being flagged more often, because plain, formulaic prose is exactly what these tools mistake for a machine. AI Detectors Get It Wrong. Writers Are Being Fired Anyway.

In every one of these cases, a number stood in for evidence and a real person was left to prove a negative.

Public accusations move faster than any fair process can, but by the time evidence is weighed, the damage to a person's name, work, and wellbeing has often already been done. That's why Dapple Grey Press has the policy it has. What follows is the reasoning behind it, in plain English first and then, for anyone who wants it, the maths.

Why we don't rely on AI detectors

Every so often, someone points at a book cover or an illustration and smugly says, "That looks AI-generated." Sometimes they back it up with a screenshot from an "AI detector". It feels scientific and smart.

I've muttered "Jesus wept." more times than I care to admit over exactly this. I've given up wanting friends at the expense of not calling out science-mimicking BS.

So let's have a little chat.

This post explains why we don't treat detector scores, or a gut feeling about how something looks, as evidence. As much as my children go on "vibes", "vibes" don't stand on their own as verifiable or irrefutable proof.

I'm backing this post up with a few recent publications in peer-reviewed literature. I'm a scientist - I don't say anything unless I've got citations to back it up.

The short version

• AI detectors give you a number, and people read that number as a verdict. This is your first red flag!

• Even a detector that sounds very accurate will wrongly accuse a meaningful number of real people. How many depends on things the detector can't tell you. This is another red flag that lawyers will be watching very closely.

• These tools are known to be less reliable for some kinds of writing and art, including the plain, clear style that good children's books often have. If you're writing children's books, you are at higher risk for an unfounded allegation, based on this poor quality models.

• Human eyes aren't reliable here either. In published studies, people were close to chance at telling real from AI-generated faces. Much to my daughters' disappointment, they aren't the accurate vibe AI detectors they think they are.

• The only reliable evidence is how a thing was made: the sketches, the drafts, the layered files. We check those privately and we don't publish them - I don't give you proprietary source code either. THIS IS THE GREENEST FLAG!

  
  

What this means in practice

If you're a reader: an "AI-generated" label from a detector or a commenter is a GUESS, not a finding. It's fair to care about how books are made. But a screenshot with a percentage on it doesn't tell you anything with any amount of confidence.

If you're a creator: you can be wrongly accused, and if you are, you're being asked to prove a negative in public. That's why we don't ask our authors and illustrators to do it, but will seek legal counsel and recourse if necessary. The burden of proof lies with the accuser.

If you're a parent, teacher or librarian: the same reasoning applies to children's work and student writing. A detector flag shouldn't be used to accuse a child or a colleague of anything.

I'll be writing more about that last point soon.

The deliciously gory maths, for those who want it

1. Accuracy figures don't answer the question you're asking

A detector's quoted accuracy is really two numbers: the true positive rate (how much AI content it catches) and the false positive rate (how much human content it wrongly flags). But what you actually want when something is flagged is:

P(AI | flagged) = TPR·π / (TPR·π + FPR·(1 − π))

where π is the base rate, the proportion of works in the pool that really are AI-generated. That's a number nobody knows, and it changes from place to place.

Take a generous detector with TPR = 90% and FPR = 1%:

Share of works that are really AI (π)	Chance a flagged work really is AI
1%	~48%
5%	~83%
20%	~96%

Now imagine 1,000 covers, 50 of them AI-generated (5%). The detector catches about 45 of those 50, and wrongly flags about 10 of the 950 human-made ones. So roughly 1 in 6 of the flagged covers were made by a real person.

If the false positive rate is 5% rather than 1%, it's about 1 in 2 wrong flags. That rate of false positives is a coin toss, which is really concerning.

The false positive rate isn't one fixed number. It varies with style, subject, genre and who made the work. The figure on the readout came from someone else's test set, not from the work in front of you.

2. A score is not a probability

Pay attention kids, because this is important.

A detector output such as "97% AI" is a classifier score, not a calibrated probability. There is typically no error bar, the thresholds are chosen by the vendor, and the test data is rarely public. In my professional opinion, this is sloppy.

Many text detectors are themselves language models, so they are, in effect, a model grading another model's homework with no independent ground truth.

3. The problem gets harder as the models get better

Caption Obvious here.

Detectors are trained against particular generators, and generators change constantly. Paraphrasing or light editing can sharply reduce detection rates. There is also a theoretical limit: Sadasivan et al. (2023) showed that the best possible detector's performance is bounded by:

AUROC ≤ ½ + TV − TV²/2

where TV is the total variation distance between the distribution of AI-written and human-written text. As models become better at imitating human writing, TV shrinks toward zero and the bound approaches ½, which is a coin flip. Short samples make matters worse, because there is simply less signal to work with.

Real-world results agree with the theory.

OpenAI withdrew its own AI text classifier in 2023 because of low accuracy: it correctly identified only about 26% of AI-written text while wrongly flagging about 9% of human-written text.

An independent evaluation of 14 detection tools (Weber-Wulff et al., 2023) found none that were accurate and reliable enough for high-stakes use, and none scored above about 80% overall.

Let's flip the question around: Would you stake your career or open yourself to paying damages if there was a 20% chance you were wrong?

Side quest: AUROC

AUROC is the area under the ROC curve. The ROC curve plots true positive rate against false positive rate as you sweep the classifier's decision threshold from strictest to most lenient. The area under that curve summarises performance across all thresholds in one number between 0 and 1.

The most useful interpretation is probabilistic: AUROC is the probability that a randomly chosen positive example (AI-generated) gets a higher score than a randomly chosen negative one (human-made). So 1.0 is perfect separation, and 0.5 is chance, which is equivalent to the ROC curve being the diagonal.

4. Detectors are biased against some writers

Liang et al. (2023) tested several detectors on essays written by non-native English speakers and found that more than half were misclassified as AI-generated, while essays by native speakers were classified far more accurately.

The reason is that many detectors treat simple, predictable phrasing as a sign of machine writing. But simple, clear language is also exactly what accessible writing aims for, including writing for children and for readers with dyslexia.

In other words, the style we work hardest to make well is the style these tools are most likely to misjudge.

5. Images are even harder

The detectors and your "vibes" assessment don't work.

Image detectors look for faint technical fingerprints left by particular generators. Those fingerprints are easily destroyed by ordinary things: resizing, compression, taking a screenshot, or uploading to a social platform (which re-encodes every image).

Furthermore, plenty of entirely human-made digital art, such as airbrushed gradients, flat vector styles and heavy filters, can look "synthetic" to a classifier.

Human judgement isn't a better fallback. Nightingale and Farid (2022) found that people were at roughly chance level when asked to tell real faces from AI-generated ones. "It just looks like AI" is therefore not a measurement of anything.

Would you stake your career or open yourself to paying damages on a coin toss?

What we do instead

Detection asks, "Does this look like AI?" Provenance establishes, "How was this made?"

Before we acquire a work (written or graphic), we take reasonable steps to establish where it came from. Depending on the work, that can include reviewing drafts, layered and working files, sketches and notes. This is done by people, privately, and the materials stay confidential to the creator and to us. We don't publish them and we won't ask a creator to publish them to satisfy an online accusation. We have also committed to engaging legal counsel on their behalf, should it become necessary.

We don't claim this is perfect. No process is, and AI tools are improving all the time. But we would rather accept the minute risk of missing something than make a false accusation against a real person on the basis of a percentage from a tool that can't tell us what we need to know at an acceptable confidence level.

Our full legal position is set out in our AI, Provenance and Creator Protection Policy.

Results have been drawn from published studies and reflect the state of the research at the time of writing. Detection tools change, and independent evaluations are the best place to check current performance.

References

• Sadasivan, V. S., et al. (2023). Can AI-Generated Text be Reliably Detected? arXiv:2303.11156.

• Liang, W., et al. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7).

• Weber-Wulff, D., et al. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19.

• Nightingale, S. J., & Farid, H. (2022). AI-synthesized faces are indistinguishable from real faces and more trustworthy. PNAS, 119(8).

• OpenAI (2023). New AI classifier for indicating AI-written text (announcement, later updated to note the tool had been withdrawn).