The problem with Turnitin's AI detector: what teachers should know

Turnitin launched its AI detection feature in April 2023, positioning it as a natural extension of what the platform already did for plagiarism. For schools already paying for Turnitin, it was a convenient upgrade — one tool for both problems. Two years later, the picture is more complicated. The false positive rate has been higher than many institutions expected, the underlying method has a structural ceiling, and students who know the system well have largely adapted.

What Turnitin's AI detector actually measures

Like all text-based AI detectors, Turnitin's works by scoring how statistically predictable submitted text is. Large language models generate text by repeatedly selecting high-probability continuations — the next word that best fits what came before. Human writers deviate from that pattern in irregular, idiosyncratic ways. A text that closely matches what a model would statistically predict scores high; one with more variation scores low.

The technical terms are 'perplexity' (how surprising the text is to a language model) and 'burstiness' (how much sentence lengths and structures vary). Turnitin measures both and produces a percentage estimate of how much of a submission it considers AI-written.

The false positive problem is well documented

Turnitin itself acknowledges a 1% false positive rate at the 80% confidence threshold — meaning roughly one in every hundred correctly human-written submissions will be flagged. That sounds small until you scale it to a class of 30, or a department of 300. At that volume, false positives become routine rather than exceptional.

The more significant issue is who gets flagged. Writers who use simple, predictable vocabulary — including non-native English speakers and students who have carefully studied a formal academic register — score systematically higher. The 2023 Stanford research that examined seven AI detectors found false positive rates up to 61% for essays by non-native speakers. Turnitin's tool is not exempt from this pattern. The company's own guidance recommends treating results as one signal among many, not as a verdict — which is an honest acknowledgment of the limitation.

Paraphrasing tools defeat detection reliably

Text-based detection and AI text generation are in an ongoing arms race, and the arms race currently has a clear winner. Students who run AI-generated text through a paraphrasing tool — Quillbot is the most commonly used — produce output that reads with far more variation, causing perplexity scores to drop sharply. Independent testing has repeatedly found that Quillbot-paraphrased AI text evades Turnitin's detection reliably.

This creates a structural problem: the submissions that get caught are mostly those where a student pasted AI output directly without editing. That's a subset of actual AI-assisted work — and arguably not the most concerning one. A student who rephrased the output was at least encountering the material.

What Turnitin can't tell you

Turnitin's AI detector answers a narrowly framed question: does this prose look like it came from a language model? It cannot tell you whether the student was present when the work was written, how long the session took, whether the content arrived via a single paste or emerged through drafts and revisions over time.

Those process questions are often the ones that matter most in a fair conversation. A student who pasted 800 words in a single event in the first two minutes of a session has a different story to tell than one who spent forty minutes typing, deleting, and revising before arriving at a polished paragraph. The final text might read similarly; the process that produced it doesn't.

A stronger evidence basis for difficult conversations

If you're using Turnitin and want to address its gaps, adding a process signal is the most direct path. Collecting work through an assignment tool that records the event stream — when typing happened, how much content arrived via paste, what the session looked like overall — gives you evidence that holds up in a formal conversation. 'The submission consisted of 90% pasted content delivered in a single event during a five-minute session' is a statement of observable fact. A Turnitin AI probability percentage is not.

Learnaway is one tool built around this approach. It doesn't read or score the text; it records only the timing and structure of the writing session — paste share, typing rhythm, session length — and makes that visible to the teacher. The analysis is language-neutral, so it doesn't produce the ESL bias that has caused some institutions to suspend text-based tools. It works alongside Turnitin or any submission system rather than as a replacement.

Turnitin's AI detection is worth treating as a weak signal — one that warrants a follow-up conversation, not a formal accusation. For a more defensible standard of evidence, combining it with process data is the approach most likely to hold up when it counts.