Detection methodology

The Mechanics Behind the Score

What the detector actually measures, how it turns a passage of text into a probability estimate, and — just as important — where it can be wrong. No black boxes, no invented precision.

Fig. 01 — Foundation

What “AI Detection” Actually Measures

AI detectors do not read meaning. They do not ask whether an idea is original. What they measure is a set of statistical properties that tend to differ between text generated by a language model and text written by a person — specifically, how predictable the word choices are, and how uniform the sentence structure is.

Two concepts underpin almost every serious detection system in use today:

Term	What it measures	Pattern in AI text
Perplexity	How surprised a language model would be by each word choice — whether the next token was the obvious, statistically likely pick, or something less expected.	Language models optimize for high-probability next tokens by design. Their output therefore scores low on perplexity — each word choice was, in a sense, the predictable one.
Burstiness	How much the perplexity (and by extension, sentence length and complexity) varies across a document. Human prose tends to swing between short, punchy moments and longer elaborations. The rhythm is uneven.	AI-generated text tends to stay in a narrow band — sentences of similar length, similar complexity, similar structure. The standard deviation of sentence-level scores is low. Detectors read that evenness as a signal.

Perplexity and burstiness are the field’s foundation, not a proprietary claim. They were popularized by early detectors and remain the conceptual core even in systems that have since moved to deep-learning classifiers. Understanding them is prerequisite to understanding why any score is what it is — and why some scores are wrong.

A third technique worth naming separately is watermarking. This works differently: instead of analyzing text after the fact, some AI providers embed a hidden statistical pattern into their model’s output during generation — a subtle bias toward certain tokens at regular intervals. A watermark-aware tool can verify whether that pattern is present. Watermarking is a provenance technique, not a detection technique, and it requires cooperation from the AI provider. Most current providers do not deploy a public-facing watermark, so detectors analyzing text in the wild cannot rely on it.

Fig. 02 — Process

The Detection Pipeline

Each submission passes through a fixed sequence of analytical stages. The steps below describe the full process from raw input to final output:

Analysis pipeline — per submission

Input & Preprocessing
Entry

The submitted text is normalized — encoding artifacts removed, whitespace standardized. Very short submissions (under roughly 150 words) are flagged at this stage; accuracy of all downstream steps degrades significantly on short text, and the result carries a lower-confidence label to reflect that.

Sentence Segmentation
Structural

The text is split into individual sentences. Sentence boundaries are identified using a combination of punctuation rules and a lightweight language parser that handles edge cases — quotations, abbreviations, parentheticals — without splitting on them incorrectly. All downstream scoring works at the sentence level before being aggregated.

Perplexity Scoring
Signal

Each sentence is passed through a reference language model. The model scores the probability of each token given what preceded it. The aggregate inverse of those probabilities gives a perplexity score per sentence — high values mean the word choices were surprising, low values mean they were predictable. Sentences below a threshold enter a flagged pool.

Burstiness Analysis
Signal

The distribution of sentence-level perplexity scores across the full document is examined. The standard deviation of those scores — the spread — is the burstiness measure. A tight cluster of similar scores signals AI origin; a wide, uneven distribution signals human writing. Both scores together are more informative than either one alone.

Cross-Model Pattern Comparison
Multi-engine

The sentence-level scores are compared against known output profiles for major generative model families — including GPT-series, Claude-family, Gemini-family, Llama-family, and others. Each family has statistically measurable tendencies in vocabulary distribution, hedging language use, paragraph symmetry, and transition phrasing. This stage asks: which model profile does this text most closely resemble?

Classifier Aggregation
Inference

All signals — per-sentence perplexity, document-level burstiness, vocabulary distribution, syntactic patterns, and model-family comparisons — feed into a trained classifier. The classifier was trained on a large corpus of human-written text across varied registers and AI-generated text from multiple model families. It outputs a probability estimate, not a binary label.

Score + Likely-Engine Output
Result

The final result shows an overall AI-probability score, sentence-level highlights indicating which spans drove the score, and — where the cross-model comparison produced a sufficiently confident match — a “likely engine” indicator. Where confidence is too low to name a specific model family, the result shows only the probability score without attribution.

On sentence-level highlights

The per-sentence breakdown is the most actionable output for a writer who wants to understand a result. A document can have a 70% AI-probability score while several of its sentences read as entirely human — and the highlights show exactly which sentences tipped the aggregate. The aggregate is a summary, not a verdict on any single passage.

Fig. 03 — Multi-engine rationale

Why Checking Against Multiple Models Beats a Single Score

The first generation of AI detectors worked against a single model — typically GPT-2 or early GPT-3. A single-model approach generates a single-model false-negative: if the text came from a different family, the detector’s reference distribution is simply wrong, and the signal it’s looking for isn’t there.

Different generative model families have genuinely different stylistic tendencies. These are not superficial differences — they emerge from distinct training data, architectures, fine-tuning approaches, and token-sampling strategies:

Model family	Documented stylistic tendency
GPT-series	Characteristic “smoothness” and balanced paragraph structure; tendency toward symmetric sentence length across paragraphs; specific transitional phrases that recur across samples. GPT-4-class outputs often achieve higher surface variety than earlier versions but still fall within a detectable perplexity band.
Claude-family	Tendency toward ethical hedging and cautious framing; characteristic uses of phrases like “it’s worth noting” and “I should mention”; highly structured responses with a tendency to enumerate even when prose would be more natural. Creative writing output is among the harder-to-detect from this family due to greater stylistic range.
Gemini-family	Sentence-level perplexity patterns closer to GPT-series than to Claude, but distinct vocabulary distribution; identifiable paragraph organization patterns; different hedging vocabulary.
Llama / open-weight	More variable depending on the specific fine-tune and instruction-following configuration; base model output is highly detectable; instruction-tuned variants approach GPT-class detection difficulty.
DeepSeek-family	A pronounced tendency toward numbered lists, parallel constructions, and highly symmetrical paragraph lengths. The stylistic range is narrower than most Western-model families, which makes perplexity and burstiness signals stronger for this family.

Checking a submission against profiles for all these families simultaneously means the analysis is calibrated for whichever model actually produced the text, rather than defaulting to the profile that happens to have been most prevalent in training data. The result is a more reliable probability estimate regardless of which model a user employed.

Fig. 04 — Engine attribution

How Likely-Engine Identification Works

The “likely engine” label on a result is a pattern-match inference, not a cryptographic proof. The field term for this is authorship attribution — applied to LLM outputs rather than human authors — and it draws on the same stylometric principles that forensic linguists have used for decades.

The process works by computing how closely the text’s statistical profile — its perplexity distribution, token-level probability curves, syntactic structures, and vocabulary patterns — matches the known output profile of each model family. When one family’s profile produces a significantly better fit than the others, the result carries a “likely” engine label at a stated confidence level.

Several constraints are worth stating clearly:

Constraint	Why it matters
Models evolve	A model family’s stylistic fingerprint shifts with each major version. Detection patterns trained on GPT-4 may not transfer exactly to GPT-4o or a later variant. Profiles are updated as new output samples become available, but there is always a lag.
Edited output blurs the signal	If a human substantially edits AI output before submission — restructuring paragraphs, rewriting sentences — the stylistic fingerprint degrades. The result will still show elevated AI probability if the editing was light, but the engine label becomes much less reliable.
Models within a family share fingerprints	GPT-3.5 and GPT-4 share enough stylistic traits that attribution at the model-version level is generally not reliable. Attribution at the family level (GPT-series, Claude-family, etc.) is the more defensible claim.
No result below the confidence threshold shows an engine label	When the cross-model comparison does not produce a clear winner — when the scores are too close across multiple families — the result reports only an overall probability. Guessing when the evidence is ambiguous would inflate the false-confidence rate.

Fig. 05 — Limitations

Where This Method Breaks Down

These are not edge cases. They are documented failure modes that apply to AI detection as a field, not just to this tool. Anyone using detection output to make a consequential judgment about a writer should understand them.

Documented limitations — read before acting on any result

No detector achieves 100% accuracy. Every system in this field — including this one — produces both false positives (flagging human writing as AI) and false negatives (missing AI text). The output is a probability estimate. It is evidence, not a verdict. It should not be used as the sole basis for any consequential action against a writer.

False positives are a documented, serious problem for non-native English writers. Research from Stanford (Liang et al., 2023) and subsequent studies found that detectors misclassify over 61% of essays written by non-native English speakers as AI-generated in some test conditions. The mechanism is structural: non-native writers tend to use standard grammatical constructions, simpler vocabulary, and regular sentence patterns — the exact features that low-perplexity detection is calibrated to flag. This bias is not unique to any single detector; it is a consequence of the underlying measurement approach.

Formally written human text can score high. Academic abstracts, legal documents, technical documentation, and boilerplate corporate prose are all written to be predictable — low perplexity is a feature of the genre, not evidence of machine authorship. Detection accuracy degrades on highly formulaic registers regardless of who actually wrote them.

Heavily edited or paraphrased AI text reduces detection confidence substantially. If a person rewrites AI-generated content at the sentence level, varies sentence length deliberately, and replaces characteristic vocabulary, the perplexity and burstiness signals weaken. This is not a flaw in the detection logic — the text has, at some level, been humanized. But it means detection results on heavily post-processed content should be read with lower confidence.

The “likely engine” label is a best-guess inference, not a verified attribution. It reflects which model family’s stylistic profile fits best given the available signals. It does not mean the text was definitively produced by that engine. Prompt settings, system instructions, temperature, and post-generation editing all shift a model’s output away from its default fingerprint.

Very short text (under approximately 150 words) produces unreliable results. Statistical signals that are meaningful across a full document become noisy when the sample is small. Short-text results carry a lower confidence indicator; treat them as directional at best.

Fig. 06 — Interpretation guide

How to Read Your Score Responsibly

The score is a probability estimate. The right mental frame is: “the detector’s signals suggest this text has roughly an X% chance of being AI-generated, given how it compares to the patterns it was trained on.” That framing matters.

Low score
(0–30%)

The text’s statistical profile does not closely match known AI output patterns. Word choices are varied and moderately unpredictable; sentence structure shows meaningful variation across the document. This is consistent with human writing across most registers. A low score does not rule out AI assistance that was heavily revised.

Mid range
(30–70%)

The signals are mixed. Some sentences show low perplexity and uniform structure; others read as distinctly human. This pattern is common in edited AI text, in human writing that happens to be formal or structured, and in mixed-authorship documents. The sentence-level highlights are particularly informative in this range — look at which specific passages are driving the score.

High score
(70–100%)

The text’s perplexity and burstiness patterns, vocabulary distribution, and structural regularities are consistent with AI-generated output across multiple signals simultaneously. A high score is a stronger indicator than a mid-range score, but the documented false-positive rate for non-native writers and formal registers applies here too. High probability is not proof.

If you received a high score on text you wrote yourself: the sentence-level highlights will show you which passages triggered the pattern-match. Text that is deliberately structured — bullet-style argumentation, heavily formatted prose, very regular sentence length — will often score high regardless of authorship. Varying sentence rhythm, introducing less expected word choices, and writing from a first-person perspective with specific examples all tend to lower the measured perplexity and burstiness scores.

If you are evaluating someone else’s text: a high score is a signal worth investigating, not a conclusion. Documentation of the writing process, drafts, and sources is the relevant evidence — the detection score is a starting point for that conversation, not the end of it.

On repeat submissions

Running the same text through detection multiple times does not change the underlying statistical properties of the text, so scores will be consistent across runs. Small variations (under roughly 5 percentage points) can occur from sampling-based components of the classifier — this is normal and does not indicate that the result is unstable.

Apply the methodology

Run a detection analysis

Paste any text and see the perplexity signals, burstiness pattern, and sentence-level breakdown that the methodology above describes.

Check text now