# Reverse Engineering GPTZero: Verdicts are Document-Level, Non-Additive, and Order-Dependent > Source: > Published: 2026-11-01 06:17:55+00:00 Over the past month building [Penned](https://penned-ai.vercel.app)—a free AI humanizer and detector designed to rewrite AI drafts into natural prose while strictly preserving facts, names, and numbers—we ran into a fascinating, counter-intuitive reality when evaluating our candidate picker against **GPTZero v3 API (`version 2026-10-07-base`)**. Most people (and even many ML practitioners) imagine AI text detectors as sentence-level statistical classifiers: each sentence has a perplexity or burstiness score, and the document score is simply an average or a cumulative tally. Our empirical benchmarks on research paper drafts revealed something completely different: **GPTZero's verdicts are document-level, non-additive, and heavily order-dependent.** Here are 7 empirical findings from our evaluation logs (`eval/gptzero/FINDINGS.md`), and what they mean for anyone building or evaluating NLP systems today. Near the 0.5 decision threshold, GPTZero exhibits an extreme non-linear cliff. Adding just **one extra paragraph** or even a **single table caption** flipped an entire document's classification from `human` to `ai`. Once that threshold was crossed, the API highlighted *every single sentence* as AI-generated—including sections that scored 0% AI when evaluated in isolation. The detector is not simply aggregating independent sentence probabilities; it appears to classify the whole document context jointly, and then distributes retroactive confidence across the text. The most striking behavior was in section combinations: Even wilder was the sensitivity to ordering. We took the results section of an academic paper: Furthermore, swapping just *one paragraph* (section 4.3) for an alternative Penned-generated candidate flipped the entire real-order paper half from **100% AI down to 40% AI (human)**. This demonstrates that document-level classification is governed by non-linear transition dynamics rather than bag-of-words or simple token perplexity. When inspecting GPTZero's `generated_prob` at the sentence level, the scores are not smooth. GPTZero internally segments documents into structural blocks. Even in a document overall classified as human (e.g. 39% AI), individual AI segments remain clearly visible: The global document score appears to weigh these segmented blocks against each other rather than averaging raw sentence scores. Numerical specificity has an outsized effect on detection: `42.8%` with `N%`) turned a AI detectors heavily associate dense, concrete domain measurements and precise numerical observations with authentic human scientific writing. How text is pasted or extracted matters significantly: Summarizing findings is inherently dangerous under modern detectors: Because LLMs are trained to summarize and synthesize, detectors have learned that synthetic summaries look overwhelmingly "machine-like", even when written by humans. We benchmarked several in-house and open-source detection models against the exact same test suite: In short: local open-source detectors cannot see what proprietary v3 detectors see. When we started building [Penned](https://penned-ai.vercel.app), our initial pipeline scored rewritten candidates paragraph-by-paragraph using local DeBERTa and a logistic regression GPT-2 feature checker. These findings showed us that a **greedy per-paragraph picker cannot work**. Because the detector's decision boundary is non-additive and order-dependent: We are currently building open benchmarks and testing these calibration pipelines in public. *Penned is free to test at [penned-ai.vercel.app](https://penned-ai.vercel.app). Check it out and let us know your thoughts on our findings!*