text model
October 19, 2026 Ben Glickenhaus · Katherine Thai · Max Spero · Bradley Emi · Annamieka Aerts
Model Description #
Pangram 4.1 is an AI detection model released by Pangram Labs on October 19, 2026. It is the successor to Pangram 4, with improvements to overall accuracy and performance under uncertainty.
As with Pangram 4, Pangram 4.1 uses technology detailed in the EditLens technical report.
Inputs #
No change from Pangram 4.
Outputs #
Pangram 4.1 returns one additional output, class_probabilities, which indicates how sure the model is of each possible label prediction.
Like Pangram 4, Pangram 4.1 returns a sequence of variable-length segments partitioning the analyzed text. The model makes token-level predictions, which it then decodes into one of three labels for each segment: Human, AI-Assisted, or AI-Generated.
Segment-Level Outputs
Unchanged from Pangram 4: text, label, ai_assistance_score, start_index / end_index, word_count / token_length, humanizer_score, is_humanized.
New in Pangram 4.1: The segment's posterior probabilities for Human, AI-Assisted, and AI-Generated.
Document-Level Outputs
No change from Pangram 4.
Model Resolution
Slightly improved from Pangram 4. You may occasionally see shorter segments in the detection readout.
Supported Languages #
Pangram can work effectively on languages outside the official support set due to generalization. Please contact us to request an evaluation on another language.
Model Training #
No change from Pangram 4.
Model Inference #
No change from Pangram 4.
Evaluations #
Methodology
For evaluation we use the prediction_short field. For binary evaluation datasets, a Mixed prediction counts as an error.
Human Datasets — False Positive Rate by Language
Pangram 4.1's English false positive rate is measured on 781,981 FineWeb examples. From this, we exclude pages that are not natural-language writing, such as spam, advertising and machine-generated listings. We evaluate multilingual false positive rates on pre-2022 subsets of FineWeb2.
Overall False Positive Rate
| Evaluation corpus | N | False positives | False positive rate | 95% confidence interval | Pangram 4 |
|---|---|---|---|---|---|
| FineWeb English, in-scope | 781,981 | 70 | 0.0090% | 0.0071% to 0.0113% | 0.0164% (128) |
| FineWeb2, all 104 evaluated languages | 1,000,000 | 33 | 0.0033% | 0.0023% to 0.0046% | 0.0042% (42) |
FineWeb2 False Positive Rate by Supported Language
| Language | Pangram 4 | Pangram 4.1 |
|---|---|---|
| Arabic | 0.0052% | 0.0026% |
| Chinese | 0.0000% | 0.0000% |
| Czech | 0.0000% | 0.0090% |
| Dutch | 0.0157% | 0.0000% |
| French | 0.0026% | 0.0052% |
| German | 0.0000% | 0.0052% |
| Greek | 0.0090% | 0.0000% |
| Hindi | 0.0090% | 0.0090% |
| Hungarian | 0.0180% | 0.0090% |
| Italian | 0.0000% | 0.0026% |
| Japanese | 0.0026% | 0.0000% |
| Korean | 0.0000% | 0.0052% |
| Persian | 0.0000% | 0.0000% |
| Polish | 0.0052% | 0.0026% |
| Portuguese | 0.0104% | 0.0157% |
| Romanian | 0.0000% | 0.0000% |
| Russian | 0.0104% | 0.0052% |
| Spanish | 0.0078% | 0.0026% |
| Swedish | 0.0000% | 0.0000% |
| Turkish | 0.0052% | 0.0026% |
| Ukrainian | 0.0271% | 0.0090% |
| Urdu | 0.0000% | 0.0000% |
| Vietnamese | 0.0078% | 0.0026% |
AI Datasets — False Negative Rate by Language + Generator Model
The English evaluation contains generations from 26 frontier open-source and closed-weight AI models.
Overall False Negative Rate
| Evaluation corpus | Coverage | N | False negatives | False negative rate | 95% Wilson confidence interval |
|---|---|---|---|---|---|
| English AI generations | 26 generator models | 520,000 | 1,899 | 0.3652% | 0.3492% to 0.3820% |
| Multilingual AI generations | 18 languages | 190,149 | 3,793 | 1.99% | 1.93% to 2.06% |
False Negative Rate by Generator Model
| Company | Generator | Pangram 4 | Pangram 4.1 |
|---|---|---|---|
| Anthropic | Claude Fable 5 | 0.255% | 0.235% |
| Anthropic | Claude Opus 5 | 0.195% | 0.145% |
| Anthropic | Claude Sonnet 5 | 0.140% | 0.130% |
| Anthropic | Claude Opus 4.8 | 0.235% | 0.215% |
| Anthropic | Claude Sonnet 4.6 | 0.215% | 0.200% |
| Anthropic | Claude Haiku 4.5 | 0.130% | 0.160% |
| Anthropic | Claude family aggregate | 0.195% | 0.181% |
| DeepSeek | DeepSeek V4 Flash | 0.455% | 0.475% |
| DeepSeek | DeepSeek V4 Pro | 0.320% | 0.335% |
| DeepSeek | V4 family aggregate | 0.388% | 0.405% |
| Gemini 3.1 Pro Preview | 0.555% | 0.600% | |
| Gemini 3.5 Flash | 0.440% | 0.595% | |
| Gemini family aggregate | 0.498% | 0.598% | |
| Gemma 4 31B IT | 0.430% | 0.500% | |
| Meta | Llama 3.3 70B Instruct | 0.550% | 0.720% |
| Mistral AI | Mistral Medium 3.5 | 0.400% | 0.400% |
| Moonshot AI | Kimi K2.6 | 0.395% | 0.450% |
| NVIDIA | Nemotron 3 Ultra 550B-A55B | 0.310% | 0.290% |
| OpenAI | GPT-5.6 Sol | 0.450% | 0.465% |
| OpenAI | GPT-5.6 Terra | 0.345% | 0.405% |
| OpenAI | GPT-5.5 | 0.345% | 0.330% |
| OpenAI | GPT-5.4 | 0.270% | 0.245% |
| OpenAI | GPT-5.4 Mini | 0.330% | 0.340% |
| OpenAI | GPT-OSS 120B | 0.155% | 0.155% |
| OpenAI | GPT family aggregate | 0.316% | 0.323% |
| Alibaba Cloud | Qwen 3.7 Max | 0.315% | 0.345% |
| Tencent | HY3 Preview | 0.330% | 0.385% |
| Thinking Machines | Inkling | 0.190% | 0.165% |
| xAI | Grok 4.3 | 0.605% | 0.720% |
| Z.ai | GLM 5.2 | 0.470% | 0.490% |
| Overall | All generators | 0.3396% | 0.3652% |
3rd Party Benchmarks
| Benchmark | Evaluation split | N | Pangram 4.1 FPR | Pangram 4.1 FNR |
|---|---|---|---|---|
| Liang et al. (2023): Non-native English | Human-authored non-native English | 91 | 0.000% | — |
| Jabarian and Imas (2025) | Standard, full length | 15,936 | 0.000% | 0.000% | | Van Vassalaer et al. (2026) | Public fully AI-generated subset | 39 | — | 0.000% |
Changes from Pangram 4 #
- +Predictions are more stable under small edits. Across 29,103 paired edits, the incidence of unintended verdict flips decreases from 1.08% in Pangram 4 to 0.15% in Pangram 4.1.
- +Dashboard UI has changed to reflect less certainty for text under 150 words.
Intended Usage and Limitations #
Pangram 4.1 is intended for long-form natural-language writing samples of at least 50 words, written primarily in complete sentences.
There are some kinds of text that we consider out of scope at this time, and for which Pangram can produce less reliable results. These include: code, markup, or Markdown text; tables of contents and reference sections; templated or automated writing; AI translations, such as from Google Translate; instructions, recipes and technical manuals; text dominated by mathematical notation; and figure text like axis labels or tabular data.
Raw text and .docx files are recommended over PDFs when available, as PDF parsing can occasionally introduce unintended artifacts.
Ethics & Safety #
False accusations of AI usage can lead to serious consequences, including reputational damage, emotional trauma, and other undue harm.
We acknowledge that our model has a non-zero error rate and its errors may result in such harms. We commit to continuing to engage with our users and the academic community to educate others on appropriately contextualizing and communicating the results of AI detection software.
We take reports of false positives extremely seriously and work to mitigate their occurrences to the best of our team's ability.
We commit to training and releasing models with the lowest possible false positive rate, and improving our evaluations to test and monitor future model releases for regressions.
Pangram Labs — Model Card for Pangram 4.1