{"slug": "your-model-s-knowledge-cutoff-is-a-marketing-date", "title": "Your Model's Knowledge Cutoff Is a Marketing Date", "summary": "Probing by Shrivu Shankar found that frontier models' actual knowledge can lag their stated cutoffs by months, with Claude Opus 4.7, 4.8, Sonnet 5, and Fable 5 all losing signal around late December 2025 despite shipping under different names, and Claude Opus 5's factual knowledge tracking January 2026 siblings despite a May 2026 published cutoff. Anthropic now publishes two dates per model—'training data cutoff' and 'reliable knowledge cutoff'—with Sonnet 4.6 trained through January 2026 but rated reliable only through August 2025. Shankar also found Sonnet 5 frequently self-identifies as GPT-4, suggesting pre-training corpora contain prior models' outputs.", "body_md": "[AI](https://sourcefeed.dev/c/ai)Article\n\n# Your Model's Knowledge Cutoff Is a Marketing Date\n\nCheap black-box probing shows what frontier models actually know can lag the model card by months.\n\n[Mariana Souza](https://sourcefeed.dev/u/mariana_souza)\n\nAsk a frontier model what it knows about last December and you're running an archaeology dig, not reading a spec sheet. That's the takeaway from a probing analysis by Shrivu Shankar ([blog.sshh.io](https://blog.sshh.io/p/exploring-claudegpt-knowledge-cutoffs)) that hit the Hacker News front page this week. His method is simple enough to steal: build a dataset of daily facts from Wikipedia, quiz each model with 8-way multiple choice on what happened on a given day, and watch the error rate climb. Where the signal dies is where pre-training actually stopped — no lab cooperation required.\n\nThe results mostly line up with what the labs publish, which is itself useful to know. But the places where they *don't* line up are where the story gets interesting for anyone shipping an LLM into production.\n\n## The same run, wearing different names\n\nShankar's quizzes suggest Claude Opus 4.7 onward — 4.7, 4.8, Sonnet 5, Fable 5 — all lose signal around late December 2025, pointing to a shared pre-training run. [Anthropic's model table](https://platform.claude.com/docs/en/about-claude/models/overview) quietly agrees: all four list a January 2026 training data cutoff. What ships as four differently-named models over six months is, at the knowledge layer, one snapshot of the internet with different post-training on top. On the OpenAI side, the GPT-5.6 family probes out to a checkpoint finishing in late February 2026, matching the February 16, 2026 cutoff in [OpenAI's model catalog](https://developers.openai.com/api/docs/models).\n\nThe anomaly is Claude Opus 5. Anthropic publishes a May 2026 cutoff, but Shankar's probes found its factual knowledge tracking the January 2026 siblings. That's a single-source finding and worth holding loosely — his quiz measures dense world-fact recall, not everything a later data pass might add. But there's a mundane explanation that doesn't require anyone to be lying: a fact has to appear many times in a corpus before a model can reliably recall it, and data collected in the final weeks before a cutoff is thin — the internet hasn't finished writing about those events yet. A continued-pretraining pass over spring 2026 data would legitimately extend the *training data* cutoff without producing the knowledge density of the core run.\n\nAnthropic, to its credit, has already conceded this distinction in public. Its docs now publish two dates per model — \"training data cutoff\" and \"reliable knowledge cutoff\" — and its own table shows how far apart they can drift: Sonnet 4.6 trained on data through January 2026 but is only rated reliable through August 2025. Five months of data the model has technically seen but can't dependably recall. The [Transparency Hub](https://www.anthropic.com/transparency) formalizes the two-date scheme. The single \"knowledge cutoff\" number that model cards have carried since GPT-3 is effectively dead; Anthropic just wrote the obituary first.\n\n## Never ask a model what it is\n\nThe stranger finding: Sonnet 5 frequently self-identifies as GPT-4 under Shankar's 50-phrasing identity probe. His read is that pre-training corpora now contain large volumes of *prior models' outputs* — chat transcripts pasted into forums, blogs, and documentation, and plausibly first-party usage data — so a new model's sense of \"what am I\" is contaminated by every model users talked about during the data window.\n\nWhatever the mechanism, the practical rule is settled: self-report is worthless metadata. OpenAI's community forums spent weeks this year on GPT-5.5 in Codex insisting its cutoff was June 2024 — wildly wrong, and enough to convince users a stealth model swap had happened. GPT-5.6 tells users December 2025 while its documented cutoff is mid-February 2026. If your application branches on a model's claimed identity, version, or cutoff date, you have a bug that hasn't fired yet.\n\n## What to actually do with this\n\nThe developer moves fall out directly:\n\n**Probe, don't trust.** If your product depends on the model knowing recent facts in a specific domain — pricing, APIs, regulations, sports, whatever — Shankar's methodology is a weekend project: 20–50 dated multiple-choice questions from your domain, run per model, per release. That's cheaper than one production incident caused by confidently stale answers.**Budget a reliability gap.** Treat the effective cutoff as the published*reliable*date minus a safety margin, and expect it to vary by domain. One HN commenter found a model that knew about events from April 2025 but whiffed on higher-profile events from January 2025; Shankar saw no systematic coding-vs-world-events split, but per-domain variance is real enough to test for.**Architect for retrieval at the margin.** Anything inside roughly the last six months before a model's release should come from web search tools or your own retrieval layer, not parametric memory. The pre-train-to-ship lag means even a day-one flagship is about half a year stale at the core, and the final months before the stated cutoff are the least trustworthy part of what it \"knows.\"**Pin model IDs and know your surface.** An OpenAI employee confirmed in the HN thread that API model versions stay fixed while ChatGPT-surface models get updated without renaming — \"our goal isn't to be opaque or sneaky, but just to not exhaust people.\" Fine for consumers; for anything you're evaluating or regression-testing, the API with pinned IDs is the only surface with stable semantics.\n\n## The cutoff is becoming an empirical question\n\nThe deeper shift here is who gets to say what a model knows. For years the cutoff was a line on a model card, taken on faith. Cheap black-box probing flips that: knowledge recency is now independently measurable, per domain, by anyone with an eval harness and an afternoon. Anthropic's two-date disclosure suggests the labs see this coming and would rather calibrate expectations than get caught by someone else's chart. Expect cutoff probing to join the standard eval stack alongside capability benchmarks — and until it does, treat every published cutoff the way you'd treat a vendor's uptime claim: probably directionally true, and worth verifying before you build on it.\n\n## Sources & further reading\n\n-\n[Exploring Claude/GPT Knowledge Cutoffs and Pre-Training Timelines](https://blog.sshh.io/p/exploring-claudegpt-knowledge-cutoffs)— blog.sshh.io -\n[Exploring Claude/GPT Knowledge Cutoffs and Pre-Training Timelines - discussion](https://news.ycombinator.com/item?id=49244085)— news.ycombinator.com -\n[Models overview - training data and reliable knowledge cutoffs](https://platform.claude.com/docs/en/about-claude/models/overview)— platform.claude.com -\n[GPT-5.6 model page - knowledge cutoff](https://developers.openai.com/api/docs/models/gpt-5.6)— developers.openai.com\n\n[Mariana Souza](https://sourcefeed.dev/u/mariana_souza)· Senior Editor\n\nMariana covers the fast-moving world of machine learning and generative AI, with a particular focus on how these technologies are reshaping development workflows. When she isn't stress-testing the latest foundation models, she's usually at a local hackathon.\n\n## Discussion 0\n\nNo comments yet\n\nBe the first to weigh in.", "url": "https://wpnews.pro/news/your-model-s-knowledge-cutoff-is-a-marketing-date", "canonical_source": "https://sourcefeed.dev/a/your-models-knowledge-cutoff-is-a-marketing-date", "published_at": "2026-08-11 00:08:38+00:00", "updated_at": "2026-08-11 00:12:15.638543+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research"], "entities": ["Shrivu Shankar", "Anthropic", "OpenAI", "Claude Opus 4.7", "Claude Opus 5", "Sonnet 5", "GPT-5.6", "Hacker News"], "alternates": {"html": "https://wpnews.pro/news/your-model-s-knowledge-cutoff-is-a-marketing-date", "markdown": "https://wpnews.pro/news/your-model-s-knowledge-cutoff-is-a-marketing-date.md", "text": "https://wpnews.pro/news/your-model-s-knowledge-cutoff-is-a-marketing-date.txt", "jsonld": "https://wpnews.pro/news/your-model-s-knowledge-cutoff-is-a-marketing-date.jsonld"}}