{"slug": "llm-drift-how-to-detect-model-drift-in-ai-agents-including-tool-calling-drift", "title": "LLM drift: how to detect model drift in AI agents, including tool-calling drift", "summary": "A blog post on detecting LLM drift in AI agents recommends running a fixed set of real cases through a model on a schedule and before every upgrade, then comparing four signals against a baseline: the output, which tools the model calls, the arguments it passes, and how many calls it makes. The post cites Stanford and Berkeley research comparing the March and June 2023 versions of GPT-4, in which accuracy on identifying prime numbers fell from 84.0% to 51.1% and the share of generated code that ran as returned fell from 52.0% to 10.0%, with the paper concluding that \"GPT-4's ability to follow user instructions has decreased over time.\" The author, who builds a model-change testing tool called EvalShift, also points to a September 2025 Anthropic postmortem on a serving-stack failure and describes replaying recorded agent traces against another model plus a CI gate.", "body_md": "[← all posts](https://www.evalshift.dev/blog)\n\n# LLM drift: how to detect model drift in AI agents, including tool-calling drift\n\nLLM drift detection for agents: output, tool-selection, argument and call-count drift, silent provider updates vs upgrades, trace replay and a CI gate.\n\nLLM drift is a change in what an application built on a language model does, caused by the model and not by your code. It comes from two places: the provider changes what runs behind a model name you call, or you switch to a new model on purpose. To detect it, run a fixed set of real cases through the model on a schedule and before every upgrade, and compare four things against a baseline: the output, which tools the model calls, the arguments it passes, and how many calls it makes.\n\nFor an agent, the last three matter more than the first. A model that skips the order lookup before issuing a refund can still write \"I've refunded your order\", and every check that reads only the reply will pass it. This post covers the types of drift and how to measure each, the difference between silent provider updates and planned upgrades, how to replay recorded agent traces against another model, and a CI gate you can copy.\n\nA disclosure: I build EvalShift, a tool for testing model changes on agents. It has a short section at the end. Everything before it works without it.\n\n## What LLM drift is, and how it differs from classic model drift\n\nIn classic machine learning, drift has a settled vocabulary. Evidently AI's\n[guide ↗](https://www.evidentlyai.com/ml-in-production/data-drift) defines data drift as \"a shift\nin the distributions of the ML model input features\", concept drift as a change in \"the\nrelationships between input and target variables\", and prediction drift as \"the distribution\nshift in the model outputs\". Detection compares distributions: a Kolmogorov-Smirnov test for\nnumeric features, a chi-square test for categorical ones, or a distance such as the Population\nStability Index.\n\nAll three assume you own the model. The weights only change when you retrain, so drift there means the world moved away from a model that stayed still. With an API model, the model itself can move. That adds a fourth kind, and for most teams it is the one that bites:\n\n| Kind of drift | What changed | Example | Who changes it | \n|---|---|---|---|\n| Data drift | The inputs | Users start asking about a product you launched last week | Your users | \n| Concept drift | What the right answer is | A refund policy changes, and old correct answers become wrong | Your business | \n| Prediction drift | The distribution of outputs | More replies end in a handoff to a human | A symptom of the others | \n| Model drift (LLM) | The model behind the name you call | A provider updates a snapshot, moves an alias or reroutes a retired model | The provider, or you on upgrade day | \n\nThe first three are monitoring problems: you watch production and notice the change.\n[LLM monitoring](https://www.evalshift.dev/blog/llm-monitoring) covers them. The fourth can be tested for directly, because\nyou can send the same inputs to the old and the new model and compare what comes back. That is\nwhat the rest of this post is about.\n\n## Silent provider updates vs planned upgrades\n\nModel drift arrives in two ways, and they need different detection.\n\n**Silent changes** happen to a model name you did not change.\n\n- +**The model was updated in place.** Researchers at Stanford and Berkeley[compared ↗](https://arxiv.org/abs/2307.09009v3) the March and June 2023 versions of GPT-4 on the\nsame tasks. Accuracy on identifying prime numbers fell from 84.0% to 51.1%, and the share of\ngenerated code that ran as returned fell from 52.0% to 10.0%. The paper's summary: \"GPT-4's\nability to follow user instructions has decreased over time\". Providers now offer dated\nsnapshots for this reason, but a name that is not pinned can still change under you.\n- +**The serving stack broke.** In September 2025 Anthropic published a[postmortem ↗](https://www.anthropic.com/engineering/a-postmortem-of-three-recent-issues) of three\ninfrastructure bugs that degraded Claude's responses. One misrouted 0.8% of Sonnet 4 requests\nwhen it was introduced on August 5, and 16% in its worst hour on August 31. Anthropic wrote that\n\"the evaluations we ran simply didn't capture the degradation users were reporting\".\n- +**A tuning change shipped and was rolled back.** OpenAI rolled out a GPT-4o update in ChatGPT on\nApril 24 and 25, 2025, and began rolling it back on April 28 after users found it sycophantic.\nIts[follow-up ↗](https://openai.com/index/expanding-on-sycophancy/) says \"our offline\nevaluations—especially those testing behavior—generally looked good\", and that its offline\nevals \"weren't broad or deep enough to catch sycophantic behavior\". That was the ChatGPT\nproduct, not a pinned API snapshot, but it shows how a change can pass the provider's own tests.\n- +**An alias moved.** Google's[model docs ↗](https://aistudio.google.com/docs/models) say a`latest` alias \"will get hot-swapped with every new release\", with two weeks' notice by email,\nand its changelog records`gemini-pro-latest` switching to`gemini-3-pro-preview` on January 21,\n2026. Anthropic's older aliases move too:`claude-sonnet-4-5`[resolves ↗](https://platform.claude.com/docs/en/about-claude/models/model-ids-and-versions) to\n\"the most recent dated snapshot for that minor version\". From the 4.6 generation on, Anthropic\nsays each model ID, dateless ones included, is a pinned snapshot.\n- +**A retired model was rerouted.** Google's[changelog ↗](https://aistudio.google.com/docs/changelog) says that from December 1, 2026,\nrequests to`gemini-2.5-pro` will automatically route to`gemini-3.8-flash` , and requests to`gemini-2.5-flash` to`gemini-3.5-flash-lite` . Those requests will keep returning 200, and the\nchangelog warns that the breaking changes \"include function calling and`thought_signature` validation\". On October 8, 2026, requests to`gemini-3.5-flash` were already being routed to`gemini-3.6-flash` .\n\n**Planned upgrades** are changes you make: a new model ID in a config file, usually because the old\none is being retired or the new one is cheaper. You know the date, and you can test before it\nships. The [deprecation schedule](https://www.evalshift.dev/blog/llm-model-deprecation-schedule) lists the dates that force\nthese for OpenAI, Anthropic and Google.\n\n|  | Silent change | Planned upgrade | \n|---|---|---|\n| What changes | The model behind a name, or the system serving it | The name itself | \n| Notice | None, or an email about an alias | You choose the date, or a deprecation notice sets it | \n| When you find out | From users, or from a scheduled check | Before release, if you test | \n| What to compare | Today's behavior against a stored baseline | The new model against the current one, on the same inputs | \n| Main defense | Pin snapshots, record the model served, run a check on a schedule | A paired comparison gated in CI | \n\nThe defenses overlap. Pinning a snapshot turns most silent changes into planned ones, because the\nname you call stops moving until you move it. OpenAI's\n[text generation guide ↗](https://developers.openai.com/api/docs/guides/text) recommends pinning\nproduction applications to snapshots such as `gpt-5.5-2026-04-23`, and Google's says \"most\nproduction apps should use a specific stable model\". Pinning does not protect you from serving\nbugs like Anthropic's, and it does not survive the snapshot's retirement date. Anthropic's model\nID docs say as much: if a pinned ID starts behaving differently, \"an infrastructure update is the\nmost likely cause.\"\n\n## The four types of drift in an agent\n\nAn agent's behavior is a trace, not a reply: which tools it called, in what order, with what arguments, and what it said at the end. Each part drifts on its own.\n\n| Type | What changes | Example | How to measure | Needs ground truth? | \n|---|---|---|---|---|\n| Output drift | The final reply | Answers get longer, change format, or state a different fact | Code checks on format, an LLM judge on a criterion, embedding similarity to the old reply | For correctness, yes. For \"did it change\", no | \n| Tool-selection drift | Which tools are called | `issue_refund` without`lookup_order` first | Compare the set of tool names per case with the baseline's | No | \n| Argument drift | The values passed to a tool | `order_id: \"a1042\"` where the baseline passed`\"A-1042\"` | Per-field comparison: exact for IDs and enums, a tolerance for numbers, similarity for free text | No | \n| Call-count drift | How many calls are made | A search repeated five times, or a confirmation step skipped | Count of calls per case, compared with a tolerance | No | \n\nNone of the last three needs a hand-written expected answer to detect a change. The baseline model is the reference: a case drifted if the new model does something different from what the old one did on the same input. Whether the change is a regression is a second question, and the answer depends on whether the old behavior was right. Keep the two apart, or every case where the old model was wrong reads as a regression when the new one fixes it.\n\n### Output drift\n\nMeasure output with the cheapest check that answers the question. A JSON reply either parses and\nmatches the schema or it does not, and that check costs nothing and never disagrees with itself.\nFor prose, an LLM judge with one pass-or-fail criterion per check, compared pairwise with the\nbaseline's reply, tells you whether the new reply is worse.\n[When to trust an LLM judge](https://www.evalshift.dev/blog/when-to-trust-an-llm-judge) covers how to check the judge.\n\nEmbedding similarity between the old and the new reply measures change, not quality. A reworded correct answer scores low. Use it to find what moved, and do not gate on it alone.\n\n### Tool-selection drift\n\nCompare the set of tool names each model called on the same case. A set and not a sequence, because a model that calls the same two tools in parallel in a different order has not drifted in any way that matters. Switch to an ordered comparison only for tools where order is the point, such as a lookup that must come before a write.\n\nThis is the type with the worst failures, because a tool call is an action. A skipped verification step or an extra write does not show up in the reply.\n\nProviders' own migration notes say tool use changes between versions, in both directions:\n\n- +Anthropic's [Sonnet 5 prompting guide ↗](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-sonnet-5) says it \"is more agentic than Claude Sonnet 4.6 by default and reaches for tools … more\nreadily\". Its general[best practices ↗](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices) warn that prompts written to stop older models undertriggering tools may make newer ones\n\"overtrigger\".\n- +OpenAI's [GPT-5 prompting guide ↗](https://developers.openai.com/cookbook/examples/gpt-5/gpt-5_prompting_guide) calls GPT-5 \"thorough and comprehensive when trying to gather context in an agentic\nenvironment\", which means more calls before an answer. Its[GPT-4.1 guide ↗](https://developers.openai.com/cookbook/examples/gpt4-1_prompting_guide) says\nthat model follows instructions \"more closely and more literally than its predecessors\".\n- +OpenAI's [function calling guide ↗](https://developers.openai.com/api/docs/guides/function-calling) notes that one snapshot,`gpt-4.1-nano-2025-04-14` , can repeat a call to the same tool when\nparallel tool calls are on.\n\nA prompt tuned against one model's habits encodes them. When the habits change, the prompt is tuned for a model you no longer run.\n\n### Argument drift\n\nSame tool, different values. Compare field by field, and pick the comparison from the field's type:\n\n- +**IDs, enums, booleans and dates:** exact match after trimming whitespace and case-folding.`A-1042` and`a-1042` are the same order.`A-1042` and`1042` are not.\n- +**Numbers:** a relative tolerance. A refund of 49.99 and one of 50.00 may be fine. One of 499.90\nis not.\n- +**Free text, such as a search query:** a similarity score, reported and not gated. Two different\nwordings of the same query are expected.\n\nSchema enforcement narrows this but does not remove it. OpenAI's function calling guide says\n`strict: true` makes calls \"reliably adhere to the function schema\". That guarantees the shape: an\n`order_id` that is a string. It does not guarantee the value is the right order.\n\n### Call-count drift\n\nCount calls per case and allow a tolerance of one, because models legitimately differ by a call: one fetches two records in one call, the other in two. A difference of three or more, or a case that hits your step limit, is a different strategy. Look at p95 as well as the median, since loops show up in the tail first.\n\n## Measure the noise before you call anything drift\n\nA model sampled twice on the same input does not always return the same thing, so some of what looks like drift is noise. Before trusting a comparison, run the current model against itself: the same cases, twice, with the same settings. The rate at which a model disagrees with itself is your noise floor, and drift is what exceeds it.\n\nThree habits keep the comparison honest:\n\n- +**Pair by case.** Compare each case's new result with the same case's old result, not one\naverage with another. Pairing removes the variation between easy and hard cases.\n- +**Report a rate with its interval.** Three drifted cases out of 40 is 7.5%, and the 95% Wilson\ninterval runs from 2.6% to 19.9%. Twelve out of 400 is 3%, with an interval of 1.7% to 5.2%. The\nfirst says almost nothing.[How many eval cases do you need](https://www.evalshift.dev/blog/how-many-eval-cases-do-you-need) works through sizing a suite so the interval is narrow enough to decide.\n- +**Fix the thresholds before you look.** Decide the budget for each type of drift before the run.\nA threshold chosen after seeing the numbers is chosen to make them pass.\n\n## Replay agent traces against a different model\n\nThe most useful baseline is your own production traffic. The question is how to send a recorded agent run to a different model, given that an agent run is a loop: the model calls a tool, the tool result goes back in, the model calls another tool.\n\nRerunning the whole loop live has two problems. The tools have side effects, so a replay that\nexecutes `issue_refund` issues a refund. And once the new model makes a different call in round\none, everything after it is a different conversation, so the two runs no longer compare.\n\nThe method that works is **teacher-forced replay**:\n\n1. +**Record each production run in full.** The input, the tools offered, every model call, every\ntool call with its arguments, and every tool result. Remove personal data before it is written.\n2. +**Replay round by round.** For round one, send the new model the recorded input and the\nrecorded toolset, and compare its tool calls with the recorded ones.\n3. +**Feed back the recorded results, never live ones.** For round two, send the recorded\nconversation up to that point, including the recorded tool results, and compare again. No tool\nis executed, so the replay has no side effects.\n4. +**Score each round.** Both models see identical context at every round, so each comparison\nisolates one model decision.\n5. +**Stop where the recording ends.** If the new model calls a tool the original run never called,\nthere is no recorded result to feed back. Record that as tool-selection drift in that round, and\ngo no further down that path.\n\nTeacher forcing has a known limit: it tests each decision given the original model's history, not the path the new model would take on its own. A new model that recovers well from its own mistakes, or compounds them, looks the same in a teacher-forced replay. That is the price of a comparison that stays paired and has no side effects, and for detecting drift it is the right trade. For the end-to-end path, run a small set of tasks against a sandbox with fake tools.\n\nReplay also needs the request reproduced, not just the messages. Carry over the generation\nsettings the original call used, such as `tool_choice`, parallel tool calls and response format.\nA replay that drops `tool_choice: required` will report drift that is really a missing setting.\n\n[Build a golden suite from production traffic](https://www.evalshift.dev/blog/build-a-golden-suite-from-production-traffic)\ncovers turning recordings into a suite, and\n[evaluating agent tool calls](https://www.evalshift.dev/blog/evaluating-agent-tool-calls) covers scoring the traces.\n\n## Catch silent drift with a scheduled check\n\nSilent drift has no pull request to hang a check on, so it needs a schedule. Four steps:\n\n1. +**Pin snapshots in production.** Call a snapshot ID, not an alias, so the name you call only\nchanges when you change it.\n2. +**Record the model that answered.** Each provider's response names it. OpenAI's`model` field is\n\"the model used for the chat completion\", and Gemini's`modelVersion` is \"the model version\nused to generate the response\". Chart responses by that value and alert on a new one. OpenAI's`system_fingerprint` , once the usual signal for backend changes, is now marked deprecated.\n3. +**Run a fixed suite on a schedule.** Nightly or weekly, run the same cases through the model you\nserve, and compare the tool calls and outputs with the run stored from the day you last\naccepted it. A drift rate whose interval sits above the noise floor is a change.\n4. +**Run the alias alongside the snapshot.** If you pin`gpt-5.5-2026-04-23` , run the`gpt-5.5` alias on the same schedule. While the two agree, the alias has not moved. When they diverge,\nyou have your upgrade test results before anyone asked you to upgrade.\n\n## A CI gate for drift\n\nBefore a model change merges, the same comparison runs as a gate. The script below takes two JSONL files of recorded traces, one line per case with the tool calls each model made, and fails the job when the drift rate for a check is above budget with 95% confidence. It reports inconclusive when the interval straddles the budget, so a small suite does not pass by default.\n\n``` python\nimport json\nimport math\nimport sys\n\nBUDGET = 0.05  # at most 5% of cases may drift on each check\nID_FIELDS = (\"order_id\", \"customer_id\")  # arguments that must match exactly\n\ndef load(path):\n    with open(path) as f:\n        return {row[\"id\"]: row for row in map(json.loads, f)}\n\ndef wilson(k, n, z=1.96):\n    \"\"\"95% interval for k drifted cases out of n.\"\"\"\n    if n == 0:\n        return 0.0, 1.0\n    p = k / n\n    centre = (p + z * z / (2 * n)) / (1 + z * z / n)\n    half = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / (1 + z * z / n)\n    return max(0.0, centre - half), min(1.0, centre + half)\n\ndef drifted(base, cand):\n    b_names = [c[\"name\"] for c in base[\"calls\"]]\n    c_names = [c[\"name\"] for c in cand[\"calls\"]]\n    b_args = {c[\"name\"]: c[\"args\"] for c in base[\"calls\"]}\n    c_args = {c[\"name\"]: c[\"args\"] for c in cand[\"calls\"]}\n    return {\n        \"tool_selection\": set(b_names) != set(c_names),\n        \"arguments\": any(\n            b_args[tool].get(field) != c_args[tool].get(field)\n            for tool in b_args.keys() & c_args.keys()\n            for field in ID_FIELDS\n        ),\n        \"call_count\": abs(len(b_names) - len(c_names)) > 1,\n    }\n\nbase, cand = load(sys.argv[1]), load(sys.argv[2])\nrows = [drifted(base[i], cand[i]) for i in sorted(base.keys() & cand.keys())]\nfailed = False\nfor check in (\"tool_selection\", \"arguments\", \"call_count\"):\n    k = sum(row[check] for row in rows)\n    lo, hi = wilson(k, len(rows))\n    if lo > BUDGET:\n        verdict, failed = \"FAIL\", True\n    elif hi > BUDGET:\n        verdict = \"inconclusive\"\n    else:\n        verdict = \"pass\"\n    print(f\"{check}: {k}/{len(rows)} drifted, 95% CI {lo:.1%} to {hi:.1%}: {verdict}\")\nsys.exit(1 if failed else 0)\n```\n\nOn 120 cases where the candidate skipped the lookup in 14 and changed an order ID's case in two, it prints:\n\n```\ntool_selection: 14/120 drifted, 95% CI 7.1% to 18.6%: FAIL\narguments: 2/120 drifted, 95% CI 0.5% to 5.9%: inconclusive\ncall_count: 0/120 drifted, 95% CI 0.0% to 3.1%: pass\n```\n\nThe argument check compares raw values, so a case-only difference counts as drift. Normalize IDs before comparing if your system treats them as the same. Wire the script into a workflow that runs on pull requests touching the model config, and on a schedule for silent drift:\n\n```\non:\n  pull_request:\n    paths: [\"agent/models.yaml\", \"prompts/**\"]\n  schedule:\n    - cron: \"0 5 * * 1\"   # Mondays, 05:00 UTC\njobs:\n  drift:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v7\n      - run: python replay.py --suite suite/ --out candidate.jsonl\n        env:\n          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}\n      - run: python drift_gate.py baseline/traces.jsonl candidate.jsonl\n```\n\n`replay.py` stands for your replay harness: whatever runs the suite through the model and writes one\nline per case. [LLM regression testing in CI](https://www.evalshift.dev/blog/llm-regression-testing-in-ci) covers the\npull-request side in more depth.\n\n## Doing this with EvalShift\n\nDisclosure again: I'm the founder. [EvalShift](https://www.evalshift.dev/docs) is built around the planned-upgrade half of\nthis post, and the scheduled check is the same command on a timer.\n\n- +**Recording.** The[capture SDK](https://www.evalshift.dev/docs/sdk-capture) records production agent runs, with every\nmodel call, tool call and tool result, to local disk or a bucket you own. Redaction runs in\nprocess before anything is written.\n- +**Replay.**`evalshift capture sync` promotes captures into a golden suite, and`--rounds all` keeps every tool round. A run then replays teacher-forced: recorded tool results\nare fed back, never live ones, and the recorded`tool_choice` and parallel-call settings are\nreplayed with each request.\n- +**The four types.**`tool_selection` compares the tools each model called, as a set by default.`tool_arguments` compares values field by field, choosing exact, numeric or similarity matching\nfrom the tool's schema.`tool_trace_structure` checks call counts with a tolerance of one,\nparallelism and refusals. Output is covered by schema checks, a pairwise LLM judge and an\nembedding similarity evaluator that measures how far the new reply moved from the current\nmodel's, not whether it is right. The[evaluators docs](https://www.evalshift.dev/docs/evaluators) cover all of them.\n- +**The gate.** A[`migration_policy`](https://www.evalshift.dev/docs/migration-policy) sets a budget per type, including` max_tool_divergence` and`max_tool_argument_drift` . The verdict is pass, conditional pass, fail\nor inconclusive, and inconclusive means the suite was too small to decide. The[GitHub Action](https://www.evalshift.dev/docs/action) runs it on pull requests, and runs under a`schedule:` trigger too,\nwithout the PR comment.\n\nOne command runs the comparison:\n\n```\nevalshift compare --suite-name support --from gpt-5.5-2026-04-23 --to gpt-5.5 --policy-gate\n```\n\nWhat it does not do: watch production. EvalShift has no dashboards and no alerts on live traffic.\nRun a monitoring tool for that, and the [trends](https://www.evalshift.dev/docs/trends) page for how scheduled runs compare\nover time.\n\n## FAQ\n\n**What is LLM drift?** LLM drift is a change in what an application built on a language model\ndoes, caused by the model rather than by your code. It happens when a provider updates the model\nbehind a name you call, moves an alias or reroutes a retired model, and when you upgrade to a new\nmodel yourself.\n\n**What is the difference between model drift and data drift?** Data drift is a change in the\ninputs a model receives. Model drift in the LLM sense is a change in the model itself while the\ninputs stay the same. Classic machine learning models only change when you retrain them, so their\ndrift is mostly data and concept drift. An API model can change underneath you.\n\n**How do you detect model drift in an LLM application?** Run a fixed suite of real cases through\nthe model on a schedule and before every upgrade, and compare each case with a stored baseline:\nthe output, the tools called, the arguments and the number of calls. Report each drift rate with\nits interval, and measure the model against itself first to know the noise floor.\n\n**What is tool-calling drift?** Tool-calling drift is a change in which tools an agent calls, the\narguments it passes or how many calls it makes, after a model update or upgrade. It is often\ninvisible in the final reply: an agent that skips a verification step can write the same answer as\none that ran it.\n\n**How do you replay agent traces against a different model?** Use teacher-forced replay: record\neach production run with its tool results, then send the new model each round's recorded context,\nfeeding back recorded tool results rather than executing tools. Compare each round's tool calls\nwith the recorded ones, and stop where the new model calls a tool the recording has no result for.\n\n**Do LLM providers change models without notice?** Snapshots are meant to stay fixed, but aliases\nmove, retired models get rerouted, and serving bugs change behavior: Anthropic's September 2025\npostmortem described three infrastructure bugs that degraded responses for weeks. Pin snapshots,\nrecord the model named in each response, and run a scheduled check.\n\n## Keep reading\n\n- +[What actually breaks when you switch LLMs](https://www.evalshift.dev/blog/what-breaks-when-you-switch-llms) : the change\nsurface beyond the answer text.\n- +[Evaluating agent tool calls](https://www.evalshift.dev/blog/evaluating-agent-tool-calls) : scoring the trace, not the\nreply.\n- +[LLM monitoring](https://www.evalshift.dev/blog/llm-monitoring) : the production half of drift detection.\n- +[LLM model deprecation schedule](https://www.evalshift.dev/blog/llm-model-deprecation-schedule) : the dates that force\nplanned upgrades.", "url": "https://wpnews.pro/news/llm-drift-how-to-detect-model-drift-in-ai-agents-including-tool-calling-drift", "canonical_source": "https://www.evalshift.dev/blog/llm-drift", "published_at": "2026-11-16 20:51:58.078914+00:00", "updated_at": "2026-11-16 20:52:00.099966+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "large-language-models", "mlops", "ai-safety"], "entities": ["EvalShift", "GPT-4", "Anthropic", "Stanford", "Berkeley", "Evidently AI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/llm-drift-how-to-detect-model-drift-in-ai-agents-including-tool-calling-drift", "markdown": "https://wpnews.pro/news/llm-drift-how-to-detect-model-drift-in-ai-agents-including-tool-calling-drift.md", "text": "https://wpnews.pro/news/llm-drift-how-to-detect-model-drift-in-ai-agents-including-tool-calling-drift.txt", "jsonld": "https://wpnews.pro/news/llm-drift-how-to-detect-model-drift-in-ai-agents-including-tool-calling-drift.jsonld"}}