I found an API key leak in MLflow, and why your AI agent may have one too

MLflow, the open-source AI agent tracing platform, leaked LLM provider API keys into its tracking backend and any OTLP export destinations when CrewAI autologging was enabled, affecting versions 2.19.0 through 3.16.x; the flaw is tracked as CVE-2026-116691 and was fixed in MLflow 3.17.0 via PR #25845. The researcher, building an open-source key-leak scanner called Weir at company Ralis, reported the issue and MLflow credited the report; the advisory directs users to upgrade to mlflow>=3.17.0, rotate any LLM provider key held by a CrewAI LLM object while mlflow.crewai.autolog() was on, and delete or redact affected traces in the tracking backend and any OTLP destination. Users who cannot upgrade can set mlflow.crewai.autolog(disable=True) as a workaround, since upgrading stops new leakage but does not remove keys already written.

An API key should go to one place: the service it belongs to. In MLflow, it also went somewhere else. MLflow is a widely used open-source platform that, among other things, records what an AI agent does, step by step, so you can debug it later. These records are called traces. With CrewAI autologging turned on mlflow.crewai.autolog , MLflow's record of the agent included the LLM provider's API key: the key the agent uses to call OpenAI, Anthropic or another model provider. The app never logged the key. The instrumentation did. So the key sat in the MLflow tracking backend, and in any other system those traces were exported to. I reported it, and MLflow credited the report. It is now CVE-2026-116691. - Advisory: https://github.com/mlflow/mlflow/security/advisories/GHSA-j7qq-vr8c-xvw3 https://github.com/mlflow/mlflow/security/advisories/GHSA-j7qq-vr8c-xvw3 - NVD: https://nvd.nist.gov/vuln/detail/cve-2026-116691 https://nvd.nist.gov/vuln/detail/cve-2026-116691 - Affected: MLflow from 2.19.0. Fixed: MLflow shipped a fix in 3.17.0 PR 25845 . From the advisory's remediation section: 1. Upgrade to mlflow =3.17.0 . 2. Rotate any LLM provider key that a CrewAI LLM object held while mlflow.crewai.autolog was on, on MLflow 2.19.0 through 3.16.x. Upgrading stops new leakage but does not remove keys already written. 3. Delete or redact the affected traces in the MLflow tracking backend and in any OTLP destination they were exported to. If you cannot upgrade yet, the advisory's workaround is mlflow.crewai.autolog disable=True . I found it with Weir, the open-source tool I'm building around one rule: a key goes to its own service and nowhere else. To see what a leak report looks like: pip install weir-scan weir canary judge --sample To test your own agent: pip install weir-scan weir-run weir-run run --config canary.toml weir-run plants fake keys, runs your agent's own flows against a local fake model, and looks for those exact values, raw or encoded, in its telemetry, console output, model requests and other outbound requests. You get leak, clean or incomplete, and it shows which planted keys reached an export. A GitHub Action fails the pull request job on a leak or an incomplete run. What it does not do: it covers only the paths the run exercised, and it misses hashed, split or truncated keys. No AI model decides a verdict, and the CLI itself sends nothing anywhere. The tools are free and Apache-2.0: https://github.com/ralis-ai/weir https://github.com/ralis-ai/weir Ralis, the company behind Weir, is looking for design partners: teams running AI agents on AWS. We run the test in your CI, as a free two-week pilot, and nothing leaves your environment. If that's you, get in touch through the repo. Built with AI assistance; I reproduced the MLflow leak before reporting it.