An API key should go to one place: the service it belongs to. In MLflow, it also went somewhere else.
MLflow is a widely used open-source platform that, among other things, records what an AI agent does, step by step, so you can debug it later. These records are called traces. With CrewAI autologging turned on (mlflow.crewai.autolog()), MLflow's record of the agent included the LLM provider's API key: the key the agent uses to call OpenAI, Anthropic or another model provider.
The app never logged the key. The instrumentation did. So the key sat in the MLflow tracking backend, and in any other system those traces were exported to.
I reported it, and MLflow credited the report. It is now CVE-2026-116691.
- Advisory: https://github.com/mlflow/mlflow/security/advisories/GHSA-j7qq-vr8c-xvw3
- NVD: https://nvd.nist.gov/vuln/detail/cve-2026-116691
- Affected: MLflow from 2.19.0. Fixed: MLflow shipped a fix in 3.17.0 (PR #25845).
From the advisory's remediation section:
- Upgrade to
mlflow>=3.17.0. - Rotate any LLM provider key that a CrewAI
LLMobject held whilemlflow.crewai.autolog()was on, on MLflow 2.19.0 through 3.16.x. Upgrading stops new leakage but does not remove keys already written. - Delete or redact the affected traces in the MLflow tracking backend and in any OTLP destination they were exported to.
If you cannot upgrade yet, the advisory's workaround is mlflow.crewai.autolog(disable=True).
I found it with Weir, the open-source tool I'm building around one rule: a key goes to its own service and nowhere else.
To see what a leak report looks like:
pip install weir-scan
weir canary judge --sample
To test your own agent:
pip install weir-scan weir-run
weir-run run --config canary.toml
weir-run plants fake keys, runs your agent's own flows against a local fake model, and looks for those exact values, raw or encoded, in its telemetry, console output, model requests and other outbound requests. You get leak, clean or incomplete, and it shows which planted keys reached an export. A GitHub Action fails the pull request job on a leak or an incomplete run.
What it does not do: it covers only the paths the run exercised, and it misses hashed, split or truncated keys. No AI model decides a verdict, and the CLI itself sends nothing anywhere.
The tools are free and Apache-2.0: https://github.com/ralis-ai/weir
Ralis, the company behind Weir, is looking for design partners: teams running AI agents on AWS. We run the test in your CI, as a free two-week pilot, and nothing leaves your environment. If that's you, get in touch through the repo.
Built with AI assistance; I reproduced the MLflow leak before reporting it.