cd /news/ai-safety/together-ai-prompt-leak-ai-coding-ag… · home › topics › ai-safety › article
[ARTICLE · art-174026] src=explainx.ai ↗ pub= topic=ai-safety verified=true sentiment=↓ negative

Together AI Prompt Leak: AI Coding Agent Exposed a Hugging Face Credential

Together AI disclosed on November 21, 2026 that an unauthorized party downloaded stored prompt samples after an AI coding agent committed a Hugging Face credential to the company's public xorl GitHub repository. The notice names MiniMax M3, GLM-5.2 and Kimi K2.6 serverless traffic on June 24, July 4, July 8 and July 14, 2026, and says the intruder tested five Dragos canary credentials embedded in those prompts. Together AI reverted the commit after automated scanning detected the credential, made zero data retention the default for new accounts, and told customers to enable ZDR and contact security@together.ai.

by read8 min views2 publishedNov 22, 2026
Together AI Prompt Leak: AI Coding Agent Exposed a Hugging Face Credential
Image: Explainx (auto-discovered)

Together AI disclosed on November 21, 2026 that an unauthorized party downloaded stored prompt samples after an AI coding agent exposed a Hugging Face credential in its public xorl GitHub repository. The official notice names MiniMax M3, GLM-5.2, and Kimi K2.6 serverless traffic on June 24, July 4, July 8, and July 14, 2026, and says five Dragos canary credentials embedded in those prompts were tested by the intruder.

TL;DR: what Together AI confirms #

Item Together AI's statement
Root cause AI coding agent committed HF repo credential during xorl development
Detection Automated scanning found credential; commit reverted
Intruder action Downloaded stored prompt samples during exposure window
Follow-on Searched samples for other credentials; tested five Dragos canaries
Data fate Party understood to have deleted data; no further leak claimed
Customer action Enable ZDR; contact security@together.ai
Scope review Ongoing; direct notice if sensitive data found

This sits in the same week as Dragos canary research and OpenRouter's MiniMax customer emails. The mechanisms differ, but the lesson for builders is the same: prompts and dev credentials are attack surface.

What is zero data retention on Together AI? #

Zero data retention (ZDR) is a Together AI account setting that tells the platform not to retain prompt and completion text for training or logging beyond what operations require. Together AI now says ZDR is the default for new accounts after this incident.

ZDR governs production inference paths. It does not automatically cover engineering datasets copied into private Hugging Face repos for QA. If your compliance story depends on ZDR alone, map every copy of prompt text, including eval harnesses and customer support exports.

Our OpenRouter ZDR explainer thread shows how badges on a marketplace can disappear while investigators work. Together AI is making the opposite move, defaulting new tenants to stricter retention, but existing accounts must still toggle settings per the November 21 notice.

Why Together AI stored prompts at all #

Together AI says the samples were collected as part of routine software testing and kept in a private Hugging Face repository. That is separate from live production routing, but it still held real user prompt text from serverless calls unless testing used synthetic data only. The company does not say how many distinct customers appear in the archive.

Teams that run Together AI agent evals already treat prompts as durable artifacts. This incident shows where those artifacts land matters as much as inference-time ZDR flags.

The xorl repository and the AI coding agent #

Green key sealed inside a capsule beside a closed project folder

xorl is Together AI's public GitHub project tied to its open inference stack. During development, an AI coding agent committed a credential that unlocked the private Hugging Face repo. Together AI's scanning caught the mistake and maintainers reverted the commit, but revert does not revoke a credential that was already copied.

That pattern matches other 2026 supply-chain stories where agents write secrets into repos:

Together AI's response includes PR credential scanning and scoped or short-lived credentials, which are table stakes for any org letting agents touch git.

Dragos canaries in the downloaded prompts #

Vault door with a key hanging beside staged prompt credentials

Together AI explicitly says the intruder tested five canary credentials associated with a Dragos security experiment. That aligns with Dragos CRO HD Moore's November 19 post on two million canary-laced prompts, which we summarized in the Dragos canary article.

Canary tokens are fake AWS keys or URLs that should never be used unless a system was compromised. When they fire, you know someone read prompt text or a provider logged it. Here, Together AI confirms post-download credential hunting, not just passive storage.

Builders running coding agents should seed their own canaries in staging prompts and never paste production API keys into model chats, as Dragos recommends.

Models and dates builders should check #

Together AI lists these serverless models in the stored sample set:

Model Provider relationship
MiniMax M3 Third-party model on Together serverless
GLM-5.2 Z.ai GLM family

| Kimi K2.6 | Moonshot Kimi line Together partners on natively | Dates named in the notice: June 24, July 4, July 8, July 14, 2026. If your team routed customer support logs, code, or credentials through those models on Together serverless on those days, assume the text could be in the downloaded archive until Together AI tells you otherwise.

This is not the same incident as OpenRouter's MiniMax reseller forwarding warning, which covers July–October live traffic on a router marketplace. Read both notices if you use multiple hops.

Investigation and customer notification #

Together AI says it engaged an external forensic firm and law enforcement. It reports no evidence of additional prompt leakage beyond this Hugging Face archive path, but content review is ongoing for personal or customer-sensitive data.

If you need a direct answer for compliance, security@together.ai is the channel named in the post. Document which workspaces used serverless on the listed models in June–July 2026.

Policy changes Together AI announced #

The notice lists four concrete controls:

  1. Scoped or short-lived credentials , plus keyless access where supported.
  2. Credential checks on all pull requests .
  3. ZDR default for new accounts .
  4. Updated development policies for AI coding agents .

Default zero data retention helps forward-looking privacy, but it does not retroactively protect text already stored in a test Hugging Face dataset. Customers must opt in explicitly in settings for existing accounts, per the post.

How to protect yourself #

API key beside an open pocket with leaked credential drops

For inference customers

  • Turn on ZDR in Together AI settings before sendingPII, source code, or secrets .

  • Rotate keys that ever appeared in prompts during the named window.

  • Prefer direct enterprise endpoints when contracts allow, asNear AI's TEE breach post and Dragos both stress for sensitive workloads.

  • Compare your stack to Together prefix-cache guidance only afterdata-handling choices are settled.

For teams using AI coding agents on public repos

  • Block commits that contain HF tokens, AWS keys, or inference API keys ; usepre-commit and CI secret scanners .
  • Never give agents production credentials ; useephemeral tokens per session.
  • Split evaluation datasets intosynthetic-only repos when possible.

For agent-specific guardrails in CI, see Cognition's Corridor acquisition and AgentBeam's write-up of this incident on stopping dangerous agent actions before they leak secrets.

Context: why November 2026 feels like prompt-leak month #

Three threads converged in one week:

Date Event
Nov 19 Dragos publishes canary study; three unnamed provider breaches
Nov 21 AM OpenRouter emails customers about MiniMax routes
Nov 21 PM Together AI publishes xorl / Hugging Face notice

Congressional concern about AI-assisted patch cycles during outages appears in November outage reporting. Prompt leaks are a different failure mode, but both show operational risk when AI touches production paths.

Java4Shell demonstrated AI finding RCE; this week demonstrates AI causing credential exposure. Security programs need both vulnerability review and agent governance.

What Together AI does not claim #

The post does not say every Together customer was affected. It does not name the intruder. It does not publish counts of prompts or customers in the Hugging Face archive. It does not assert HIPAA or GDPR conclusions; review is ongoing.

Treat "data deleted" as Together AI's understanding, not independently verified in the notice itself.

Compliance checklist for security teams #

Security reviewers can work through this list without waiting for a personalized email:

  1. Inventory which products called Togetherserverless (not dedicated endpoints) inJune–July 2026 .
  2. Filter logs forMiniMax M3 ,GLM-5.2 , andKimi K2.6 model strings on the four named dates.
  3. Classify whether prompts could containregulated data ,source code , orlive credentials .
  4. Enable ZDR on active workspaces and document the toggle date.
  5. Open a ticket with**security@together.ai** if regulated data might be in scope.
  6. Rotate any API keys, OAuth tokens, or database passwords that ever appeared inside prompts.
  7. Review xorl forks if your org contributed patches; ensure you never committedHF or cloud tokens .

Pair this with internal canary-token drills modeled on Dragos's public methodology. You do not need two million prompts; a dozen unique canaries per environment is enough to catch unexpected log sinks.

Why AI coding agents multiply secret risk #

Coding agents autocomplete environment files, CI configs, and integration tests. A single bad suggestion that lands on main can expose months of customer prompts stored elsewhere.

Mitigations that match Together AI's post-incident list:

Control What it blocks
Short-lived HF tokens Stolen creds expire before archive download
PR secret scanning Agents cannot merge keys into xorl-like repos
Separate synthetic eval data HF test repos hold fake prompts only
Human review on credential paths Agents never touch secret stores unsupervised

If your workshop team is rolling out Claude Code or Codex broadly, pair tool training with secret hygiene, not just prompt craft. explainx.ai covers live formats in Claude workshop guidance.

Sources: Together AI security notice (November 21, 2026), Dragos canary post (November 19, 2026).

── more in #ai-safety 4 stories · sorted by recency
── more on @together ai 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/together-ai-prompt-l…] indexed:0 read:8min 2026-11-22 · —