Railway's November 10 incident report shows an attacker walking from one build job to decrypted customer secrets in under a day, and going for AI keys first. We lay the four root causes out as one chain, show why secrets turned straight into AI bills, and list where each AI provider stands on refunds.
This explains reporting by
Railway: Incident Report, October Security Incident (November 10, 2026). Read the original first:
https://blog.railway.com/p/incident-report-october-security-incident
In one minute #
- An attacker escaped Railway's build isolation on October 20 and was fully evicted on October 21 at 09:24 UTC, per Railway's timeline.
- In between, the attacker read 353 build histories, got a cloud credential through a misconfigured OIDC binding, and decrypted environment variables for at most 1.85% of workspaces, about 100,000. Railway calls that an upper bound.
- Railway says the attacker queried the variables database for AI provider keys first, and estimates the unauthorized AI spend at about $740,000.
- Refunds differ by provider: AWS Bedrock and Google Gemini have started refunds, OpenAI and SpaceXSI are case by case, Anthropic is still under discussion, and OpenRouter refuses. Railway says it covers what providers will not.
- If you ran anything on Railway in October, rotate every AI provider key and cloud secret it held, and check provider bills for the October 20 to 22 window.
The chain, step by step #
Railway's report lists four root causes. Read in order, they are one path from a single build job to customer secrets:
- A partial filesystem isolation escape in the multi-tenant build system. Railway timestamps it at 07:28 UTC on October 20, and says the attacker got a build credential from it.
- That build credential was accepted as a trusted client by an internal Temporal instance (a workflow engine), which exposed other builds' stored histories. Railway says 353 build histories were read between 13:50 and 15:30 UTC.
- An erroneously defined OIDC binding in a cloud environment. At 03:26 UTC on October 21 the attacker used it to get a short-lived cloud credential.
- Long-lived credentials on an orchestration machine that could decrypt encrypted variables. Between 04:42 and 07:15 UTC the attacker decrypted some per-environment keys.
No single step was the whole breach. A build escape alone gives one credential. It took the over-trusted internal service, the wrong OIDC binding and the decryption keys sitting on a machine to turn that into customer secrets.
Why secrets became AI bills so fast #
Railway says that between 04:07 and 05:21 UTC on October 21 the attacker queried the variables database and prioritized AI provider keys. Some queries also looked for cloud, database and payment credentials.
The reason is simple economics. A stolen AI key turns into value in minutes: the thief calls the model and the victim pays the bill. A stolen database password needs more work and more risk. Railway believes the attacker wanted AI credentials for resale or for model distillation, which means using a strong model's answers to train a cheaper one.
Environment variables are where most apps keep these keys, and a PaaS stores them for you. That is convenient, and it means the platform's decryption path is part of your threat model. Railway says its estimate of unauthorized AI spend is about $740,000.
Railway also reset linked-account tokens for 1,852 users. Those include bring-your-own agent keys and GitHub OAuth tokens.
Who is refunding what #
Railway says it notified GitHub, GitLab, Anthropic, OpenAI, OpenRouter, SpaceXSI, Google and Stripe to revoke keys that may have been exposed. On refunds, the report says:
- AWS Bedrock and Google Gemini: existing fraud refund policies apply, and refunds have started.
- OpenAI and SpaceXSI: case by case, through a shared channel with Railway support.
- Anthropic: still under discussion. Railway is covering the costs in the meantime.
- OpenRouter: a no-refund policy. Railway is covering the costs.
Railway says it will cover charges providers decline to refund. Customers report unauthorized charges through the form in the report or through station.railway.com.
What to do if you deployed on Railway #
Railway's own list: report unauthorized charges, rotate credentials and re-authenticate, redeploy your latest commit, reconnect GitHub, and recreate CLI and CI credentials. Railway notes that commits pushed while builds were d were not deployed.
Our additions, based on what the attacker went after: rotate every AI provider key stored in a Railway variable even if you got no notice, because the 1.85% figure is per environment, not per secret. Set a spend limit or budget alert on each provider account. Look at provider usage logs for October 20 to 22, the window between the first escape and the session reset.
Rotate first: AI provider keys (OpenAI, Anthropic, Gemini, OpenRouter, Bedrock)
Then: cloud keys, database URLs, Stripe keys, GitHub tokens
Check: provider usage and billing for 2026-10-20 to 2026-10-22
Lessons for anyone running builds or storing keys #
Railway's fixes double as a checklist. It says it moved to per-build ephemeral virtual machines with narrowly scoped registry-push credentials, rebuilt CI with 2FA approval and SIEM alerting, switched internal access to short-lived just-in-time credentials, and limited decryption to the secrets service with central audit logging of every encrypt and decrypt.
Each one breaks a link in the chain above. A fresh VM per build makes an escape worth less. Scoped credentials stop a build token from being trusted by an internal service. Short-lived access limits what a stolen credential is worth. And if only the secrets service can decrypt, a long-lived key on an orchestration machine no longer exists to steal.
For teams that only store keys on a platform, the lesson is smaller and cheaper: give each environment its own provider key, cap its spend, and know how to rotate it in minutes.
Who is affected #
| Case | Status |
|---|---|
| Railway workspaces with environment variables | Up to 1.85% of workspaces (about 100,000) may have had variables decrypted. Railway calls this an upper bound. |
| Railway users with linked accounts | 1,852 users had linked-account tokens (BYOK agent keys, GitHub OAuth) reset and were notified. |
| Customer Git repositories and databases | Railway says it found no evidence they were downloaded or accessed (per BleepingComputer's reading of the report). |
What to do #
- Rotate every AI provider key and cloud secret stored in Railway variables.
- Check AI provider bills and usage logs for October 20 to 22.
- Report unauthorized charges through Railway's form or station.railway.com.
- Redeploy your latest commit and recreate CLI and CI credentials.
- Give each environment its own capped AI key, wherever you host.
What is still unknown #
- Railway has not said how many workspaces actually had secrets used, only the 1.85% upper bound.
- Who the attacker is has not been stated. Railway filed an FBI IC3 complaint and says IOCs will be shared after law enforcement signs off.
- The $740,000 figure is Railway's own estimate; no provider has published its share.
- BleepingComputer reports that Wiz links this to a wider campaign it calls Black October. We did not read Wiz's own report, so we do not repeat its details.
Sources #
- Railway: Incident Report, October Security Incident (November 10, 2026) — the original report
- BleepingComputer: Railway breach leads to $740,000 in fraudulent AI charges
AI News Report· every headline, every morning.