We tracked AI key theft over 35 days. Here's what we learned A 35-day honeypot experiment using two fake LLM gateways and more than 40 fabricated organizations with GitHub repositories drew over 3 million requests, of which 9,375 prompt requests used keys matched to 179 documented leaks, according to the researchers who ran it. Of the 179 published keys, outside requests used 65 to fetch the model list and 18 to send prompts, and requests using 16 keys returned HTTP success; in 82 IP/key pairs the model-list check preceded the first successful prompt, with a median gap of 37 seconds. The researchers recorded 1,605,260 saved request captures and 1,417,507 captures from prompt endpoints, and identified some traffic as reseller/gateway activity tied to harnesses such as OMP and Hermes, Telegram agents and black-market reseller domains. In order to discover what would happen we created two honeypot LLM gateways: one mirroring LiteLLM and another one mirroring a generic OpenAI-compatible endpoint. We then proceeded to create over 40 individual organizations, each with at least on Github repository in a way that was convincing to threat actors. API keys along with the addresses of the gateways were added to these repositories in such a way that it looked like they were added by mistake by AI or human error. From this we observed over 3 million requests to our gateways, most of which were probing attacks and some of which ended up using inference in a pattern that we identified as reseller/gateway activity. Some connected to harnesses like OMP or Hermes, others connected to Telegram agents and some were even directly sourced from black market reseller domains directly. The experiment How the keys were exposed For realism we initially created GitHub projects spanning a wide array of types: ticket triage, document indexing, agent tools, and other AI applications. To make our accidental leaks seem realistic, we included a working key and our gateway’s URL in code examples in the repositories. In order to make our honeypot more attractive we advertised multiple SOTA models including astra, fable, opus 5 and sol. The attacker prompts were routed to hosted open-weight models in order to control costs while providing a believable attack surface. We also included guardrails that ensured model specific questions wouldn't reveal the true identity of our model and instead provide a facade that they were actually interacting with a SOTA frontier model from a company gateway. NYC LiteLLM-style gateway 36 days SFO Separate model router 9 days | How long the keys had been public at the snapshot cutoff | | |---|---| | Exposure time | Keys | |---|---| | Less than a day | 19 | | One to seven days | 91 | | More than a week | 64 | | No precise publication time | 5 | What we tracked To ensure that we could extract as much from our honeypot as possbile we recorded as much signal las possible including: the key used, headers, referrers, ip, geoip, route, model, prompts, http status and more. We also saved every prompt and its response for further analysis. The snapshot contains traffic from both gateways. Here is what the three counts mean: - 1,605,260 saved request captures across all endpoints. - 1,417,507 captures from prompt endpoints , including rejected requests. - 9,375 prompt requests using keys matched to our 179 documented leaks , counted in the request logs. The 9,375 figure is the documented-leak subset, not the total prompt volume. Most captured traffic used credentials we could not match to those publication records. Published-key use Which leaked keys were used? In total we published 179 keys. Outside requests used 65 to fetch the model list and 18 to send prompts. Requests using 16 keys returned HTTP success. We compared the projects and leak locations to investigate the difference. 9,375 prompt requests using our published keys 4,711 prompt requests returned HTTP success 80 recorded source addresses behind those successes A model-list request checked the advertised catalog; a prompt request tried to use a model. In 82 IP/key pairs, the list check preceded the first successful prompt. The median gap was 37 seconds , measured between API requests, not from publication. Leak location and project type Which leak locations drew requests We compared the projects, the files containing the keys, and the time each key had been public. The table covers three batches of 40 repositories. To give each key the same observation period, we counted only its first 24 hours after release. | Keys checked and used during their first day | | | | |---|---|---|---| | Repository batch | Keys released | Keys checked | Keys used for prompts | |---|---|---|---| | Early developer projects · NYC | 40 | 38 | 7 | | Utility projects · NYC | 40 | 12 | 4 | | Utility projects · SFO | 40 | 5 | 1 | In the early batch, one source checked 37 keys within a minute of each release. These were model-list requests. The entire batch produced only seven prompt requests, all from another source repeating the same message. Broad collection did not lead to sustained use of every key it found. The two utility batches let us hold several things constant. All 80 projects were Python tools, advertised gpt-6-sol , and contained a key beside the gateway URL in config/development.env . Yet only 17 keys were checked and five were used for prompts. The other 63 never appeared in our gateway logs during more than four days of observation. The four New York utility keys used for prompts came from api-response-code-map , http-header-safety-check , pagination-field-check , and operation-id-audit . San Francisco’s came from cors-wildcard-credentials-check . They were all API-related tools. These projects differed in release order and organization, so we cannot tell whether their subject matter drove the selection. File format may matter too. Six of the early batch’s seven prompt-tested keys came from Go projects with JSON configuration; one came from a Node project, and none from its 14 Python projects. All seven prompt requests came from one source. Language and file format changed together, so we cannot tell which affected that source’s choice. We also tried other leak locations. A GitHub Gist key received eight prompt requests. Keys in two documentation pages and two downloadable starter archives received no gateway requests over roughly six days. One of six keys exposed in commit history was checked and used for prompt attempts. Those small groups show which surfaces produced observable use, but not which were scanned. The newest keys had less time to be found: 19 had been public for less than a day. But 91 keys had at least three days of exposure without a request. All 64 keys whose first request we could time were checked within six hours of release. We did not retain repository crawl or download telemetry, so we cannot distinguish a key that was never collected from one that was collected and ignored. Once a few keys were adopted, their repeated use dominated the traffic. Six keys accounted for 98.9% of prompt requests using our leaked credentials. Repeated use of these keys drove the request volume. Requests with other credentials The arithmetic question behind most captured prompts A request can carry any string as its API key. We recorded those attempts even when authentication failed, so a saved prompt does not necessarily mean the sender had a working key. Many attackers tried to brute force over 100+ admin keys that didn't work. This relates to an original vulnerability with LiteLLM where many gateways shipped with a default admin key What the repeated credential tests looked like Prompt requests carried 93,100 nonempty credential values outside our publication records. Of those, 78,130 were only 1–16 characters long. 93.7% of prompt requests outside the publication match returned HTTP 401 . Almost all of these attempts carried curl-labelled headers. The clearest repeated pattern was this message: Exact submitted message What is 483 927+1145? Answer with only the final number. We captured 1,401,673 copies across 92,596 supplied credential values and 20 requested model names. None of those values matched our documented leaks. Replaying a fixed question while changing the token and model is consistent with automated credential and model testing. Repeated use of our leaked keys A key check described how a relay collected API keys This source made 1,215 prompt requests using our leaked keys. Within that traffic, we recorded 21 copies of one exact message. We’re highlighting it because it described how a relay collected and managed API keys. It began: “The relay pool keeps a small SQLite database of API keys collected from public channels.” The prompt described a “relay pool”: keys and API connections managed by a gateway. The system would retry keys that had hit their quota once an hour, test all the keys every six hours, and update the gateway’s routing each day. It would choose connections based on throughput, response time, and account balance, and could block collection sources by chat title. After that description, the only instruction to the model was Ответь одним словом: OK —“Answer with one word: OK.” The requests also carried the software label keypool in their user-agent header. 21 copies of this exact message 5 leaked keys used for these checks 3 model names requested Of those requests, 12 returned HTTP 200 , 7 returned 429 , and 2 returned 404 . This is the system described in the submitted prompt; we did not inspect the collector’s code. Gateway connections Following the traffic to an API relay my.git.christmas resolved to an IP that was sending prompts to our API. Requests from that IP also mentioned the domain. Veridrop lists it as an OpenAI-compatible API relay https://veridrop.org/leaderboard/my.git.christmas?src=directory . Requests from the same IP referred to a fo2api deployment. Together, the DNS record and prompts linked the source to relay infrastructure, though they didn’t tell us how many users were behind it. | Evidence linking the recorded address to a relay | | |---|---| | Observation | Evidence | |---|---| | Requests to our API | 618 HTTP-successful prompt requests used our published keys. | | DNS match | my.git.christmas resolved to the same address recorded in those requests when we checked. | | Submitted context | Requests mentioned the domain and work on a fo2api deployment. | Relay development and testing Troubleshooting a relay through our gateway The sender also used our gateway to troubleshoot fo2api. One prompt asked a coding agent to fix a production JSON tool-schema error affecting ChatCompletions, Responses, and Anthropic routes, and pointed it to specific backend files to inspect. From a chat export into a gateway’s key pool Earlier turns asked the agent to check keys from a chat export, find ones that could serve StepFun models, and add the working ones to a Sub2API-based gateway. NaraRouter’s https://router.bynara.id/ endpoint appeared in the export and in the testing output. A generation probe returned 402 , and a later account listing marked that connection as an error. The user then supplied an OpenLux https://doc.openlux.ai/en endpoint and key, asking the agent to check it and add it to the API. The following tool output included model tests, retries, and routing mappings. A later listing showed the OpenLux account as active. An APIMart https://apimart.ai/ key came next, along with a request for auto-glm , auto-kimi , auto-qwen , and auto-stepfun : model names that would route requests between the best working models in each family. The APIMart test output reported 402 errors for insufficient balance, and its account was listed as an error. A separate history from another IP included the CLIProxyAPI README https://github.com/router-for-me/CLIProxyAPI . That task was to configure a coding agent to send service tier: "fast" through the proxy. OpenLux and APIMart also appeared in the README’s sponsor list; those links were incidental to the documentation lookup. These were instructions and tool results inside submitted histories, not calls from those providers to our gateway. Telegram bot traffic What Telegram histories revealed about the traffic Telegram message metadata and earlier replies appeared in 88 captured requests from three recorded source addresses, including the source linked to my.git.christmas. Another of those addresses was in Helsinki on Xorek-labelled hosting, in the AEZA network. It sent 31 of the captures containing Telegram conversation metadata. The submitted histories included 12 distinct sender IDs marked as non-bot accounts. This is the same source discussed in the hosting section hosting-infrastructure ; the location describes its server, not the people in the conversations. The reply history highlighted telegram bot named @obed2021bot https://t.me/obed2021bot , with the same account ID across all three sources. In its public profile it is called “Обед” and advertises GPT 6 Astra Max along with free access to whomever has access. The message metadata from this bots requests also included two fields: guest query id and guest bot caller user : Telegram fields for guest-mode bot interactions https://core.telegram.org/bots/api message . Those fields give us enough evidence that we can say that Telegram conversations were being passed through an inference gateway to our system. After removing duplicate message metadata, we found 29 distinct sender IDs marked as non-bot accounts. They appeared in 16 private chats and 10 supergroups. These are IDs in the conversation histories, not verified users. We have omitted personal identifiers. Selected fields; personal identifiers omitted { "format": "telegram message context/v1", "message": { "reply to message": { "from": { "is bot": true, "username": "obed2021bot" } } } } We couldn’t verify who operated the bot or whether our replies reached Telegram. Browser testing Testing our leaked keys from a browser 6 requests using our published keys carried a www.hvoyai.com referrer. Each returned HTTP 200 and used the default message in Hvoy’s current tester https://www.hvoyai.com/api-key-tester/ : “Calculate 17 + 26. Reply with only the number.” The tester takes an endpoint, key, and model name, then sends the request directly from the visitor’s browser. The matching prompt and browser headers point to visitor-side testing, not a request from Hvoy’s servers. We checked the current implementation; we don’t have a historical copy. Following the test requests A browser test led back to VPN infrastructure Other requests from the IP behind the Hvoy tests included instructions for maintaining updates.listokdoma.ru . When we checked DNS, that domain and related subdomains pointed to the same IP. This links the recorded address to the VPN configuration in the prompts; it doesn’t link Hvoy to the VPN. | What we found at the recorded source address | | |---|---| | Clue | Recorded evidence | |---|---| | Domain match | listokdoma.ru , manage.listokdoma.ru , and updates.listokdoma.ru resolved to the recorded IP when we checked. | | VPN maintenance | 7 captured requests included system context naming updates.listokdoma.ru alongside Xray/3x-ui VPN setup and Nginx reverse-proxy maintenance. 3x-ui manages Xray proxies https://github.com/MHSanaei/3x-ui . | | Coding agent | Requests carried omp/18.3.1 headers and context describing Oh My Pi https://github.com/can1357/oh-my-pi configuration and model roles. | | Session handling | Submitted instructions described skill and memory services that inject user id , team id , and agent id from a session. | | Request tools | Other requests from the same address carried browser and Python headers. | Using our published keys, the same recorded address made 155 prompt requests; 132 returned HTTP success. Asked to review a coding agent’s actions Some prompts asked for a safety score for a coding-agent action, formatted as