{"slug": "findings-from-the-1m-vercel-sandbox-challenge", "title": "Findings from the $1M Vercel Sandbox challenge", "summary": "Vercel paid out $242,000 in bounties across 1,285 reports during its two-week $1M Vercel Sandbox hacker challenge, which surfaced one Linux kernel zero-day vulnerability that allowed host memory leaks, host crashes, and host code execution from inside a microVM. Researcher Paulos Yibelo of pwn.ai demonstrated the memory-leak attack against Vercel production sandboxes and the crash and code-execution attacks on his own KVM and Firecracker machines matching Vercel Sandbox configuration, prompting Vercel to work with AWS and other industry partners on a fix while CVEs remain pending. A separate internal finding by Vercel intern Marcos Hernanz uncovered a cross-tenant data leak in the Linux device-mapper storage layer, and Trail of Bits spent four engineer-weeks reviewing the source code, delivering 20 confirmed findings and concluding the virtualization boundary is sound.", "body_md": "In August, we launched the [$1M hacker challenge for Vercel Sandbox](https://vercel.com/blog/one-million-dollar-hacker-challenge-for-vercel-sandbox), inviting security researchers to try and escape Vercel Sandbox. We also coordinated with cybersecurity experts Trail of Bits to review the source code and security architecture.\n\nHere is a summary of the challenge:\n\n1,285 reports over two weeks\n\n$242,000 in bounties paid out\n\nOne Linux kernel zeroday vulnerability uncovered\n\nNo reports showed anyone accessing customer data in Vercel Sandbox. The most severe finding was the defect in the upstream Linux kernel.\n\nThe program's most important report arrived on day three.\n\n[Paulos Yibelo](https://hackerone.com/paulos__?type=user) (pwn.ai), a HackerOne researcher working from a standard sandbox with no special access, found a defect in the Linux kernel's networking stack. Paulos discovered that specially crafted packets sent from within the microVM enabled three different attacks:\n\nLeak host memory over the network\n\nCrash the host machine\n\nExecute code on the host\n\nIn the first attack, memory from the host, the machine your sandbox shares with other tenants, shows up in packets the guest sends over the internet. This can happen continuously, for as long as the guest keeps sending.\n\nThis attack was demonstrated against Vercel production sandboxes.\n\nThe researcher also found that by sending specific packets, they could compromise the host.\n\nFirst, by sending two packets, the researcher crashed the host machine, shutting down every sandbox running on it. Second, using the same flaw, they were able to run commands with full administrator privileges on the host, breaking out of the sandbox’s isolation boundary.\n\nThe packet-related crash and code execution attacks were demonstrated on the researcher’s own KVM and Firecracker machines matching Vercel Sandbox configuration.\n\nAlmost every major cloud provider isolates customer workloads on this same layer of the Linux kernel, so any defect is shared-infrastructure risk.\n\nImmediately after being notified, we collaborated with the researcher to understand and mitigate the vulnerability, working with AWS and other industry partners to prioritize a fix, and ensuring that Vercel customers and other developers using the kernel were safe.\n\nThe researcher also contacted the kernel maintainers and worked with them to create candidate patches. Fixes are under private review and CVEs are pending, so we are not publishing additional details until the patches are public. We are disclosing the existence of the vulnerability because it was separately disclosed in public.\n\nThe HackerOne challenge made bounties available to any researcher, but we also extended our program to include internal evaluations and an independent investigation.\n\nBefore launching the public bounty, we performed live security assessments internally with full source access and support from the engineers who created and maintain Vercel Sandbox. We also engaged Trail of Bits, bringing on two engineers to evaluate our security architecture.\n\nNot every finding came from the outside. Marcos Hernanz, a Vercel intern, found a cross-tenant data leak in the Linux device-mapper storage layer. This vulnerability let any sandbox read recycled disk blocks from previously deleted sandboxes.\n\nTrail of Bits spent four engineer-weeks inside our source code, with full access to the repositories and two enterprise test teams. They delivered 20 findings, and each was tested, confirmed, and referenced to the exact code.\n\nThey also performed an architectural review and concluded that our virtualization boundary is sound. Each sandbox runs on its own Firecracker microVM with a dedicated guest kernel, and the device model is deliberately tiny. No guest shares a running kernel with the host, or with another tenant.\n\nOn top of that foundation, they gave us three structural recommendations, which we implemented:\n\n**Find and eliminate all trust in the guest.** Our control plane accepted values returned by software running inside the microVM, and every one of those values is tenant input. Anything that crosses the boundary should be derived server-side or signed with a key the guest cannot reach.\n\n**One policy, one decider.** Egress enforcement was split across subsystems that each parsed the same network policy independently, and they did not always agree. We now use one canonicalization routine per data type, one component per decision, and revalidate connections when policy changes.\n\n**Assume the next zero day.** We strengthened the protections around each sandbox to limit the damage if an attacker exploits the virtualization software. We keep host kernels up to date, have disabled nested virtualization, and use Firecracker’s jailer to run each microVM process with restricted permissions and isolate it from the rest of the host.\n\nWe received 1,285 reports in two weeks. This volume is only possible because researchers now hunt at AI speed.\n\nHere is a breakdown of reports by category:\n\n877 duplicates\n\n319 not applicable\n\n57 valid\n\n32 informative\n\nIn our report triage we have validated:\n\n1 Critical finding\n\n5 High findings\n\n9 Medium findings\n\n42 Low findings\n\n32 Informative findings\n\nWe have paid out $242,000 in bug bounties.\n\nThe medium, Low, and Informative findings are small policy-boundary gaps, like a firewall rule that behaved differently than documented and a field that trusted the guest more than it should.\n\nThe most severe finding was the Linux kernel zero day defect, and no reports showed anyone reading a customer’s files, secrets, or application data.\n\nWe were most impressed with the heterogeneity of the reports. It was clear that researchers used different setups and have different obsessions. Some clearly know Linux namespaces and steered their agents straight at the kernel boundary. Others went after the network policies, and many went after areas we didn’t think to put on the menu.\n\nThat variety is what you're funding with a public challenge. Here's one of the more interesting findings it produced.\n\nFiled on day one of the challenge, this report is our favorite example of why we run a public bug bounty, even after internal assessments and a white-box engagement.\n\nWhen you snapshot a sandbox, the control plane has to record where the snapshot data landed: `{storage_url, size, checksum}`. The flaw the researcher found was subtle: that answer came from software running inside the guest, and services running on the host trusted it. That is a trust-boundary violation. The guest is where our customers (and hackers) run untrusted code, so the host can never take its word as authoritative.\n\nThe researcher combined a known, documented container-escape primitive with this trust flaw. They intercepted the snapshot response inside their own sandbox and rewrote it, swapping the storage objects of two snapshots belonging to two separate accounts they owned. That meant restoring \"your\" snapshot materialized the other account's entire filesystem.\n\nThe researcher demonstrated that each account could read and modify the other’s files, verified the results using hashes, and preserved the test resources so we could independently reproduce the findings.\n\nThe limits of this attack matter. Aiming the swap at a specific victim required knowing that victim's internal snapshot storage key, a random 96-bit identifier the platform generates per snapshot and never exposes through the API. The only place that key appears is inside the victim's own sandbox. The researcher owned both accounts, which is what made the proof possible; there was no demonstrated path for a stranger to target another tenant. We still treated it as a priority fix, because a boundary that depends on a secret staying secret is not a boundary.\n\nTrail of Bits independently flagged the same root cause in their assessment, which told us the finding was real, important, and discoverable from two completely different angles.\n\nThe fix shipped within a week, and the control plane now binds every snapshot object to the project that created it and rejects anything else. Snapshot creation moved out of the guest-reachable environment into a host-side service, and storage paths are project-scoped with strictly validated URLs.\n\nWe would like to thank every researcher who filed a report, whether their finding was a zero day or a duplicate. Every technique discovered in the challenge informs how we protect the workloads running on Vercel Sandbox.\n\nThank you also to Trail of Bits for your comprehensive internal investigation and guidance.\n\nRead the [original challenge announcement](https://vercel.com/blog/one-million-dollar-hacker-challenge-for-vercel-sandbox) and learn more about Vercel Sandbox in [the documentation](https://vercel.com/docs/sandbox).", "url": "https://wpnews.pro/news/findings-from-the-1m-vercel-sandbox-challenge", "canonical_source": "https://vercel.com/blog/findings-from-the-1-million-dollar-vercel-sandbox-challenge", "published_at": "2026-10-14 07:00:00+00:00", "updated_at": "2026-10-15 00:04:20.684484+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-safety"], "entities": ["Vercel", "Vercel Sandbox", "Paulos Yibelo", "pwn.ai", "HackerOne", "Trail of Bits", "Marcos Hernanz", "AWS"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/findings-from-the-1m-vercel-sandbox-challenge", "markdown": "https://wpnews.pro/news/findings-from-the-1m-vercel-sandbox-challenge.md", "text": "https://wpnews.pro/news/findings-from-the-1m-vercel-sandbox-challenge.txt", "jsonld": "https://wpnews.pro/news/findings-from-the-1m-vercel-sandbox-challenge.jsonld"}}