# How we use agents to patch and deploy 40M lines of Chromium code in < 1 day

> Source: <https://www.browserbase.com/blog/how-we-deploy-custom-chromium>
> Published: 2026-11-18 21:20:57+00:00

**Recommendation:** Use coding agents to port your fork's patches onto each upstream release and gate them with the full build and test suite before the vendor's biweekly release ships.
More in the playbook "Working with AI coding agents": <https://wpnews.pro/playbooks/coding-agents>

At Browserbase, we maintain an internal fork of Chromium with >18,000 lines of stealth and security patches to enable agents to browse the web without being flagged as malicious bots. It also happens to be one of the largest open-source projects, with > 40 million lines of code. That’s equivalent to roughly 1,125 copies of the bible.

Chromium’s size also means that it’s one of the largest attack surfaces on the planet, [with 400-500 new CVEs per month.](https://nvd.nist.gov/vuln/search?keyword=chrome&resultType=records) Our customers rely on us to keep the browser running their sessions up to date. When upstream fixes a vulnerability, we need to ensure that we minimize the time between the release and our fix.

On top of that, Google releases a major version of Chromium every 2 weeks (previously every 4). To upgrade our custom fork, it used to take days to move our patches to the new source code (in a time before coding agents), and we spent long nights praying the 6-hour build would succeed overnight.

In the last 2 years, we’ve drastically improved our deployment pipeline, taking what used to be a week-long sprint into fully tested patches before Google officially releases the next major version. Some of our [patches](https://chromium-review.googlesource.com/c/chromium/src/+/8504035) were upstreamed to the official project.

In this blog post, we’ll dive into Chromium, why it’s so hard to deploy, how we maintain our patches, what made builds suck, and how we use coding agents to streamline our workflow.

### Tell me more about Chromium

For those who don’t know, [Chromium is an open-source](https://github.com/chromium/chromium), multi-process browser built on the V8 JS engine that powers Google Chrome. It’s the base for many browser products today: Dia/Arc, Brave, MS Edge, and, of course, Browserbase. So what actually makes a browser upgrade that hard? The source code is public, and we already know how to compile it. Why can't we just pull the latest version, apply our changes, and ship it?

Chromium is an extremely complex piece of software, and is [basically a full OS](https://chromium.googlesource.com/chromium/src/+/main/docs/process_model_and_site_isolation.md). It has access to [hardware resources](https://developer.chrome.com/docs/capabilities) like USB, Bluetooth, serial, HID, GPU, sensors, and printing, each exposed to web pages through permissioned interfaces.

A full process-and-execution environment where a privileged browser process starts, monitors, and can kill dozens of isolated child processes with inter-process communication ([Mojo](https://www.chromium.org/chromium-os/developer-library/reference/cpp/cpp-mojo/)) and [full sandboxing](https://chromium.googlesource.com/chromium/src/+/master/docs/design/sandbox_faq.md) with a JIT JS/WASM compiler and GC to run untrusted code from any website.

[A full networking stack](https://www.chromium.org/developers/design-documents/network-stack), including DNS, HTTP/1.1, HTTP/2, HTTP/3, and QUIC, TLS via BoringSSL, caching, cookies, and proxy handling. Memory management and protection through IndexedDB, Cache Storage, SQLite databases, and [quota management](https://chromium.googlesource.com/chromium/src/+/main/storage/browser/quota).

To interact with it, there’s a [GUI rendering engine (Blink)](https://developer.chrome.com/docs/web-platform/blink)**,** 2D rasterization, [translation of OpenGL ES to D3D](https://github.com/google/angle), WebGPU, [audio and video decoding](https://chromium.googlesource.com/chromium/src/+/main/media) (FFmpeg, libvpx, dav1d), DRM, WebRTC for real-time communication, and camera and microphone access.

On top of all that, there’s a full product layer with extensions, sync, autofill, and password management, built as [several hundred feature components](https://chromium.googlesource.com/chromium/src/+/main/components) on top of everything above.

## Applying patches by hand

To start, why are we patching Chromium in the first place? Surely Google knows what they’re doing.

We classify our patches into 3 major categories: security, performance, and stealth.

Security patches help us defend ourselves and our customers from CVEs and potential vulnerabilities. We’re in an unprecedented time in cybersecurity and must take a proactive approach to keeping our customers safe.

Performance patches help us run our browsers faster and more memory-efficient, so we don’t have to watch paint dry while our builds compile.

Stealth patches help us reliably access websites across the internet. If you’d like to learn more, [send us a message.](https://www.browserbase.com/contact)

Back in 2024 (before the initial release of our product), we applied patches by hand. We kept a running list of patches in Notion and would hand-apply them over the course of a week. You had to open the old and new implementations side-by-side and figure out where the old modification belongs. Sometimes, old parts of the patch weren’t necessary anymore, so we found which parts to keep and which parts were redundant, by hand.

To put things into perspective, the SOTA coding model at the time was Claude 3.5 Sonnet and Nvidia’s valuation was only $3.011T. Agents weren’t really a thing yet, and AI wasn’t good enough to handle these changes. This system was functional, but not efficient. You can easily find out which file needs to be changed, but not how to change it or why.

Chromium source already has its [own test suite](https://www.chromium.org/developers/testing/browser-tests/), but that doesn’t equate to it being ready to roll up and deploy. It has to cover our patches, infra shape, deployment config, and how automation clients ([Stagehand](https://www.stagehand.dev/), Playwright, etc.) use our browser.

To solve this, we use named patches, better intent documentation, and exhaustive regression tests to give coding agents enough context to handle upgrades. Agents run in loops following this pattern: inspect a rejected patch, find where the behavior moved upstream, regenerate the diff, replay the series, and test.

### Patch regressions

Just because we apply a patch properly doesn’t mean the browser behaves correctly. Let’s walk through an example with page forward and back.

Normally, a page navigation loads a document, and Chrome’s back/forward cache keeps the old document, and it’s JS heap to restore it if you come back. The cache means the browser can skip fetching, parsing, and executing whatever JS the page has. A restored page receives `pageshow` with `persisted` set to `true`. [How the back/forward cache work](https://web.dev/articles/bfcache)s.

However, [Playwright](https://playwright.dev/python/) maintains its own state using [CDP](https://www.browserbase.com/blog/what-is-cdp) events and tracking the frame, doc, and which lifecycle events have arrived. Our tests covered Chromium restoring a page and sending `Page.frameNavigated`, but the automation client was still waiting for the restored document’s lifecycle events. This means an automation call would hang until the navigation timeout. Our patch re-announces `DOMContentLoaded` and `load` after the restore's `frameNavigated`, using the restored document's loader ID.

To write a good test, you have to make sure load events fire, the automation call finishes, and the document actually gets restored. Something close to this:

### Unit tests aren’t enough

Sometimes all tests passed, but it still missed issues. In our upgrade to Chromium 157 HTTPS-First balanced mode was enabled by default. This means navigations to an HTTP site could be retried as [HTTPS](https://developer.mozilla.org/en-US/docs/Glossary/HTTPS) and fail (since there’s no native forward navigate button on headless chrome) instead of loading the HTTP page. Our binary tests passed since they used loopback addresses (loopback hosts and ports other than 80 were exempt from this behavior).

We caught a failure in application verification (checking if the new version works on our infra) when loading HTTP sites with the config we actually use. To add that case to our suite, we gave the browser a public hostname on port 80, mapped it to a local server, and made the HTTPS destination refuse connections.

It looked a little like this:

The HTTP server is still local. We're just changing the URL the browser evaluates, so its navigation policy takes the path it would take for a public HTTP site. The original patches inadvertently avoided that policy.

To ensure our Chromium works, we must retain separate patch, binary, and application checks. Binary and patch tests check for local invariants, while application tests help ensure our browser’s behavior stays consistent in production.

## Hold on, 6-hour builds?

Yes, you heard that right. It used to take ~6 hours to compile and build Chromium after we applied our patches. The team spent many late nights preparing to kick off a build at midnight and sleeping with one eye open to make sure build errors didn’t ruin the morning.

Chromium has a long history of build infrastructure—[Ninja](https://ninja-build.org/) originated from work on Chromium. It’s [designed](https://ninja-build.org/manual.html) to put a policy in a generator and keep execution focused on an explicit dependency graph. The [current Chromium build](https://chromium.googlesource.com/chromium/src/+/main/docs/linux/build_instructions.md) uses GN to generate `.ninja` files (the graph) and Siso as its main build tool (action explanations). You’d think caching meant keeping old obj. files, but the build must also believe those files are still valid.

We dove into the source code and found two independent causes of unnecessary full rebuilds. Both made a small Chromium update look much larger to the build system than it really was.

### Cache money

Our cache directories used to include the Chromium version, which sounds reasonable enough. Each version gets its own directory, and you can immediately tell what’s in it, but every version bump changes the entire workspace.

There were absolute paths in the build’s dependencies, compiler commands, and build state. The build system couldn’t recognize over 3,586 build steps as work it had already completed because moving the workspace changed the paths in its commands.

The solution was changing to a stable workspace across all versions of Chromium, which required a one-time rebuild so all subsequent updates could reuse it. Now the only thing that triggered a rebuild was an update to src.

### Fixing our Linker

Now that compilation is much faster, we addressed the linking bottleneck. During testing, we found that in a 346-second build step, compilation took 128 seconds while linking took 151 seconds.

[Linking](https://people.cs.pitt.edu/~xianeizhang/notes/Linking.html) is how the build process connects the compiled object files, resolving references, pulling in required libraries, and laying out the code and data in the final executable.

In the example above, it was ~94% serial. The obvious solution is to parallelize (or so we thought).

The linking process uses [ThinLTO](https://clang.llvm.org/docs/ThinLTO.html), which lets the compiler optimize across translation unit boundaries and combine module summaries during linking. We added more linker threads, but found that it didn’t actually improve the measured link speed. Instead, we had to investigate which part of the link was slowing down the system and what runtime benefits we would trade off by removing it.

We found a way to cut roughly 54 seconds from linking by switching the algorithm that arranged functions in the executable. The original config used `cdsort` (cache-directed sort), which uses a more detailed locality model, considering function sizes, call frequencies, and distances between callers and callees to choose an arrangement.

In experimentation, we switched it to `hfsort`, which uses simpler rules and groups a function with its most frequent caller, subject to size and execution-density limits, then orders those groups by how frequently they execute relative to their size.

[Here’s what they look like in code.](https://github.com/llvm/llvm-project/blob/main/lld/ELF/CallGraphSort.cpp)

Although the link became faster, this config actually slowed down our browser startup time by 3.33%, so we eventually switched back to keep our browsers fast for our customers.

## Taking Security seriously

It seems like every day a new vulnerability is released. A side effect of LLMs getting better at coding is that they’re very good at finding supply chain attacks and critical vulnerabilities in even our most robust software.

It’s our job to protect our infrastructure and those who use it from these vulnerabilities. By tightening our iteration and deployment cycles, we can take a best-effort approach to secure Chromium.

If you think about a browser automation session, it takes arbitrary code and content from the internet (an extremely large place) and processes it, while potentially being signed into customer applications.

Thankfully, Google already considered this, and Chromium addresses isolation natively. The browser itself is a sandbox, so a renderer exploit doesn’t automatically give an attacker control of the host. It was designed to restrict compromised code and explicitly assumes that sandboxed code may be malicious.

## Pre-firing the release

Chromium's source is public, so we can follow their updates as they prepare for a release. We’ve built internal tools to watch for upstream tags, their ancestry, and file diffs. For example, if a tag with code changes is followed by a version-only tag, we know that we should attempt to build that version.

There are other patterns we check for, but we’ll keep those proprietary. Coding agents are extremely useful here. We invested time in building an environment for background agents to inspect dependency and compiler changes, replay patches, investigate rejects, remove obsolete backports, check new defaults, and collect and build tests.

Since the environment sets both non-deterministic (skills) and explicit (CI) guardrails, we can just run agents against it instead of manually sifting through the code like before.

## Are we actually running the new browser?

Once we have the full build, we need to deploy the binary to our VPC.

We follow a standard deployment process, bundling our binary into an image, pushing that image to ECR, then deploying a canary version to a small percentage of our machines, monitoring for a bit, then rolling out to all of our customers.

To validate deployment, we compare the proposed version across binaries and the connected browser session. Agents go out and collect evidence for us across security fixes, versioning, and test results. It must have the right browser and driver executables, the version reported through Playwright, and CDP’s `Browser.getVersion`.

We also compare the built binary against the prod runtime base, since the binary that loads in the builder can depend on libraries or symbols that are unavailable in the runtime environment.

If you want to roll back versions, you unfortunately can’t just put the old executable back. The new version might have already migrated a user data directory into a new format that the old version can’t read.

## Scar tissue is the moat

There were many sleepless nights that our team endured to make this possible. A large part of the value-add is the years of scar tissue that our engineering team has from working on Chromium.

Interestingly, a majority of the improvements we made came from interpretability, understanding the system, and making it easier to reason about. We learned why a patch existed or why an object needed to be rebuilt.

With improvements to LLMs, making the system as understandable as possible becomes a rich environment for agents to gather maximum context and iterate in loops until they satisfy requirements. Shorter feedback loops let us try multiple optimization paths and only keep the best ones. We can also look ahead and fix security issues before the release happens.

It used to take a full week of pain and suffering to deploy every other version of Chromium (even with the 4-week cycles). Now we can test and build a stable version before Google (with roughly 1000x fewer engineers). Shoutout to Claude.

If any of these problems sound interesting to you, we’re hiring across various roles including [engineering](https://www.browserbase.com/careers).

→ Kyle

## Start building with Browserbase

Run headless browsers for your agents and automations at scale. Get started for free in minutes.
