How OpenAI scales PostgreSQL to 800 million users OpenAI scales a single PostgreSQL primary, backed by nearly 50 cross-region read replicas, to serve ChatGPT's 800 million users at millions of queries per second, according to the company's January 2026 engineering post. Rather than shard the existing deployment, OpenAI has frozen new tables in Postgres and routes new features to sharded systems like Azure Cosmos DB, while moving write-heavy workloads off the primary. The company attributes most outages to a retry-driven overload loop and counters it with replica reads, lazy writes, rate-limited backfills, transaction timeouts and a hot standby. OpenAI scales PostgreSQL for ChatGPT with one primary server that takes every write and nearly 50 read replicas spread across regions, and that handles millions of queries per second for 800 million users. Most of the work goes into keeping load off the primary, and into limits at every layer so one spike can't snowball into an outage. Below is each problem they hit and how they fixed it, checked against OpenAI's own engineering post https://openai.com/index/scaling-postgresql/ from January 2026, plus the parts I'd copy in a normal app, with SQL I tested on Postgres 18. The video above covers the same ground. This came as a shock to me. I had no idea a single Postgres write instance could go that far. The primary is an Azure Database for PostgreSQL flexible server, and OpenAI says its Postgres load grew more than 10x in the past year. They haven't sharded it, because sharding the existing workloads would mean changing hundreds of application endpoints and could take months or years. Instead they've drawn a line: no new tables go into this Postgres deployment, and new features default to sharded systems like Azure Cosmos DB. Writes are the weak spot. Postgres uses multiversion concurrency control MVCC , so updating even one field copies the whole row into a new version. Under heavy writes that means write amplification, dead rows that queries have to skip past, table and index bloat, and autovacuum work. Reads scale out across replicas. Writes all land on one machine. OpenAI says its Postgres outages usually follow the same loop. Something upstream causes a sudden jump in load. The cache layer fails and every request misses, or a new feature launch floods the database with writes. A burst of expensive multi-table joins eating the CPU can do it too. Queries slow down and start timing out. Clients retry, the retries add even more load, and it keeps getting worse until the whole service is down. Every fix below breaks that loop somewhere. Reads go to replicas wherever possible. The only reads left on the primary are the ones inside write transactions, and those get tuned so they stay fast. For writes, OpenAI moved the shardable, write-heavy workloads to Cosmos DB. The harder-to-shard ones are still being moved. Then they went through their own code. They fixed bugs that wrote the same data twice and added lazy writes to spread spikes out. Backfills got strict rate limits, so filling in a column can't hammer the primary more on that below . You can hunt for duplicate writes in a small app the same way. Turn on the pg stat statements extension, sort by how often each query runs, and look for writes that run more often than they should. OpenAI found a query that joined 12 tables, and spikes in it caused some of their worst incidents. Their advice is to avoid multi-table joins where you can, and to break up the ones you need and do the join in application code instead. They also say many of the bad queries came from ORMs. Prisma, Drizzle, SQLAlchemy and Django's ORM all make it easy to write code that turns into slow SQL without you noticing, so read the SQL your ORM generates for your hottest endpoints. Then set a timeout for transactions that sit open doing nothing. OpenAI uses idle in transaction session timeout , because those sessions stop autovacuum from cleaning up dead rows: ALTER DATABASE postgres SET idle in transaction session timeout = '60s'; Swap postgres for your database name. New connections pick it up. When I tested it with a 3-second value, a session that ran BEGIN and then sat idle got cut off with FATAL: terminating connection due to idle-in-transaction timeout . A single writer is a single point of failure. OpenAI runs the primary in high-availability mode with a hot standby, a replica kept in sync that can be promoted quickly if the primary dies or needs maintenance. In the video I called this zero downtime, but the post only promises to keep downtime short. The bigger fix is that most critical requests only read data, and those reads now go to replicas. If the primary goes down, writes fail but ChatGPT keeps answering reads, so it's no longer their worst severity of incident. They also run more than one replica in every region, so losing one replica doesn't take a region down. Sometimes one workload, like a new feature with inefficient queries, eats most of the CPU and slows everything else on the same instance. OpenAI splits requests into high-priority and low-priority tiers and routes each tier to its own instances. A heavy low-priority job can then max out its own servers without touching high-priority traffic. They do the same between products, so one product's traffic can't hurt another. Each Azure PostgreSQL instance allows at most 5,000 connections, and OpenAI has had incidents where a connection storm used them all up. Plain Postgres has no fixed number. max connections is a setting, 100 by default. Their fix is PgBouncer in statement or transaction pooling mode, so many clients share a few real connections. In their benchmarks the average time to get a connection dropped from 50 ms to 5 ms. PgBouncer runs in the same region as the clients and the replica it sits in front of, and every read replica has its own Kubernetes deployment running several PgBouncer pods behind one Kubernetes Service. To be fair, I haven't hit this limit myself, because I'm not serving hundreds of millions of users. But connections leak in ordinary apps too, through idle ones you forgot to close or one connection per request, so a pooler is worth having long before you're at OpenAI's size. OpenAI serves most reads from a cache. The danger is a sudden drop in the hit rate, because every miss turns into a database query at the same moment. Their fix is a lock per cache key. When many requests miss on the same key, one takes the lock, reads from Postgres and refills the cache. The rest wait for the cache instead of all going to the database. I think this is genius, because the data only needs to come from the database once. If you're caching LLM answers rather than rows, I wrote about semantic caching https://www.hassandev.me/blog/youtube/semantic-caching-stop-paying-for-the-same-answer-3gU BA 19f0 separately. The primary streams its write-ahead log WAL , the record of every change, to each replica. With nearly 50 replicas that's a lot of network and CPU on one machine, and replica lag gets worse as you add more. The fix is cascading replication: the primary sends WAL to a few intermediate replicas, and they pass it on to the rest. OpenAI says this could take them past a hundred replicas. In the video I talked about it as if it's already running, but it isn't. They're still testing it with the Azure team, because it makes failover harder. OpenAI rate-limits at the application, the connection pooler, the proxy and the query layer. They also changed their ORM so it can rate-limit or fully block a specific query pattern, which lets them cut off a runaway expensive query fast. They also warn against short retry intervals, because those cause retry storms. The post doesn't give numbers, so here's what I'd do: wait 5 or 10 seconds before the first retry, back off to 30 or 60 seconds if it keeps failing, add some randomness so clients don't retry in sync, and after a few failures tell the user something's wrong and alert yourself. Some schema changes, like changing a column's type, make Postgres rewrite the whole table. OpenAI only allows lightweight changes, such as adding or removing certain columns. Creating and dropping indexes concurrently is allowed. Every schema change has a strict 5-second timeout. When they backfill a column, the rate limit is so tight that it sometimes takes over a week. A week sounds absurd when you can change a schema in seconds on a small app. But it never causes downtime, and at their scale I think that's the right trade. Here's how to see it for yourself. Start with a table of a million rows: CREATE TABLE users id bigint PRIMARY KEY, email text NOT NULL, age integer ; INSERT INTO users id, email, age SELECT g, 'user' || g || '@example.com', 20 + g % 50 FROM generate series 1, 1000000 AS g; Then compare the table's file number before and after each change. A new number means Postgres wrote a whole new copy of the table: \timing on SELECT pg relation filenode 'users' ; -- Metadata only: no rewrite ALTER TABLE users ADD COLUMN plan text DEFAULT 'free'; SELECT pg relation filenode 'users' ; -- Changing the type rewrites every row ALTER TABLE users ALTER COLUMN age TYPE bigint; SELECT pg relation filenode 'users' ; Run it with psql -f . Adding the column took 6 ms and the file number stayed at 16384. Changing age to bigint took 3.3 seconds and the number changed to 16394. That was on a million small rows on my machine. On a big production table it can run for minutes, and the table is locked the whole time. OpenAI doesn't say which timeout setting they use. I'd set two at the top of every migration: SET lock timeout = '2s'; SET statement timeout = '5s'; ALTER TABLE users ADD COLUMN email domain text; lock timeout matters even for instant changes. An ALTER TABLE needs an exclusive lock, so it waits behind any open transaction on the table, and every query that arrives after it queues behind the ALTER . In my test, one transaction sat open on users and an ALTER TABLE without a timeout went into the queue. A one-row lookup by primary key then took 6.5 seconds. With the timeouts above, the migration gave up after 2 seconds with canceling statement due to lock timeout , and the same lookup took 1.7 seconds. You run the migration again later. To fill the new column, update a few thousand rows at a time and pause between batches: bash /usr/bin/env bash Fill users.email domain 5,000 rows at a time, with a pause between batches. set -euo pipefail max id=$ psql "$DATABASE URL" -qtAc "SELECT max id FROM users" for start = 1; start <= max id; start += 5000 ; do psql "$DATABASE URL" -qc " UPDATE users SET email domain = split part email, '@', 2 WHERE id = $start AND id < $start + 5000 AND email domain IS NULL" sleep 0.2 done echo "done up to id $max id" Run it with your connection string in DATABASE URL : DATABASE URL="postgresql://user:password@localhost:5432/postgres" ./backfill.sh It printed done up to id 1000000 after 72 seconds, 40 of them spent in the pauses. A single UPDATE did the same million rows in 9 seconds, but it locks every row until it commits and writes all its WAL in one burst that every replica then has to replay. Raise the sleep if your database is busy. OpenAI's backfills sometimes take over a week. OpenAI reports low double-digit millisecond p99 latency on the client side and five nines of availability. In the past 12 months they had one SEV-0 Postgres incident, during the ChatGPT ImageGen launch, when write traffic jumped more than 10x as over 100 million new users signed up in a week. They're still moving the remaining write-heavy workloads off Postgres, and they haven't ruled out sharding it later. If you're a backend developer, this post is gold. Most of us reach for a new database the moment an app gets a few thousand users, and OpenAI is running ChatGPT on one Postgres primary. Most of their fixes work on a small app too. Reading the SQL your ORM writes and putting a timeout on every migration cost you an afternoon, and neither needs a new database. Go read their post https://openai.com/index/scaling-postgresql/ too, because they explain all of it better than I did in the video. Originally published on hassandev.me https://www.hassandev.me/blog/youtube/800-million-users-1-database-r 7KmsmepBc .