{"slug": "autoscaling-lakebase-postgres", "title": "Autoscaling Lakebase Postgres", "summary": "Neon's Lakebase Postgres autoscaling changes compute size an average of 32,016 times per month per production database, or about once every 81 seconds, using in-place VM resizing and an algorithm that tracks CPU, memory, and working set. The architecture separates compute from storage, allowing resizing without stopping Postgres.", "body_md": "Choosing a database instance size before you know the workload is an old building pattern. The process is generally wonky and feels very wasteful of compute, especially [now that compute is becoming a luxury](https://www.reuters.com/technology/openai-projects-50-billion-spending-computing-power-this-year-brockman-says-2026-05-05/).\n\nLakebase Postgres, the [Neon database](https://neon.com/docs/postgres/overview), omits the sizing experience altogether thanks to autoscaling. [The average production database on Neon changes compute size 32,016 times per month](https://neon.com/autoscaling-report), or about once every 81 seconds. That autoscaling responsiveness comes from in-place VM resizing and an algorithm that tracks CPU, memory, and the database’s working set.\n\n## The architectural requirement\n\nOur autoscaling implementation is rooted on the [lakebase architecture](https://neon.com/docs/introduction/architecture-overview). Traditional Postgres runs as a stateful process tied to a machine and its disks; replacing or resizing that machine is a database operation because the machine owns both execution and durable state. But Lakebase Postgres separates those responsibilities:\n\n- The compute layer runs Postgres and executes queries. It uses RAM and local NVMe for low-latency access, and owns no durable state.\n- The storage layer owns durability and history. WAL is replicated by safekeepers running on SSDs, pageservers (also SSDs) reconstruct page versions, and object storage keeps the long-term immutable record.\n*(This blog post focuses on compute, but*[we wrote a deep dive on the storage piece](https://neon.com/blog/wal-s3-lakebase-storage-for-the-era-of-agents)if you are also interested.)\n\nA compute node can therefore start, stop, move, or change size without moving the database underneath it. This is an essential foundation.\n\nNow, when it comes to implementing autoscaling, there are two parts to the story: first, one has to determine when to adjust capacity up and down, and second, how to do it without stopping Postgres.\n\nLet's cover both in order.\n\n## Part I: The algorithm\n\n### The three autoscaling signals\n\nTo deduce when to resize, the Lakebase Postgres autoscaling algorithm tracks three signals, with each signal producing its own target compute size:\n\n- CPU load:\n`cpuGoalCU`\n\n- Memory use:\n`memGoalCU`\n\n- Compute-cache working set size:\n`lfcGoalCU`\n\nThe final scaling target is the largest of the three, [constrained to the minimum and maximum compute sizes that the user has configured for that database (the autoscaling limits)](https://neon.com/docs/introduction/autoscaling#configuring-autoscaling):\n\n### CPU (cpuGoalCU)\n\nCPU is the most straightforward of the three signals. The algorithm keeps a close watch on how hard the processor is working:\n\n- Every\n**five seconds**, the autoscaler-agent reads the** VM’s one-minute CPU load average**. - The CPU goal aims to keep that load at or below 90% of available CPU capacity.\n- When the load rises above that target, cpuGoalCU increases. When sustained load falls, the goal falls with it.\n\nUsing a one-minute average filters very short fluctuations while still responding to meaningful changes in demand. The five-second polling interval lets the system update the target as that average moves.\n\nCPU alone, however, is not enough to autoscale Postgres properly. A query waiting for data to arrive over the network can show low CPU use while performing poorly. The algorithm also needs to account for memory and cache pressure.\n\n### Memory (memGoalCU)\n\nMemory has a different failure mode from CPU. If demand briefly exceeds the available CPU, queries become slower; but if Postgres allocates more memory than the VM has, the kernel can terminate processes. The autoscaler therefore needs a much faster signal than CPU for memory exhaustion.\n\nSo the system watches memory at two frequencies:\n\n- Every\n**five seconds**, the autoscaler-agent reads** overall memory metrics from the VM**. - Every\n**100 milliseconds**, the vm-monitor checks** memory used by Postgres**.\n\nThe memory goal keeps use below 75% of allocated RAM. That headroom gives the system space to respond to new allocations and leaves memory for the guest operating system and other processes.\n\nThe vm-monitor also checks every proposed downscale. Memory cannot be removed if doing so would leave the running processes without enough space.\n\n#### A bit of history\n\nThis polling approach replaced an earlier design based on the cgroup `memory.high`\n\nevent. Crossing `memory.high`\n\ncaused Linux to reclaim memory and throttle the processes inside the cgroup. Polling proved more predictable and stable while still giving the system a 100-millisecond view of Postgres memory.\n\n### The compute cache (lfcGoalCU)\n\nThe third signal measures whether the workload’s active data fits close to Postgres. The high level story is this:\n\nLakebase Postgres separates storage and compute; when a page is not available locally, the compute requests it from the pageserver; the returned page is cached for subsequent reads. The compute cache, which we originally called the Local File Cache or (LFC), is a disk-backed cache sized to fit in the kernel page cache. It acts as a resizable extension of Postgres shared buffers. When a compute grows, the vm-monitor expands the cache to use part of the added memory.\n\nFor many OLTP workloads, performance changes sharply once the working set fits in local memory. This exposes a blind spot in CPU-only autoscaling: cache misses leave queries waiting on network requests, which reduces CPU use. The system may therefore see low CPU pressure at the exact moment when a larger cache would improve performance. So in Lakebase Postgres, there’s a third autoscaling signal that estimates the Postgres working set directly.\n\nThis is the most interesting part of the algorithm, so let’s look at how that estimate works.\n\n### Zooming in: how we estimate the Postgres working set\n\nA workload’s working set is the set of database and index pages it accesses repeatedly over a given period. To exactly count every page for the purpose of autoscaling would require too much memory, so the classic way to solve for this is to rely on [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog), a probabilistic cardinality estimator that can estimate the number of distinct items in a set using a small, fixed amount of state.\n\nFor each Postgres page access, a standard HyperLogLog implementation,\n\n- Hashes the page identifier.\n- Uses the first bits of the hash to select a register.\n- Counts the leading zeroes in the remaining bits.\n- Updates the selected register if this observation exceeds its previous value.\n\nThe distribution of those register values would provide an estimate of how many distinct pages have been observed.\n\nHowever, there’s an issue with simply using HyerLogLog for autoscaling: a standard HyperLogLog only grows. Once a register has observed a value, it cannot tell which item produced it or when that item was last seen.\n\nThat makes it good at answering, “How many distinct pages has this compute accessed since Postgres started?” But autoscaling needs a different answer, closer to “How many distinct pages belong to the workload running now?”\n\nWithout a time boundary, an old import or analytical query would remain in the estimate and keep the compute oversized long after that work ended. So we changed what the HyperLogLog registers store.\n\n### Adding time to HyperLogLog\n\nThis is how things actually work in Lakebase Postgres:\n\nInstead of setting a bit when a hash is observed, the estimator stores the current timestamp at that position. To estimate cardinality since time T, it treats positions updated after T as set and older positions as unset.\n\nThis produces an estimate for any window ending at the present, including\n\n- Distinct pages accessed in the last minute\n- Distinct pages accessed in the last five minutes\n- Distinct pages accessed in the last hour\n\nSo, going back to the algorithm, this is how the granularity actually works: **every 20 seconds**, the autoscaler-agent collects **working-set estimates for windows from one to 60 minutes**.\n\nBut the story does not end here. As surely you’re noticing, this is a wide time window. How do we actually choose it?\n\n### Choosing the working set time window\n\nThe problem is this: there is no universal window that describes a database’s current working set. If we pick a short window, the autoscaling engine responds quickly when a workload ends, but it would discard cache too aggressively between bursts. If we pick a long window, the algorithm would protect the cache, but it would also keep memory allocated for work that is no longer running.\n\nThe algorithm solves this by looking at how the working set changes overtime. For example: for a steady workload, the estimated number of pages initially grows, and then levels off. Extending the window adds time, but few new pages are added, because the same working set is being accessed repeatedly.\n\nNow, consider a heavy workload that ended recently. Short windows contain only the current, lighter workload; but once the window reaches far enough into the past to include the previous workload, the estimate jumps. The algorithm searches for that jump, which marks the end of the current plateau.\n\nIn short,\n\nThe implementation starts its search after five minutes. This prevents the compute from shrinking immediately during a short pause and then regrowing for the next burst. But if the algorithm finds no sharp increase, it uses the 60-minute estimate - that is the expected result for a stable workload whose working set remains active throughout the hour.\n\n### Projecting cache growth\n\nThere’s one last piece to it. Measuring the current working set lands slightly too late: suppose a workload begins scanning a new set of pages. If the compute cache grows only after those pages have been read, early pages may already have been evicted to make room for later ones. The cache then has to fetch some of the same data again.\n\nSo the algorithm also projects working-set growth forward. It examines how the estimate increases from one duration to the next and allocates enough cache for the working set expected by the next control interval.\n\nBecause cache metrics are fetched every 20 seconds, the projection covers only a fraction of a minute. Longer projections would react earlier, but they would also amplify brief spikes and make the compute oscillate.\n\nThe projected size becomes `lfcGoalCU`\n\n. And the algorithmic goal is to fit the working set within the portion of memory available to the compute cache, up to 75% of the compute’s RAM.\n\n## Part II: Resizing the running compute\n\nTo recap: the scaling target was,\n\nThose three signals tell the system what size to aim for. Applying that size means changing CPU and memory on a running VM without interrupting Postgres.\n\nEach Postgres instance in Lakebase Postgres runs inside its own virtual machine in a Kubernetes cluster. We use VMs because they provide a strong isolation boundary and, unlike a conventional container allocation, allow CPU and memory to be added to or removed from a running guest.\n\nFour components coordinate each compute resize:\n\n- The\n**autoscaler-agent** runs on every Kubernetes node. It collects metrics from the Postgres VMs on that node, calculates target sizes, and initiates scaling. - The\n**vm-monitor** runs inside each VM. It watches Postgres memory closely, validates downscaling requests, and resizes the compute cache. - A\n**modified Kubernetes scheduler** maintains the global view of available resources. Every upscale must be approved by the scheduler before memory is committed. **NeonVM** applies the change. It is a custom Kubernetes resource and controller, built with QEMU and KVM, that can add or remove CPU and memory from a running VM.\n\n### Scaling up\n\nAs we just saw, scaling up happens when one of the three goals calls for more compute than the VM currently has. An upscale follows this sequence:\n\n- The autoscaler-agent calculates the new target from the CPU, memory, and working-set goals.\n- The Kubernetes scheduler checks whether the node can satisfy the request without overcommitting memory.\n- Once approved, the autoscaler-agent updates the NeonVM resource.\n- The NeonVM controller adds CPU and memory to the running VM.\n- The vm-monitor expands the compute cache to use the new capacity.\n\nThe scheduler is the single source of truth for allocation. It sees both ordinary Kubernetes scheduling and autoscaling requests. Without that coordination, the scheduler could place a new workload on a node at the same moment the autoscaler committed the remaining memory to a Postgres VM.\n\nIf a node is too full to grow in place, NeonVM can live-migrate the VM to another node. The VM keeps its IP address, so existing connections stay open. Lakebase Postgres computes have little durable local state to move, so migration is mostly VM memory and runtime state.\n\n### Scaling down\n\nA downscale uses the exact same components, with one extra check inside the VM. The vm-monitor confirms that removing memory will still leave enough for Postgres and the rest of the guest. If it would not, the downscale does not proceed.\n\n#### Scaling down counts as much as scaling up\n\nSome autoscaling systems are quick to add capacity but slow to give it back, leaving databases oversized long after a spike has passed. Lakebase Postgres treats both directions the same way. The goal is to track the workload as closely as possible moment to moment, so you stop paying for capacity as soon as you stop needing it.\n\n## Wrap up\n\nLakebase Postgres watches the workload as it runs and resizes compute to match in real time. The lakebase architecture makes this possible: since storage is decoupled and durable on its own, compute is free to move without worrying about the data.\n\nThe resulting system scales in both directions, on a live database, without dropping connections. Most importantly, it looks past the obvious signal: tracking CPU alone would miss a workload stalled on cache misses, so the algorithm also tracks memory pressure and a time-aware estimate of the working set.\n\nThe final loop runs at three timescales:\n\n- 100 milliseconds: the vm-monitor checks Postgres memory to catch rapid allocation\n- 5 seconds: the autoscaler-agent reads CPU and overall memory\n- 20 seconds: the autoscaler-agent evaluates working-set estimates across windows from one to 60 minutes\n\nThat is how a production database can change size more than [32,000 times per month](https://neon.com/autoscaling-report).\n\nAs compute gets more expensive and more contested, paying for a peak you rarely reach is a building pattern that might not be possible very soon. Autoscaling prepares Postgres for workloads where wasted compute is not an option.\n\n## Run it\n\nLakebase Postgres runs in two places, on the same infrastructure and with the same core feature set. What differs is what surrounds it:\n\n- On Neon, it anchors a complete set of cloud backend primitives for developers, startups, and agent platforms\n- On Databricks, it is integrated with the rest of the Data Intelligence Platform: Unity Catalog governance, lakehouse analytics, notebooks, and AI workflows\n\nAsk your agent to deploy either of those, and put autoscaling to the test.", "url": "https://wpnews.pro/news/autoscaling-lakebase-postgres", "canonical_source": "https://neon.com/blog/autoscaling-lakebase-postgres", "published_at": "2026-08-31 12:00:00+00:00", "updated_at": "2026-08-31 16:54:40.160143+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools"], "entities": ["Neon", "Lakebase Postgres"], "alternates": {"html": "https://wpnews.pro/news/autoscaling-lakebase-postgres", "markdown": "https://wpnews.pro/news/autoscaling-lakebase-postgres.md", "text": "https://wpnews.pro/news/autoscaling-lakebase-postgres.txt", "jsonld": "https://wpnews.pro/news/autoscaling-lakebase-postgres.jsonld"}}