Dev ToolsNews
A global TRACE log sink in OpenAI's CLI can write hundreds of terabytes a year to a local SQLite file — and it's been festering for months.
Here's a failure mode you don't usually associate with an AI coding assistant: it slowly destroys your hardware. According to a GitHub issue against OpenAI's Codex, the CLI's local feedback-logging database can write on the order of 640 TB per year to a developer's SSD — quietly, in the background, with no error, warning, or visible disk bloat to tip you off. One reporter clocked 37 TB written in 21 days of uptime, traced almost entirely to Codex's SQLite logs.
That's not a corner-case crash. That's a default configuration burning through the warranted write endurance of a consumer drive in well under a year. And the root cause is the kind of one-line decision that any of us could have shipped on a tired Friday.
A global TRACE default, plus write amplification #
The mechanism is mundane and that's exactly why it's worth your attention. Codex installs a SQLite feedback-log sink configured with a global default of Level::TRACE
:
Targets::new().with_default(Level::TRACE)
That single with_default(Level::TRACE)
persists everything — not just Codex's own events, but the TRACE chatter of every dependency in the tree. The reporter's breakdown is brutal: TRACE accounts for ~71% of retained log bytes, and the single noisiest source is raw websocket payload logging from the responses endpoint at 527 MiB. Add mirrored OpenTelemetry events (codex_otel.log_only
and codex_otel.trace_safe
) and you've explained roughly 96% of the volume before you've logged a single thing a human would ever read.
pie title Retained log bytes by level (single-machine sample)
"TRACE" : 70.7
"INFO" : 25.7
"DEBUG" : 3.0
"WARN" : 0.6
The content is pure noise. The top TRACE source by frequency is inotify
filesystem events — 128,764 hits for ld.so.cache
being opened, tens of thousands more for locale.alias
, passwd
, nsswitch.conf
. Underneath that sits tokio-tungstenite's internal poll_read
/poll_next
plumbing logged line by line. None of this helps debug a coding session; it's the runtime's own breathing, written to disk forever.
But the retained database size — about 1 GiB — is a lie about the real write volume, and this is the part developers should internalize. In a 15-second window the reporter saw ~36,211 rows inserted while the retained row count stayed flat at 681,774. The max row ID had already passed five billion. Codex is running a continuous insert-index-WAL-then-prune cycle: it writes rows, updates indexes, flushes to the write-ahead log, then deletes them to stay under a retention cap. The on-disk footprint looks innocent. The flash underneath is being hammered. That's textbook write amplification, and it's why a 1 GiB database can represent terabytes of physical writes.
Why "it's just logs" is the wrong reflex #
If your instinct is "logs are cheap," that instinct was calibrated on spinning rust. NAND flash doesn't work that way. SSDs are rated in TBW — terabytes written — because each cell tolerates a finite number of program/erase cycles. A typical 1 TB consumer drive carries a warranty around 600 TBW. At 640 TB/year, Codex alone would spend that entire endurance budget in roughly twelve months, equivalent to ~640 full-drive overwrites in a year. Laptops with soldered, non-replaceable SSDs make that an expensive lesson.
The deeper smell here is architectural, and it's common across the new wave of agentic dev tools: telemetry that was designed for a server fleet gets shipped to run permanently on a developer's laptop. On a backend you'd ship TRACE spans to a sampled collector with retention policies and you'd never blink. Mirror that same firehose into a local embedded database with a tight retention window and you've built a perfect write-amplification engine. OpenTelemetry's own SDK logs showing up twice in the top sources — once via log_only
, once via trace_safe
— is the tell that nobody costed out what "persist all targets at TRACE" means on an endpoint device.
There's also a privacy wrinkle worth flagging: the reporter deliberately omitted the raw websocket/SSE payload samples because they can contain private conversation content. So the default behavior isn't only writing terabytes — it's writing your prompts and model responses to an unencrypted local SQLite file by default. OpenAI's own troubleshooting guidance already warns users to scrub logs for sensitive information before sharing them, which is a tacit acknowledgment of what ends up in there.
What to actually do about it #
This isn't a freshly discovered one-off. A near-identical report (issue #22444) and threads in OpenAI's own developer forum describe the same SQLite trace flood filling disks "ever since a few versions ago," with at least one user maintaining a homegrown bash script to nuke the WAL and kill the process. When multiple independent reporters and an open PR are converging on the same SQLite sink, treat it as a real, reproducible defect rather than one weird machine.
Concrete steps if you run Codex CLI:
Check whether it's happening to you. The files live at~/.codex/logs_2.sqlite
, plus the-wal
and-shm
sidecars. A continuously growing WAL while Codex idles is your signal.
ls -la ~/.codex/logs_2.sqlite*
sqlite3 ~/.codex/logs_2.sqlite 'select max(rowid) from /* your log table */;'
Watch physical writes, not file size. The on-disk DB stays ~1 GiB; the damage is in cumulative writes. On Linux, track per-process or per-device write bytes (iostat
,/proc/<pid>/io
, or your SMARTTotal_LBAs_Written
/Data Units Written
counter viasmartctl -a
) rather than trustingdu
.Don't just SQLite holds the WAL open; deleting underneath a live process gets you a half-detached file and a confused client. Stop Codex first, then clear the files — which is precisely why frustrated users wrapped it in a script.rm
the DB with Codex running.Until a fix lands, treat long-lived idle Codex sessions as a liability. The worst case in these reports isleaving Codex open, where the inotify/websocket TRACE stream keeps writing with no active work happening.
The right upstream fix is the boring one the reporter proposed: stop using a global TRACE default for the persistent sink, raise thresholds for dependency noise (log
, hyper_util
, tokio-tungstenite, inotify, low-level OTel), and stop persisting full raw protocol payloads — store event summaries instead. That alone reportedly removes ~96% of retained bytes without disabling feedback logs at all. It's a filter-list change, not a redesign.
The takeaway #
The interesting thing here isn't that a bug shipped — it's how invisible it was. No crash, no error, a deceptively small database file, and months of reports before it broke through. That's the new tax on running always-on agentic tooling on your own machine: server-grade observability defaults are landing on endpoint hardware, and the cost shows up not in your logs but in your SMART counters. Until Codex narrows that sink, the move is simple — go look at ~/.codex/
, watch your drive's lifetime-writes counter, and don't leave the agent idling for days. The fix is trivial; the habit of checking what your tools write to disk is the part worth keeping.
Sources & further reading #
Codex logging bug may write TBs to local SSDs— github.com - Troubleshooting – Codex app | OpenAI Developers— developers.openai.com - How do I troubleshoot errors or issues when using Codex CLI?— milvus.io - Disk space exhaustion when leaving Codex open - Codex - OpenAI Developer Community— community.openai.com
Priya Nair· AI & Developer Experience Writer
Priya covers AI frameworks, developer productivity tooling, and the startup ecosystem across South and Southeast Asia, bringing a researcher's rigour and a practitioner's empathy to every story. She is deeply sceptical of benchmarks and asks hard questions so her readers don't have to.
Discussion 0 #
No comments yet
Be the first to weigh in.