{"slug": "codex-s-trace-by-default-logging-is-quietly-eating-ssds", "title": "Codex's TRACE-by-Default Logging Is Quietly Eating SSDs", "summary": "OpenAI's Codex CLI tool writes up to 640 TB per year to SSDs due to a default TRACE-level logging configuration, causing rapid wear on consumer drives. A developer reported 37 TB written in 21 days, with the issue traced to a SQLite feedback-log sink that logs all dependency chatter. The problem has persisted for months, potentially destroying hardware by exhausting SSD write endurance.", "body_md": "[Dev Tools](https://www.devclubhouse.com/c/dev-tools)News\n\n# Codex's TRACE-by-Default Logging Is Quietly Eating SSDs\n\nA global TRACE log sink in OpenAI's CLI can write hundreds of terabytes a year to a local SQLite file — and it's been festering for months.\n\n[Priya Nair](https://www.devclubhouse.com/u/priya_nair)\n\nHere's a failure mode you don't usually associate with an AI coding assistant: it slowly destroys your hardware. According to a [GitHub issue against OpenAI's Codex](https://github.com/openai/codex), the CLI's local feedback-logging database can write on the order of **640 TB per year** to a developer's SSD — quietly, in the background, with no error, warning, or visible disk bloat to tip you off. One reporter clocked **37 TB written in 21 days of uptime**, traced almost entirely to Codex's SQLite logs.\n\nThat's not a corner-case crash. That's a default configuration burning through the warranted write endurance of a consumer drive in well under a year. And the root cause is the kind of one-line decision that any of us could have shipped on a tired Friday.\n\n## A global TRACE default, plus write amplification\n\nThe mechanism is mundane and that's exactly why it's worth your attention. Codex installs a [SQLite](https://www.sqlite.org) feedback-log sink configured with a global default of `Level::TRACE`\n\n:\n\n```\nTargets::new().with_default(Level::TRACE)\n```\n\nThat single `with_default(Level::TRACE)`\n\npersists *everything* — not just Codex's own events, but the TRACE chatter of every dependency in the tree. The reporter's breakdown is brutal: TRACE accounts for ~71% of retained log bytes, and the single noisiest source is raw websocket payload logging from the responses endpoint at 527 MiB. Add mirrored OpenTelemetry events (`codex_otel.log_only`\n\nand `codex_otel.trace_safe`\n\n) and you've explained roughly 96% of the volume before you've logged a single thing a human would ever read.\n\n```\npie title Retained log bytes by level (single-machine sample)\n    \"TRACE\" : 70.7\n    \"INFO\" : 25.7\n    \"DEBUG\" : 3.0\n    \"WARN\" : 0.6\n```\n\nThe content is pure noise. The top TRACE source by frequency is `inotify`\n\nfilesystem events — 128,764 hits for `ld.so.cache`\n\nbeing opened, tens of thousands more for `locale.alias`\n\n, `passwd`\n\n, `nsswitch.conf`\n\n. Underneath that sits tokio-tungstenite's internal `poll_read`\n\n/`poll_next`\n\nplumbing logged line by line. None of this helps debug a coding session; it's the runtime's own breathing, written to disk forever.\n\nBut the retained database size — about 1 GiB — is a lie about the real write volume, and this is the part developers should internalize. In a 15-second window the reporter saw ~36,211 rows inserted while the retained row count stayed flat at 681,774. The max row ID had already passed **five billion**. Codex is running a continuous insert-index-WAL-then-prune cycle: it writes rows, updates indexes, flushes to the write-ahead log, then deletes them to stay under a retention cap. The on-disk footprint looks innocent. The flash underneath is being hammered. That's textbook write amplification, and it's why a 1 GiB database can represent terabytes of physical writes.\n\n[Shadow GPS — know where it is, always Real-time GPS tracking for vehicles, gear and loved ones. No monthly contracts.](https://www.devclubhouse.com/go/ad/12)\n\n## Why \"it's just logs\" is the wrong reflex\n\nIf your instinct is \"logs are cheap,\" that instinct was calibrated on spinning rust. NAND flash doesn't work that way. SSDs are rated in **TBW** — terabytes written — because each cell tolerates a finite number of program/erase cycles. A typical 1 TB consumer drive carries a warranty around **600 TBW**. At 640 TB/year, Codex alone would spend that entire endurance budget in roughly twelve months, equivalent to ~640 full-drive overwrites in a year. Laptops with soldered, non-replaceable SSDs make that an expensive lesson.\n\nThe deeper smell here is architectural, and it's common across the new wave of agentic dev tools: telemetry that was designed for a server fleet gets shipped to run permanently on a developer's laptop. On a backend you'd ship TRACE spans to a sampled collector with retention policies and you'd never blink. Mirror that same firehose into a local embedded database with a tight retention window and you've built a perfect write-amplification engine. OpenTelemetry's own SDK logs showing up *twice* in the top sources — once via `log_only`\n\n, once via `trace_safe`\n\n— is the tell that nobody costed out what \"persist all targets at TRACE\" means on an endpoint device.\n\nThere's also a privacy wrinkle worth flagging: the reporter deliberately *omitted* the raw websocket/SSE payload samples because they can contain private conversation content. So the default behavior isn't only writing terabytes — it's writing your prompts and model responses to an unencrypted local SQLite file by default. OpenAI's own [troubleshooting guidance](https://developers.openai.com/codex/app/troubleshooting) already warns users to scrub logs for sensitive information before sharing them, which is a tacit acknowledgment of what ends up in there.\n\n## What to actually do about it\n\nThis isn't a freshly discovered one-off. A near-identical report (issue #22444) and threads in OpenAI's own developer forum describe the same SQLite trace flood filling disks \"ever since a few versions ago,\" with at least one user maintaining a homegrown bash script to nuke the WAL and kill the process. When multiple independent reporters and an open PR are converging on the same SQLite sink, treat it as a real, reproducible defect rather than one weird machine.\n\nConcrete steps if you run Codex CLI:\n\n**Check whether it's happening to you.** The files live at`~/.codex/logs_2.sqlite`\n\n, plus the`-wal`\n\nand`-shm`\n\nsidecars. A continuously growing WAL while Codex idles is your signal.\n\n```\nls -la ~/.codex/logs_2.sqlite*\n# watch the row id climb over a few seconds\nsqlite3 ~/.codex/logs_2.sqlite 'select max(rowid) from /* your log table */;'\n```\n\n**Watch physical writes, not file size.** The on-disk DB stays ~1 GiB; the damage is in cumulative writes. On Linux, track per-process or per-device write bytes (`iostat`\n\n,`/proc/<pid>/io`\n\n, or your SMART`Total_LBAs_Written`\n\n/`Data Units Written`\n\ncounter via`smartctl -a`\n\n) rather than trusting`du`\n\n.**Don't just** SQLite holds the WAL open; deleting underneath a live process gets you a half-detached file and a confused client. Stop Codex first, then clear the files — which is precisely why frustrated users wrapped it in a script.`rm`\n\nthe DB with Codex running.**Until a fix lands, treat long-lived idle Codex sessions as a liability.** The worst case in these reports is*leaving Codex open*, where the inotify/websocket TRACE stream keeps writing with no active work happening.\n\nThe right upstream fix is the boring one the reporter proposed: stop using a global TRACE default for the persistent sink, raise thresholds for dependency noise (`log`\n\n, `hyper_util`\n\n, tokio-tungstenite, inotify, low-level OTel), and stop persisting full raw protocol payloads — store event summaries instead. That alone reportedly removes ~96% of retained bytes without disabling feedback logs at all. It's a filter-list change, not a redesign.\n\n## The takeaway\n\nThe interesting thing here isn't that a bug shipped — it's how *invisible* it was. No crash, no error, a deceptively small database file, and months of reports before it broke through. That's the new tax on running always-on agentic tooling on your own machine: server-grade observability defaults are landing on endpoint hardware, and the cost shows up not in your logs but in your SMART counters. Until Codex narrows that sink, the move is simple — go look at `~/.codex/`\n\n, watch your drive's lifetime-writes counter, and don't leave the agent idling for days. The fix is trivial; the habit of checking what your tools write to disk is the part worth keeping.\n\n## Sources & further reading\n\n-\n[Codex logging bug may write TBs to local SSDs](https://github.com/openai/codex/issues/28224)— github.com -\n[Troubleshooting – Codex app | OpenAI Developers](https://developers.openai.com/codex/app/troubleshooting)— developers.openai.com -\n[How do I troubleshoot errors or issues when using Codex CLI?](https://milvus.io/ai-quick-reference/how-do-i-troubleshoot-errors-or-issues-when-using-codex-cli)— milvus.io -\n[Disk space exhaustion when leaving Codex open - Codex - OpenAI Developer Community](https://community.openai.com/t/disk-space-exhaustion-when-leaving-codex-open/1383993)— community.openai.com\n\n[Priya Nair](https://www.devclubhouse.com/u/priya_nair)· AI & Developer Experience Writer\n\nPriya covers AI frameworks, developer productivity tooling, and the startup ecosystem across South and Southeast Asia, bringing a researcher's rigour and a practitioner's empathy to every story. She is deeply sceptical of benchmarks and asks hard questions so her readers don't have to.\n\n## Discussion 0\n\nNo comments yet\n\nBe the first to weigh in.", "url": "https://wpnews.pro/news/codex-s-trace-by-default-logging-is-quietly-eating-ssds", "canonical_source": "https://www.devclubhouse.com/a/codexs-trace-by-default-logging-is-quietly-eating-ssds", "published_at": "2026-06-22 10:03:50+00:00", "updated_at": "2026-06-22 10:13:26.443276+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools", "ai-products", "ai-infrastructure"], "entities": ["OpenAI", "Codex", "SQLite", "GitHub", "NAND flash"], "alternates": {"html": "https://wpnews.pro/news/codex-s-trace-by-default-logging-is-quietly-eating-ssds", "markdown": "https://wpnews.pro/news/codex-s-trace-by-default-logging-is-quietly-eating-ssds.md", "text": "https://wpnews.pro/news/codex-s-trace-by-default-logging-is-quietly-eating-ssds.txt", "jsonld": "https://wpnews.pro/news/codex-s-trace-by-default-logging-is-quietly-eating-ssds.jsonld"}}