{"slug": "codex-logging-bug-may-write-tbs-to-local-ssds", "title": "Codex logging bug may write TBs to local SSDs", "summary": "A bug in OpenAI's Codex CLI causes SQLite feedback logs to write approximately 640 TB of data per year, potentially consuming the full write endurance of a 1 TB SSD in under a year. The issue, reported on GitHub, shows that TRACE-level logs account for 70.7% of retained bytes, with websocket and telemetry logs contributing an additional 25.3%.", "body_md": "-\n[Notifications](/login?return_to=%2Fopenai%2Fcodex)You must be signed in to change notification settings -\n[Fork 13.7k](/login?return_to=%2Fopenai%2Fcodex)\n\n# Codex SQLite feedback logs can write ~640 TB/year and rapidly consume SSD endurance #28224\n\n[CLIIssues related to the Codex CLI](https://github.com/openai/codex/issues?q=state%3Aopen%20label%3A%22CLI%22)Issues related to the Codex CLI\n\n[bugSomething isn't working](https://github.com/openai/codex/issues?q=state%3Aopen%20label%3A%22bug%22)Something isn't working\n\n[performance](https://github.com/openai/codex/issues?q=state%3Aopen%20label%3A%22performance%22)\n\n## Description\n\n# Codex SQLite feedback logs can write ~640 TB/year and rapidly consume SSD endurance\n\n## Issue\n\nCodex is continuously writing a large amount of data to the local SQLite feedback log database:\n\n`~/.codex/logs_2.sqlite`\n\n`~/.codex/logs_2.sqlite-wal`\n\n`~/.codex/logs_2.sqlite-shm`\n\nOn my machine, after about **21 days of uptime**, the main SSD has written about **37 TB**. Process/file-level checks show Codex SQLite logs are the main continuous writer.\n\nThat extrapolates to roughly **640 TB/year**. On a **1 TB SSD**, that is about **640 full-drive writes per year**. Some consumer SSDs are rated around **600 TBW**, so this could consume roughly a full drive's warranted write endurance in less than a year.\n\n## Evidence\n\nCurrent retained rows in `logs_2.sqlite`\n\n:\n\n| metric | value |\n|---|---|\n| retained rows | 681,774 |\n| estimated retained log content | 1,035.6 MiB |\n\nLevel distribution:\n\n| level | estimated MiB | byte % |\n|---|---|---|\n| TRACE | 732.5 | 70.7% |\n| INFO | 266.5 | 25.7% |\n| DEBUG | 30.6 | 3.0% |\n| WARN | 5.9 | 0.6% |\n\nLargest target+level pairs:\n\n| target | level | estimated MiB |\n|---|---|---|\n`codex_api::endpoint::responses_websocket` |\nTRACE | 527.4 |\n`codex_otel.log_only` |\nINFO | 141.2 |\n`codex_otel.trace_safe` |\nINFO | 121.2 |\n`log` |\nTRACE | 97.4 |\n`codex_client::transport` |\nTRACE | 60.1 |\n`codex_core::stream_events_utils` |\nDEBUG | 27.5 |\n`codex_api::sse::responses` |\nTRACE | 19.1 |\n\nThe top sources are mostly global TRACE logs, mirrored telemetry logs, and raw websocket/SSE payload logging. `TRACE`\n\nalone is about **70.7%** of retained bytes. `codex_otel.log_only`\n\n+ `codex_otel.trace_safe`\n\nadd another **25.3%**. Filtering these categories should remove roughly **96%** of retained log bytes in this sample without fully disabling feedback logs.\n\n## Sanitized examples from the most frequent TRACE source: `target=log`\n\nThese are high-frequency retained samples. Raw websocket/SSE payload bodies are intentionally not included because they may contain private conversation content.\n\n```\n128,764x TRACE log: inotify event: ... mask: OPEN, name: Some(\"ld.so.cache\")\n 37,982x TRACE log: inotify event: ... mask: OPEN, name: Some(\"locale.alias\")\n 23,843x TRACE log: inotify event: ... mask: OPEN, name: Some(\"passwd\")\n  3,639x TRACE log: <tokio-tungstenite checkout>/src/compat.rs:131 AllowStd.with_context\n  3,505x TRACE log: <tokio-tungstenite checkout>/src/lib.rs:245 WebSocketStream.with_context\n  3,362x TRACE log: <tokio-tungstenite checkout>/src/compat.rs:154 Read.read\n  3,356x TRACE log: <tokio-tungstenite checkout>/src/compat.rs:157 Read.with_context read -> poll_read\n  3,230x TRACE log: <tokio-tungstenite checkout>/src/lib.rs:294 Stream.poll_next\n  3,227x TRACE log: <tokio-tungstenite checkout>/src/lib.rs:304 Stream.with_context poll_next -> read()\n  3,213x TRACE log: inotify event: ... mask: OPEN, name: Some(\"nsswitch.conf\")\n  2,001x TRACE log: WouldBlock\n  1,217x TRACE log: Masked: false\n  1,169x TRACE log: Opcode: Data(Text)\n  1,169x TRACE log: First: 11000001\n```\n\n## Sanitized examples from frequent INFO sources\n\nThe dominant INFO sources are mostly repeated OpenTelemetry mirror events. IDs are redacted.\n\n```\n843x INFO codex_client::custom_ca:\n  using system root certificates because no CA override environment variable was selected ...\n\n334x INFO codex_otel.trace_safe:\n  session_loop{thread_id=<redacted>}:submission_dispatch{otel.name=\"op.dispatch.user_input\" submission.id=<redacted> codex.op=\"user_input\"}:turn{otel.name=\"session_task.turn\" thread.id=<redacted> ...}\n\n333x INFO codex_otel.log_only:\n  session_loop{thread_id=<redacted>}:submission_dispatch{otel.name=\"op.dispatch.user_input\" submission.id=<redacted> codex.op=\"user_input\"}:turn{otel.name=\"session_task.turn\" thread.id=<redacted> ...}\n\n332x INFO codex_otel.log_only:\n  session_loop{thread_id=<redacted>}:submission_dispatch{otel.name=\"op.dispatch.user_input_with_turn_context\" submission.id=<redacted> codex.op=\"user_input_with_turn_context\"}:turn{otel.name=\"session_task.turn\" thread.id=<redacted> ...}\n\n332x INFO codex_otel.trace_safe:\n  session_loop{thread_id=<redacted>}:submission_dispatch{otel.name=\"op.dispatch.user_input_with_turn_context\" submission.id=<redacted> codex.op=\"user_input_with_turn_context\"}:turn{otel.name=\"session_task.turn\" thread.id=<redacted> ...}\n```\n\n## Write amplification\n\nThe retained DB size hides the real write volume. In a 15-second sample:\n\n| metric | before | after |\n|---|---|---|\n| retained rows | 681,774 | 681,774 |\n| max row id | 5,003,347,015 | 5,003,383,226 |\n\nAbout **36,211 rows were inserted in 15 seconds**, while retained row count stayed flat. This suggests continuous insert-and-prune write amplification: rows are inserted, indexed, written to WAL, then pruned.\n\n## Likely cause\n\nThe SQLite feedback log sink is installed with a global TRACE default:\n\n```\nTargets::new().with_default(Level::TRACE)\n```\n\nThis persists all targets at TRACE level by default, including dependency/internal logs and large raw protocol payloads.\n\n## Proposed fix\n\nKeep feedback logs enabled, but narrow what is persisted by default:\n\n- Do not use global TRACE for the SQLite feedback log sink.\n- Drop or raise thresholds for low-value dependency noise, especially\n`target=log`\n\n,`hyper_util`\n\n, tokio-tungstenite internals, inotify spam, and low-level OpenTelemetry SDK logs. - Avoid persisting full raw websocket/SSE payloads by default. Store summaries instead: event kind, duration, success/error, token usage, and payload byte length.\n- Avoid persisting mirrored\n`codex_otel.log_only`\n\n/`codex_otel.trace_safe`\n\nevents unless they are explicitly useful for feedback debugging. - Add a global logs DB size/write cap. Per-thread caps are not enough when many threads/processes exist.\n\nAn optional escape hatch such as `sqlite_logs_enabled = false`\n\nwould still be useful, but the main fix should be better default filtering.\n\n## Related issues and discussions\n\n[Excessive SQLite WAL writes during streaming due to TRACE logs ignoring RUST_LOG #17320](https://github.com/openai/codex/issues/17320)[Codex Desktop rapidly grows logs_2.sqlite / WAL during normal active use #24275](https://github.com/openai/codex/issues/24275)[app-server: feedback log sqlite (logs_N.sqlite) grows unbounded — ~0.75 GB/day, no retention/rotation #26374](https://github.com/openai/codex/issues/26374)`logs_2.sqlite-wal`\n\ngrows indefinitely and remains allocated after deletion because stale/suspended Codex TUI processes keep the deleted WAL open #22444[Heavy I/O activity from idle](https://github.com/openai/codex/issues/20563)`codex`\n\nprocesses. #20563[Severe disk I/O / 100% disk active time on Windows WSL2 when using Codex extension / CLI #27020](https://github.com/openai/codex/issues/27020)[goals_1.sqlite write amplification: ~11 MB/s sustained writes (11 GB lifetime) on a 4 KB database #27911](https://github.com/openai/codex/issues/27911)[Codex Desktop becomes unusable on long active threads due to app-server/renderer memory and TRACE log churn #21134](https://github.com/openai/codex/issues/21134)[app-server: source /feedback logs from sqlite at trace level #12969](https://github.com/openai/codex/pull/12969)\n\n## Metadata\n\n## Metadata\n\n### Assignees\n\n### Labels\n\n[CLIIssues related to the Codex CLI](https://github.com/openai/codex/issues?q=state%3Aopen%20label%3A%22CLI%22)Issues related to the Codex CLI\n\n[bugSomething isn't working](https://github.com/openai/codex/issues?q=state%3Aopen%20label%3A%22bug%22)Something isn't working\n\n[performance](https://github.com/openai/codex/issues?q=state%3Aopen%20label%3A%22performance%22)\n\n### Type\n\n### Fields\n\n[Give feedback](https://github.com/orgs/community/discussions/189141)", "url": "https://wpnews.pro/news/codex-logging-bug-may-write-tbs-to-local-ssds", "canonical_source": "https://github.com/openai/codex/issues/28224", "published_at": "2026-06-22 07:30:17+00:00", "updated_at": "2026-06-22 07:39:18.610038+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-products"], "entities": ["OpenAI", "Codex", "SQLite", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/codex-logging-bug-may-write-tbs-to-local-ssds", "markdown": "https://wpnews.pro/news/codex-logging-bug-may-write-tbs-to-local-ssds.md", "text": "https://wpnews.pro/news/codex-logging-bug-may-write-tbs-to-local-ssds.txt", "jsonld": "https://wpnews.pro/news/codex-logging-bug-may-write-tbs-to-local-ssds.jsonld"}}