{"slug": "databricks-cost-optimizer-audit-spend-with-codex-or-claude-code", "title": "Databricks Cost Optimizer: Audit Spend with Codex or Claude Code", "summary": "Databricks Cost Optimizer, a read-only FinOps toolkit and Agent Skill for Codex and Claude Code, identifies cost spikes and applies only approved optimizations, achieving a 68.5% reduction in daily effective-list-price usage from $343 on August 5 to $108 on August 9 in its first deployment. The toolkit, developed by Kyle Huirev Vision, audits Databricks system tables and inventory APIs without editing warehouses, stopping Apps, pausing Jobs, changing permissions, or creating budgets.", "body_md": "A read-only-first Databricks FinOps toolkit and Agent Skill for finding cost spikes, explaining their workload impact, and applying only explicitly approved optimizations.\n\nIt works in both Codex and Claude Code:\n\n- Codex discovers\n`.agents/skills/optimize-databricks-costs/`\n\n. - Claude Code discovers\n`.claude/skills/optimize-databricks-costs/`\n\n. - Both copies use the same open\n`SKILL.md`\n\nformat and are kept identical by CI.\n\n- SQL polling intervals that repeatedly beat warehouse auto-stop\n- Oversized or continuously active SQL warehouses\n- Always-running Databricks Apps with little demonstrated use\n- Failed, cancelled, duplicate, or canary Job runs\n- High-cost users and service principals, resolved to their real workloads\n- Missing cost-allocation tags and weak budget coverage\n- Savings proposals that silently create freshness, latency, or availability impacts\n\nThe default workflow is read-only. It separates work into four explicit stages:\n\n- Audit current cost and usage.\n- Propose optimizations.\n- Identify every affected workload and obtain approval for the impact.\n- Apply only approved changes, with rollback commands and live verification.\n\nThe bundled audit script uses Databricks system tables and inventory APIs. It does not edit warehouses, stop Apps, pause Jobs, change permissions, or create budgets.\n\n- Python 3.10+\n- Databricks CLI 0.229 or newer; the live test used 0.296\n- A configured Databricks CLI profile\n- Access to a running SQL warehouse\n- Permission to read the relevant\n`system.billing`\n\n,`system.query`\n\n,`system.lakeflow`\n\n, and`system.compute`\n\ntables\n\nAccount-admin access gives the broadest audit, but the toolkit reports whatever the current identity is allowed to see.\n\n```\ngit clone https://github.com/kylehuirevvision/databricks-cost-optimizer.git\ncd databricks-cost-optimizer\ncodex\n```\n\nThen invoke:\n\n```\n$optimize-databricks-costs audit my Databricks account from a cost perspective\n```\n\nCodex loads repository skills from `.agents/skills`\n\naccording to the [official OpenAI skill documentation](https://learn.chatgpt.com/docs/build-skills).\n\n```\ngit clone https://github.com/kylehuirevvision/databricks-cost-optimizer.git\ncd databricks-cost-optimizer\nclaude\n```\n\nThen invoke:\n\n```\n/optimize-databricks-costs audit my Databricks account from a cost perspective\n```\n\nClaude Code loads project skills from `.claude/skills`\n\naccording to the [official Claude Code skill documentation](https://code.claude.com/docs/en/skills).\n\nFind a warehouse ID:\n\n```\ndatabricks warehouses list\n```\n\nRun the read-only audit:\n\n```\npython3 .agents/skills/optimize-databricks-costs/scripts/audit.py \\\n  --warehouse-id YOUR_WAREHOUSE_ID \\\n  --recent-days 14 \\\n  --output-dir reports/latest\n```\n\nUse a non-default CLI profile with `--profile PROFILE`\n\n. Preview every SQL statement without connecting to Databricks with `--dry-run`\n\n.\n\nThe command writes JSON evidence and a starter Markdown summary under `reports/`\n\n. That directory is ignored by Git because reports commonly contain account IDs, emails, query metadata, and resource names.\n\nIf any system-table query fails, the collector still writes the evidence it obtained but exits nonzero and marks the summary incomplete. Missing evidence is never reported as zero cost.\n\n*Sanitized Databricks billing view. August 10 is a partial day and is excluded from the comparison below.*\n\nIn the first account where this workflow was used, completed daily effective-list-price usage fell from **$343 on August 5 to $108 on August 9**, a **68.5% reduction**. SQL usage fell from **$257 to $42**, an **83.7% reduction**, and accounted for most of the measured improvement.\n\nThis is an observed before-and-after result, not a controlled experiment or a universal savings claim. Workload volume can vary, recent billing can be partial or restated, and negotiated invoice or underlying cloud-infrastructure costs may differ.\n\nAn anonymized engagement found tiny cached queries running approximately every 7.5 minutes against a warehouse with a 10-minute auto-stop. The queries performed almost no compute, but their cadence kept the warehouse billable for every hour in a two-week window. After stopping the recurrence, reducing warehouse size, shortening auto-stop, and stopping unused Apps, the following day's cost dropped materially.\n\nSee [the sanitized case study](/kylehuirevvision/databricks-cost-optimizer/blob/main/examples/polling-defeats-auto-stop.md) for the reasoning pattern and caveats.\n\n```\npython3 -m unittest discover -s tests -v\npython3 scripts/sync_skill.py --check\npython3 scripts/check_public_safety.py\n```\n\nMaintainers who have Codex's built-in `skill-creator`\n\nmay additionally run its `quick_validate.py`\n\nagainst both skill directories. Public users do not need that internal helper.\n\nThe audit estimates USD cost using the effective list price in `system.billing.list_prices`\n\n. This is not necessarily the amount on a negotiated invoice. Recent billing records can arrive late or be restated, so equal-period comparisons exclude the current day and a configurable ingestion-lag window by default.", "url": "https://wpnews.pro/news/databricks-cost-optimizer-audit-spend-with-codex-or-claude-code", "canonical_source": "https://github.com/kylehuirevvision/databricks-cost-optimizer", "published_at": "2026-08-10 10:53:21+00:00", "updated_at": "2026-08-10 11:12:27.077902+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "mlops"], "entities": ["Databricks", "Codex", "Claude Code", "Kyle Huirev Vision", "OpenAI", "Anthropic"], "alternates": {"html": "https://wpnews.pro/news/databricks-cost-optimizer-audit-spend-with-codex-or-claude-code", "markdown": "https://wpnews.pro/news/databricks-cost-optimizer-audit-spend-with-codex-or-claude-code.md", "text": "https://wpnews.pro/news/databricks-cost-optimizer-audit-spend-with-codex-or-claude-code.txt", "jsonld": "https://wpnews.pro/news/databricks-cost-optimizer-audit-spend-with-codex-or-claude-code.jsonld"}}