{"slug": "we-had-6-features-2-were-eating-our-budget", "title": "We Had 6 Features. 2 Were Eating Our Budget", "summary": "A B2B SaaS platform discovered that two of its six AI features were consuming 71% of its $4,200 monthly AI budget after implementing per-feature cost attribution. The Compliance Checker and Audit Trail Narrator were silently driving costs due to design flaws, not errors. Fixes reduced combined costs from $2,992 to $500 per month without cutting features or degrading user experience.", "body_md": "Twelve months ago we were burning $4,200/month on AI infrastructure and could not tell you which feature was responsible for a single dollar of it.\n\nThree engineers. Two monitoring tools. One Datadog dashboard. All of them answered the same question: how much.\n\nNobody was answering which feature. Which user. Which service.\n\nThose are the only three numbers that matter when the bill starts climbing.\n\nOur B2B SaaS platform handles enterprise document workflows. Six features hit GPT-4o regularly:\n\n**Contract Analyzer** extracts clauses and flags risk on upload.\n\n**Executive Summary Generator** runs on demand when a user asks for it.\n\n**Smart Search** does semantic search across the document corpus.\n\n**Compliance Checker** runs on every document save.\n\n**Inline Redline Suggester** triggers on text selection.\n\n**Audit Trail Narrator** generates human readable audit logs nightly.\n\nGut ranking for cost: Contract Analyzer first, Smart Search second, everything else minor.\n\nGut ranking for revenue impact: Contract Analyzer and Executive Summary driving most upgrade decisions.\n\nWe were wrong on both. Not slightly wrong. Completely wrong.\n\nWe instrumented every LLM call with feature level, service level, and user level tags. Forty eight hours of data:\n\n```\nFeature                      Monthly Cost    % of Total    Avg Cost/Call\n-------------------------------------------------------------------------\nCompliance Checker           $1,890          45%           $0.087\nAudit Trail Narrator         $1,102          26%           $0.240\nContract Analyzer            $672            16%           $0.310\nExecutive Summary Generator  $294            7%            $0.180\nSmart Search                 $168            4%            $0.021\nInline Redline Suggester     $74             2%            $0.009\n```\n\nContract Analyzer, the one we had spent two weeks optimising, was third.\n\nTwo features we had never once discussed in a cost review were consuming 71% of the budget.\n\nCompliance Checker ran on every document save.\n\nAutosave interval: 30 seconds.\n\n40 active enterprise users. That is 4,800 GPT-4o calls per hour. Every working hour. Every working day. Silently. Perfectly. Expensively.\n\nNo errors. No timeouts. No failed requests. No alert ever fired. Every log looked clean.\n\nThe feature was not broken. The design was. Nothing in our stack could tell the difference between working correctly and working expensively because those look identical at the response level.\n\n**Fix:** Compliance check moved to manual trigger and document submission only.\n\n**Result:** $1,890 to $190/month. One line of code.\n\nThe Audit Trail Narrator ran nightly. Reasonable on paper.\n\nBut it was generating a full GPT-4o narrative for every document that had any activity, including documents touched by automated processes, system integrations, and background jobs.\n\nRoughly 60% of those documents had zero human readers of the audit logs. We were generating prose for an audience that did not exist. Every single night.\n\n**Fix:** Scoped to human triggered activity only. Minimum three human edits before narration runs.\n\n**Result:** $1,102 to $310/month.\n\n**Combined recovery: $2,592/month. No feature cut. No model downgrade. No user impact.**\n\nBoth bugs had an error rate of zero. That is what makes them dangerous. Silent. Working. Expensive.\n\nOnce per feature attribution was running, we rolled it up into cost per user by plan tier.\n\n```\nPlan          Avg Cost to Serve/Month    MRR per Seat    Margin\n----------------------------------------------------------------\nStarter       $8.40                      $149            94%   ✓\nGrowth        $67.00                     $149            55%   ✓\nEnterprise    $198.00                    $149            -33%  ✗\n```\n\nOur Enterprise customers were costing us $49 per seat per month more than they were paying.\n\nThey were our most active users, heavy on Compliance Checker and Audit Trail Narrator. Flat pricing made the loss invisible. Per user attribution made it impossible to ignore.\n\nWe repriced Enterprise to usage based. That conversation with customers was not difficult because the data was exact. Per user, per feature, per month numbers. No estimates. No gut feel. Nothing to argue with.\n\nBeyond feature and user, attribution broke down cost by service, specifically which microservice was originating each LLM call.\n\nOur document-processing-service was making compliance calls that our compliance-service was also making downstream on the same document. We were double billing ourselves on overlapping prompts. Completely invisible until you track at the service layer.\n\nAnother $180/month recovered from a bug with zero user facing symptoms.\n\nThree dimensions. Three different bugs. Any single dimension alone would have missed two of them.\n\nWe had Datadog. We had CloudZero.\n\nCloudZero is built for cloud infrastructure cost allocation. AWS, GCP, Azure broken down by team and resource tag. It does that well. But it has no native AI or LLM tracking. OpenAI, Anthropic, Cohere are not in its data model. You end up back on manual tagging workarounds and provider dashboards that only show totals.\n\nThe gap is not about dashboards or visualisations. It is about where in the stack the data gets captured. We needed instrumentation sitting between our application code and the provider API, tagging every call at the moment it happens with which feature triggered it, which user caused it, and which service originated it. Then surfacing cost per feature, cost per user, and cost per service in one place, with budget alerts that fire before the bill arrives, not after.\n\nThat is the layer standard monitoring tools do not reach.\n\nCan you answer this in under 60 seconds:\n\nWhich feature is your most expensive to run?\n\nWhich users are most expensive to serve?\n\nWhich service is generating the most LLM spend?\n\nIs any of that negative for your unit economics at current pricing?\n\nIf you would have to dig for any of those answers, the risk is not in your monitoring. It is in the layer your monitoring does not reach.\n\nTotal spend is a receipt. Attribution is a map.\n\nWe had the receipt for fourteen months before we got the map.\n\nWe use [CostReveal](https://costreveal.com) for this. The SDK wraps your existing provider calls and tags every call by feature, service, and user. One dashboard surfaces all three dimensions with real time budget alerts and hard limits that stop runaway spend before calls complete. Took one evening to instrument. Both bugs above showed up within 72 hours.\n\nDocs at [docs.costreveal.com](https://docs.costreveal.com) if you want to go straight to setup.\n\nHave you ever found a feature quietly bleeding budget with a perfect error rate? Or discovered a plan tier you were losing money on? Drop it in the comments. Curious how common this pattern actually is.", "url": "https://wpnews.pro/news/we-had-6-features-2-were-eating-our-budget", "canonical_source": "https://dev.to/arpitstack/we-had-6-features-2-were-eating-our-budget-2bph", "published_at": "2026-06-24 12:17:44+00:00", "updated_at": "2026-06-24 12:39:44.597109+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure", "developer-tools", "ai-products"], "entities": ["GPT-4o", "Datadog", "Compliance Checker", "Audit Trail Narrator", "Contract Analyzer", "Executive Summary Generator", "Smart Search", "Inline Redline Suggester"], "alternates": {"html": "https://wpnews.pro/news/we-had-6-features-2-were-eating-our-budget", "markdown": "https://wpnews.pro/news/we-had-6-features-2-were-eating-our-budget.md", "text": "https://wpnews.pro/news/we-had-6-features-2-were-eating-our-budget.txt", "jsonld": "https://wpnews.pro/news/we-had-6-features-2-were-eating-our-budget.jsonld"}}