# How I Cut My Anthropic API Bill by 50% With a Local Python Tool

> Source: <https://dev.to/saikat_de_4c1cb4fd6050ecb/how-i-cut-my-anthropic-api-bill-by-50-with-a-local-python-tool-pol>
> Published: 2026-05-25 22:13:57+00:00

My Anthropic bill doubled two months in a row. Not because I was building something bigger — because I kept asking the same questions, sending bloated prompts, and defaulting to Sonnet for tasks that Haiku could handle. I built a tool to fix it. Here's how it works.

AI API costs compound fast for three reasons. First, if you're iterating on a project, you ask similar questions repeatedly — "how does X work," "what's wrong with this code" — and pay full price every time. Second, prompts accumulate context: documentation snippets, error traces, boilerplate instructions that add hundreds of tokens but contribute nothing to the answer. Third, most people just use whatever model they defaulted to first. Claude Opus at $15/1M input tokens for a query that Haiku could answer for $1/1M is a 15x cost multiplier on every single call.

I built `ai-cost-optimizer`

— a local CLI that sits between your terminal and the Anthropic API. It runs a semantic cache, a prompt compressor, and a model router on every request before anything hits the network. No cloud, no subscription, no data leaving your machine. Just a Python package you install once.

The cache stores every response as a vector embedding. On each new request, it computes the embedding for your prompt and checks cosine similarity against everything stored. If similarity is above the threshold (default: 0.80), it returns the cached answer — no API call, zero cost.

``` bash
$ aiproxy ask "What is the capital of France?"

  Model         claude-haiku-4-5-20251001
  Cached        No
  Input tokens  18
  Output tokens 9
  Cost          $0.000063
  Cache saved   $0.000000 (cached for next time)
  Total saved   $0.000000

$ aiproxy ask "Capital of France?"

  Model         claude-haiku-4-5-20251001
  Cached        Yes
  Input tokens  0
  Output tokens 0
  Cost          $0.000000
  Cache saved   $0.000063
  Total saved   $0.000063
```

"Capital of France?" is semantically identical to the first query. Cache hit. The API never sees it.

The cache uses `sentence-transformers/all-MiniLM-L6-v2`

for embeddings (80 MB, runs entirely in-process) and `usearch`

for fast ANN lookup. Cold load is ~1.5 seconds on first run; subsequent queries are sub-100ms.

Long prompts are expensive not because they're long, but because most of that length is filler. The compressor uses BM25 to score each sentence by relevance to the query, keeps the top-scoring sentences, and discards the rest.

In plain terms: it reads your prompt, figures out which sentences actually relate to what you're asking, and throws out the ones that don't. No summarization, no LLM — pure lexical scoring, deterministic, fast.

Real example from a documentation query:

```
Original prompt:  370 tokens  ($0.001110 at Sonnet pricing)
Compressed:        61 tokens  ($0.000183 at Sonnet pricing)
Tokens saved:     309 tokens  (83% reduction)
```

The threshold for compression is configurable (`MAX_PROMPT_TOKENS=500`

in `.env`

). Prompts under that limit are sent as-is.

The router classifies each prompt and picks the cheapest model that can handle it. The logic is rule-based: token count, keyword signals for complexity, and a few heuristics for code vs. prose vs. reasoning tasks.

| Query | Routed to | Input cost per 1M tokens |
|---|---|---|
| "What is 2+2?" | Haiku | $1.00 |
| "Explain binary search trees" | Sonnet | $3.00 |
| "Review this system architecture" | Opus | $15.00 |

You can override the router by passing `--model`

explicitly. But if you don't, it defaults to the cheapest model that fits the task, and in practice that means Haiku handles the majority of short factual queries.

After two weeks of normal development use — asking questions about code, debugging errors, generating short snippets:

``` bash
$ aiproxy stats

  Cache entries       14
  Cache hits           3
  Estimated savings   $0.000300
  Total API calls     23
  Total cost          $0.160000
```

The compression savings don't show in `stats`

yet — that's a known gap I'm fixing next.

```
git clone https://github.com/desaikat/ai-cost-optimizer.git
cd ai-cost-optimizer
python -m venv .venv
source .venv/Scripts/activate  # Windows: .venv\Scripts\activate
pip install -e .

cp .env.example .env
# Add your ANTHROPIC_API_KEY to .env

aiproxy ask "What is the difference between a list and a tuple in Python?"
```

There's also a Streamlit dashboard (`aiproxy-dashboard`

) that shows cumulative spend, cache hit rate, model distribution, and compression savings over time.

`.exe`

Repo: [github.com/desaikat/ai-cost-optimizer](https://github.com/desaikat/ai-cost-optimizer)

Two things I'm actively unsure about and would value input on:

Open issues and PRs welcome.