{"slug": "tokeneff-an-open-source-llm-cost-meter-that-runs-locally", "title": "tokeneff: An Open-Source LLM Cost Meter That Runs Locally", "summary": "A developer released tokeneff, an open-source CLI that runs a local proxy on localhost:7860 to meter LLM API spending in real time, storing usage data in a local SQLite database. The tool distinguishes itself with month-end spend forecasting, separate CNY and USD tracking, and a BYOK mode that keeps keys and request metadata entirely local. It is installable via pip and supports a platform mode routing through the TokenEff gateway for wholesale pricing.", "body_md": "**Most LLM dashboards show you the bill *after* the damage is done.**\n\nYou run a coding agent for an afternoon, ship a feature, and two days later your\n\nOpenAI dashboard says you spent $47. On what? Which model? Which request? You\n\nhave no idea — and by then it's too late to do anything about it.\n\nThis is the problem we kept hitting, so we built **[tokeneff](https://github.com/zangxin75/token-efficiency)** — an open-source CLI that puts a real-time electricity meter on your LLM API spend.\n\n```\n⚡ tokeneff 电表  (CNY)\n\n  今日花费       ¥0.0284\n  本月累计       ¥0.2524\n  月终预测       ~¥0.93 (100% 置信)\n  累计节省       ¥0.0421\n\n  今日模型花费分布\n  deepseek-v4-flash  ¥0.0192   15,797 tok\n  glm-4-flash        ¥0.0092    8,273 tok\n```\n\n`tokeneff` runs a tiny local proxy on `localhost:7860`. You point your LLM client's `base_url` at it, and it:\n\n```\nyour client → tokeneff proxy (localhost:7860) → LLM upstream\n                   ↓\n              local SQLite meter\n```\n\nIt's a transparent pipe. Your requests still go direct to the provider; tokeneff just reads the `usage` counts from the response and keeps a running tab.\n\nThere are a few LLM cost trackers on GitHub already. Here's where tokeneff fits:\n\n| Feature | tokeneff | toktrack | tokencost | LLM-Cost-Guardian | \n|---|---|---|---|---|\n| Capture method | **local proxy** | reads CLI logs | manual / proxy | local proxy | \n| Real-time token count | ✅ | ❌ | ✅ | ✅ | \n| **Month-end forecast** | ✅ | ❌ | ❌ | ❌ | \n| **Dual region / currency** | ✅ | ❌ | ❌ | ❌ | \n| **BYOK + platform dual-mode** | ✅ | ❌ | ❌ | ❌ | \n| Budget alerts | ✅ | ❌ | ✅ | ✅ | \n\nThree things nobody else does:\n\nThe killer feature. Based on your usage trend (weighted: last 7 days × 70% + linear extrapolation × 30%), it predicts what you'll spend by month-end — with a confidence score. Seven days of data and it locks in. \"At this rate, I'll hit $31 by the 31st\" is the kind of number that actually changes behavior.\n\nIf you're a Chinese developer using DeepSeek/GLM/Kimi, your spend is in CNY ¥. If you're using OpenAI/Claude, it's in USD $. Most tools mash these together into one meaningless number. tokeneff tracks them **separately** — CNY spend and USD spend are never mixed.\n\nBYOK mode is 100% free and local — your keys, your upstream, zero markup. But there's also a **platform mode** that routes through the TokenEff gateway (one key, all models, wholesale pricing). The meter shows the savings: official price vs platform price, side by side. You switch when it makes sense, not when you're forced to.\n\nThis matters more than people admit. A cost tracker that sends your request metadata to a cloud service to compute your bill is... not great. tokeneff:\n\n`~/.tokeneff/meter.db` is a SQLite file you can `rm` anytime\n\n```\npip install tokeneff\ntokeneff setup          # pick provider, paste key\ntokeneff start          # starts the proxy on :7860\ntokeneff stats          # see the meter\ntokeneff dashboard      # live TUI (refreshes every 0.5s)\n```\n\nPoint your client at `http://localhost:7860/v1` and start building. Watch the meter while you work — not after.\n\n**Ubuntu 22.04 gotcha:** the default `setuptools` (59.6.0) is too old for PEP 621 and installs as `UNKNOWN-0.0.0`. Run `pip install --user --upgrade \"setuptools>=70\"` first. (Yes, this is in the README too.)\n\nStar ⭐ the [repo](https://github.com/zangxin75/token-efficiency) if this solves a problem for you. Issues and PRs welcome — especially new provider additions.\n\n*Building tokeneff as part of a broader open-source LLM gateway stack. BYOK meter first, gateway later. The meter is and will remain free + local.*", "url": "https://wpnews.pro/news/tokeneff-an-open-source-llm-cost-meter-that-runs-locally", "canonical_source": "https://dev.to/nio_zanghotmailcom_zang/tokeneff-an-open-source-llm-cost-meter-that-runs-locally-5440", "published_at": "2026-09-13 02:10:15+00:00", "updated_at": "2026-09-13 02:56:36.931380+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "large-language-models", "ai-infrastructure", "ai-products"], "entities": ["tokeneff", "TokenEff", "GitHub", "OpenAI", "DeepSeek", "GLM", "Kimi", "SQLite"], "alternates": {"html": "https://wpnews.pro/news/tokeneff-an-open-source-llm-cost-meter-that-runs-locally", "markdown": "https://wpnews.pro/news/tokeneff-an-open-source-llm-cost-meter-that-runs-locally.md", "text": "https://wpnews.pro/news/tokeneff-an-open-source-llm-cost-meter-that-runs-locally.txt", "jsonld": "https://wpnews.pro/news/tokeneff-an-open-source-llm-cost-meter-that-runs-locally.jsonld"}}