AI coding tools are getting ridiculously good.
You can open a project, describe a feature, and let an agent inspect files, modify code, run tests, debug failures, and sometimes work for a long time without you touching the keyboard.
That feels amazing.
Until you look at your usage.
Suddenly one small feature has consumed a huge amount of tokens.
And the strange part is that you may have only written a few sentences.
So where did all those tokens go?
The answer is simple:
Your prompt is only a tiny part of what an AI coding agent actually processes.
Imagine you type:
Fix the authentication bug.
That looks like five tokens worth of work.
But the agent may then:
Every one of those steps may add more context.
OpenAI's own documentation says that larger codebases, longer-running tasks, and sessions that require more context consume substantially more usage than simple scripts or functions.
So the real equation is closer to:
Your prompt
+ repository context
+ conversation history
+ tool output
+ generated code
+ test logs
+ repeated agent iterations
= token usage
And that can become very large very quickly.
One of the easiest ways to waste tokens is giving the agent too much context.
Instead of:
Review my project and fix the checkout issue.
Try:
The bug is in the checkout flow.
Start with:
src/features/checkout/
src/api/payments.ts
Do not inspect unrelated folders unless necessary.
You are giving the agent a boundary.
This is especially important in large monorepos.
If your project contains:
frontend/
backend/
mobile/
infrastructure/
docs/
scripts/
legacy/
the agent usually doesn't need everything just to fix one button.
More context does not automatically mean a better answer.
Relevant context is what matters.
Developers often keep one AI conversation alive for hours.
For example:
Build login
↓
Fix dashboard
↓
Create payments
↓
Debug deployment
↓
Optimize database
The problem is that the agent may keep carrying information from earlier work.
Your deployment problem probably doesn't need all the context from the login implementation.
A better workflow is:
Session 1 → Authentication
Session 2 → Payments
Session 3 → Deployment
Treat AI conversations like branches.
When the problem changes significantly, create a clean context.
This is one of the biggest token traps.
Developers regularly paste thousands of lines like:
npm run build
[5000 lines of output]
But somewhere near the bottom the useful information is simply:
TypeError: Cannot read properties of undefined
src/auth/session.ts:82
Give the AI the useful part first.
Build fails with:
TypeError: Cannot read properties of undefined
src/auth/session.ts:82
Here is the surrounding function:
...
If the agent actually needs the full log, it can ask for it or inspect it through tools.
Don't make thousands of irrelevant lines part of the context by default.
Not every coding task needs your most powerful model.
You probably don't need maximum reasoning to:
Reserve expensive models for tasks such as:
A useful mental model is:
Simple task → fast/cheap model
Complex task → stronger model
Using the biggest model for every tiny task is like hiring a senior architect to rename CSS classes.
Bad prompt:
Improve this API.
What does "improve" mean?
The agent may explore architecture, performance, naming, security, validation, documentation and testing.
That means more exploration.
More exploration means more tokens.
Instead:
Optimize this endpoint only for database query count.
Goal:
Reduce the current 8 queries to 3 or fewer.
Do not change the API response format.
Run the existing tests when finished.
Now the search space is much smaller.
The AI knows exactly when it should stop.
Files such as:
AGENTS.md
CLAUDE.md
.cursor/rules/
can be extremely useful.
But developers sometimes turn them into huge documentation dumps.
Remember that persistent instructions can become part of your agent's context repeatedly.
Cursor, for example, explains that applicable rules are included in model context to give the agent persistent guidance.
Instead of writing 5,000 lines of instructions, keep the important rules concise.
Stack:
Next.js + TypeScript + PostgreSQL
Rules:
- Use server components by default
- Use Zod for validation
- Never access DB directly from UI components
- Run npm test before completion
- Do not modify migrations without approval
That's usually far more useful than an enormous internal handbook.
This pattern can get expensive:
Agent writes code
↓
Test fails
↓
Agent changes code
↓
Test fails
↓
Agent changes code
↓
Test fails
↓
...
After a few failures, stop the loop.
Ask:
Stop editing.
Explain why the last three attempts failed.
Identify the root cause before making another change.
This forces the agent back into diagnosis instead of continuing random trial-and-error.
Sometimes you only want to understand a problem.
Don't immediately tell the AI:
Fix it.
Do not modify anything yet.
Inspect the relevant files and explain:
1. the likely cause
2. which files need changes
3. the smallest possible fix
Then approve the implementation.
OpenAI actually recommends beginning some Codex workflows in an "Ask" style before moving into implementation, particularly when understanding the codebase or problem first is useful.
This can prevent the agent from performing an expensive exploration-and-edit loop you never needed.
There is another optimization most developers never think about: prompt caching.
OpenAI explains that caching works best when repeated, static instructions remain at the beginning of a prompt while changing information is placed later.
Conceptually, prefer:
Project rules
Architecture rules
Coding conventions
Task-specific request
Current error
Current file
rather than constantly rewriting your core instructions.
Reusable, stable context is easier for systems to optimize than completely different giant prompts every time.
Build the feature.
I try to give AI something closer to:
Task:
Add password reset.
Scope:
src/features/auth/
src/api/auth/
Requirements:
- Email reset link
- Token expires after 30 minutes
- Existing login behavior must not change
First:
Inspect the relevant files and propose the smallest implementation.
Then:
Implement it.
Finally:
Run the related tests.
Do not inspect unrelated folders unless required.
This doesn't just save tokens.
It usually produces better engineering work.
This is the important part.
The goal shouldn't be:
Use as few tokens as possible.
If an AI agent consumes $5 of compute but saves you three hours of engineering work, that may be an excellent trade.
The real goal is:
Don't spend tokens on context and work that doesn't improve the result.
Agentic coding is moving toward longer, more autonomous tasks. OpenAI reported in 2026 that more than 70% of sampled Codex users had asked it to perform at least one task estimated to exceed an hour of human work, and some heavy users were running many agent tasks in parallel.
That means token efficiency is slowly becoming another engineering skill.
Just like we learned to think about:
CPU
Memory
Database queries
Cloud costs
API requests
developers now also need to think about:
Context
Tokens
Agent loops
Model choice
Tool calls
The best AI developer won't necessarily be the person who uses AI the most.
It may be the developer who knows exactly how much AI is actually needed to solve the problem.
How are you managing token usage in Cursor, Claude Code, Codex, or other coding agents?
I'd be interested to hear what has worked for you.