Token Optimization: What We Learned the Hard Way

A developer building an Autonomous SDR Researcher in n8n found that token costs were far higher than expected due to search tool content injection, verbose prompts, and overuse of reasoning models. By implementing prompt compression with role separation, tiered model routing, and context window hygiene, they reduced per-call token counts by roughly half without affecting output quality. The team now displays total ITP-measured cost on product pages to highlight the gap between tool cost and actual cost.

In 2026, token budgets are no longer an afterthought. According to McKinsey's The State of AI in 2024 https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-in-2024-a-year-of-reset-and-opportunity , organizations are increasingly prioritizing API cost optimization as a critical factor in scaling AI deployments, with token reduction becoming central to enterprise AI strategy. We felt this pressure directly when we started building multi-step LLM pipelines in n8n: the first version of our Autonomous SDR Researcher was burning through context windows at a rate that made the unit economics unworkable. The goal was straightforward: build a pipeline that researches leads, drafts outreach, and generates sales collateral without requiring a human to babysit each step. The assumption was that throwing a capable reasoning model at every task would produce the best output. That assumption was wrong, and fixing it taught us more about token efficiency than any documentation ever could. The first failure was invisible until we ran the numbers. We had integrated web search into the research node, which seemed like the obvious move. What we didn't account for: the search tool injects the full retrieved page content into the context window. Each search call was pulling in 30,000 to 40,000 input tokens, billed at the model's per-token rate on top of the search fee itself. I ran the math after our first week of ITP testing. The Autonomous SDR Researcher runs three searches per lead. The search fee was $0.03 per lead. But the token cost from injected content added another $0.06. The search fee was a third of the actual bill. We now show the total ITP-measured cost on every ForgeWorkflows product page, not just the API line item, because that gap between "what the tool costs" and "what the tool actually costs" is where budgets quietly collapse. The second failure was model selection. We routed every task through a single reasoning model: classification, summarization, generation, formatting. A reasoning model is the right tool for synthesis and judgment. It is the wrong tool for deciding whether a string matches a regex pattern. We were paying reasoning-tier prices for tasks a lightweight classification model handles in a fraction of the tokens. Third: our prompts were verbose. We had written them to be thorough, including extensive context, examples, and edge-case instructions in every call. The intent was quality. The result was that we were re-injecting the same 800-token instruction block on every node in a five-step chain. That's 4,000 tokens of repeated overhead per run, none of which changed between executions. Prompt compression with role separation. We split our monolithic prompt into a system prompt injected once, cached and a minimal per-call user message. The system prompt carries the persona, constraints, and output format. The user message carries only the variable input. In n8n, this maps cleanly to the systemMessage and userMessage fields in the LLM node. After restructuring, the per-call token count dropped by roughly half on our generation nodes. The output quality didn't change. The structure forced us to be precise about what the model actually needed to know at each step. Tiered model routing. Not every task needs a reasoning engine. We now route tasks by complexity: a lightweight model handles classification, extraction, and formatting; the reasoning layer handles synthesis, judgment, and anything requiring multi-step inference. The routing logic itself is a simple conditional node in n8n, checking a task type field set earlier in the pipeline. This isn't a novel idea, but most teams don't implement it because it requires upfront work to categorize tasks. That categorization pays for itself quickly. Context window hygiene. Every piece of content injected into a context window should earn its place. We audited each node in the pipeline and asked: does the model need this to produce the right output, or are we including it out of habit? Web search results were the biggest offender. We now extract only the relevant passages before injection, using a lightweight extraction step that strips boilerplate, navigation text, and repeated content. The injected payload shrank from 30,000+ tokens to under 5,000 on most searches. The tradeoff is an extra processing step and occasional extraction errors when page structure is unusual. That's a real cost, and it's worth naming: this approach adds pipeline complexity and a new failure mode. For high-volume pipelines, the savings justify it. For low-volume or one-off builds, it may not. One place where these techniques converge is document generation. Our Sales Playbook Generator https://dev.to/products/sales-playbook-generator produces structured sales collateral from a set of inputs. The build required careful attention to which context the generation node actually needed versus what we were reflexively passing through. The setup guide https://dev.to/blog/sales-playbook-generator-guide walks through how we structured the prompt chain to keep each node's input minimal without losing output coherence. It's a concrete example of tiered routing and prompt compression working together in a single pipeline. Caching is the third lever, and it's underused. If your pipeline calls the same model with the same system prompt repeatedly, many API providers support prompt caching that reduces the per-token rate on cached prefixes. The savings depend on your provider's pricing structure, but the mechanism is the same: identify the stable portion of your prompt, keep it consistent across calls, and let the cache do the work. The failure mode here is subtle: if you modify the system prompt frequently during development, you'll invalidate the cache constantly and see no benefit. Discipline in prompt versioning matters. For teams building in n8n specifically, the GitHub-as-AI-memory pattern https://dev.to/blog/github-as-ai-memory-token-efficient-dev-workflows is worth reading. It addresses a related problem: how to give a pipeline persistent context without re-injecting full history on every run. Instrument before you optimize. We spent time optimizing nodes that weren't the actual cost drivers. A token counter on every node, logging to a simple spreadsheet, would have shown us in day one that the search injection was the dominant expense. We'd build that instrumentation into the pipeline from the start, not add it after the fact when the numbers looked wrong. Treat extraction as a first-class step, not an afterthought. Every time we pull external content into a pipeline, whether from web search, a document, or a database, we'd now build the extraction and filtering step before the generation step, not after. The instinct is to pass everything to the model and let it sort out relevance. That instinct is expensive. A cheap extraction pass that reduces payload size almost always saves more than it costs. Audit the hidden multipliers before shipping. The search fee looked like $0.03. The actual per-lead cost was $0.09. That 3x gap existed because we were measuring the tool fee, not the total token impact. Before any pipeline goes into regular use, we now run a full ITP cost trace that accounts for every token injected at every step, not just the obvious API calls. The full catalog https://dev.to/blueprints reflects this: every build shows the measured total, not the advertised line item.