{"slug": "optimizing-ai-workflows-what-i-learned-from-four-text-analysis-trials", "title": "Optimizing AI Workflows: What I Learned from Four Text-Analysis Trials", "summary": "A developer ran four text-analysis trials on the AIDBDeveloper platform to measure how small workflow design choices affect token usage, caching, and cost across several thousand API requests and millions of tokens. The experiment split documents into sentences, tokens, and multi-word terms using GPT 5.6 Sol, GPT 5.4 mini, and GPT 5.6 Terra, and found that more explicit system messages improved cache usage and cut extracted terms from 1,114 to 431, while removing a single instruction to reply with only a full stop measurably raised costs. Tokenization stayed stable across runs, producing exactly 1,650 tokens for one text and 1,762 for the other in both trials.", "body_md": "Complex AI workflows can become expensive surprisingly quickly. Not necessarily because the models themselves are expensive, but because small design decisions are repeated hundreds or thousands of times.\n\nA system message that is too short to benefit from input caching, a model that explains every decision when nobody needs those explanations, a task that gives the model too much freedom, or a single step that gets stuck repeating the same function calls can have a surprisingly large effect once the workflow grows.\n\nI wanted to look at this using a real process rather than a synthetic benchmark.\n\nThe experiment is based on a text-analysis workflow that splits a document into sentences, tokens and multi-word terms, and then performs several syntactic, secondary and free-form classifications. Each trial involved several thousand API requests and several million tokens. I ran four trials using the same two source texts under different conditions, and compared token usage, caching, estimated cost and preliminary result quality.\n\nThe whole experiment was built and executed inside **AIDBDeveloper**, the platform I introduced in a previous article. If you want some background on the application, its modular dashboards and its provider-independent context management, you can find that introduction [here](https://dev.to/miguel_diaz_k26/a-short-introduction-to-my-work-with-ai-3imi).\n\nThe video attached to this article contains the complete walkthrough. What follows is a shorter overview of the experiment and some of the results that I found most interesting.\n\nThe first design decision was not to ask one model to analyse the whole document.\n\nThe application remains responsible for orchestration, data storage and everything that can be done deterministically. AI is used only for the individual tasks that actually require interpretation.\n\nThe process starts by extracting sentences from the source text. For these trials I used GPT 5.6 Sol with low reasoning effort. Each sentence is then split into words, numbers and punctuation marks using GPT 5.4 mini. Multi-word units — which I call *terms* — are extracted with GPT 5.6 Terra at medium reasoning effort, and Terra is also used for the subsequent classification steps. \n\nToken classifications are processed in batches of five, with ten model instances working in parallel on different sentences. Later steps reuse information produced by previous ones whenever that can reduce the model's decision space.\n\nThis was one of the ideas I wanted to test throughout the experiment: instead of asking the model to be as intelligent as possible, try to make each individual task simple enough that it needs as little intelligence as possible.\n\nI used two short articles about logical fallacies that I had written years ago. Each document was processed twice.\n\nThe first trial on TEXT 1 used shorter system messages in an attempt to save input tokens. That turned out not to be particularly successful. Some steps did not use the input cache, term extraction was far too permissive, and the workflow produced many more classifications than necessary.\n\nFor the second trial, the system messages were made considerably more explicit. Cache usage improved substantially and the number of extracted terms and classifications fell.\n\nTEXT 2 was then processed under essentially the same improved configuration. In its second trial, however, I deliberately removed one apparently trivial instruction: after finishing its function calls, the model was no longer told to answer with only a single full stop.\n\nThat provided a useful demonstration of how expensive unnecessary politeness can become.\n\nThe infographic above summarizes some of the structural differences between the four runs.\n\nOne result remained completely stable: tokenization. TEXT 1 produced exactly 1,650 tokens in both trials, while TEXT 2 produced 1,762 in both.\n\nTerm extraction was a different story. TEXT 1 went from **1,114 terms in the first trial to 431 in the second**, after improving the instructions. The total number of classifications also fell from **15,673 to 9,580**.\n\nThat is not simply a token-saving exercise. In the first trial the model was identifying almost any group of words as a term, so the larger result was actually a symptom of poorer analysis.\n\nThe two TEXT 2 trials were much closer structurally because the system messages and classification functions were essentially unchanged.\n\nThe individual source documents were short, but the resulting workload was not.\n\nAcross the relevant trials, total usage ranged from approximately **3 to 8 million tokens per trial**, with roughly **2,000 to 3,000 requests** each. Output remained around 600,000 tokens in most runs, except for the final trial where it rose to roughly 800,000. \n\nThe usage screenshot gives some idea of the scale involved. Once a process reaches this size, looking only at a final API bill is not particularly useful. You need to know which step generated those tokens, which context was used and what the model was doing at that moment.\n\nFor that reason, every model-backed execution instance in my workflow records its configuration, start and end times, input and output data, token usage and a copy of the context that produced the result.\n\nThat made it possible to analyse the workflow internally instead of treating the whole experiment as one large anonymous API workload.\n\nOne of the main goals of the TEXT 1 comparison was to improve input-cache usage.\n\nIt worked.\n\nBetween the two TEXT 1 trials, the estimated cost of uncached input fell by almost **73%**. When all three input-related categories — uncached input, cached input and cache writes — were combined, the reduction was approximately **18%**. \n\nBut something else happened at the same time.\n\nThe output cost also fell by almost **15%**.\n\nThat mattered much more than I initially expected because output tokens represented around **64% of the total estimated cost** in that comparison. The total theoretical cost therefore fell from **$11.39 to $9.59**, approximately **16%** overall. \n\nIn other words, I had been trying to optimize input caching, and caching did improve, but a large part of the final saving actually came from producing less output.\n\nThe chart also shows the two TEXT 2 trials. The last one is deliberately worse, because the model was allowed to generate explanatory responses after completing its function calls.\n\nThat apparently small change was enough to push the estimated cost from **$11.67 to $14.97**.\n\nFunction calling introduces a slightly annoying detail.\n\nAfter the model has finished making all the requested function calls, it normally returns a final assistant message. In an interactive conversation that makes sense. In an automated classification process, it often does not.\n\nThe useful information has already been delivered through the function calls. A final message saying what the model has done is just additional output.\n\nIn most of the trials, the system instructions explicitly told the model to finish with a single full stop.\n\nWhen I removed that instruction in the second TEXT 2 trial, the model began explaining its work again and again. Nobody was going to read those explanations, but they were still generated and billed.\n\nOne classification step gives a particularly clear example: its output increased from roughly **234,000 tokens to 426,000 tokens**.\n\nAlmost twice as much output for no useful additional result.\n\nThat was one of the simplest lessons from the whole experiment: in an automated workflow, a natural-language answer can be pure overhead.\n\nThe second TEXT 2 trial also produced another interesting problem.\n\nDuring one classification step, a model instance became stuck repeatedly calling the same functions for the same batch of five tokens, using exactly the same parameters.\n\nI managed to get it unstuck without stopping the complete process, but its token usage had already increased dramatically.\n\nThe interesting part was that cached-input percentage barely changed. The repeated context was still being cached efficiently; the process was simply repeating far too much work.\n\nThis is a useful reminder that good cache statistics do not necessarily mean that a workflow is efficient.\n\nYou can cache an error very efficiently.\n\nI also added a parameterized cost calculation that allowed me to apply a different model's rates to exactly the same recorded token usage.\n\nAs an experiment, I replaced GPT 5.6 Terra with GPT 6 Astra in the calculation.\n\nThe trials that cost roughly **$10–15** using the actual model mix rose to approximately **$42–65** with Astra pricing — around **4.5 times as much**. \n\nThis is only a cost simulation. It does not imply that Astra would actually consume the same number of tokens or produce identical results.\n\nBut it illustrates the scale of the decision.\n\nChoosing a model substantially more capable than the task requires can outweigh many of the smaller optimizations applied later.\n\nThere is little value in making a workflow cheaper if the results become worse.\n\nThe preliminary quality review produced a very uneven picture.\n\nSentence extraction was extremely consistent. Tokenization was even more stable: it produced identical results across equivalent trials.\n\nWord-level syntactic classification still needs some refinement, but the general quality was reasonably good.\n\nThe harder part was multi-word analysis. Term extraction remained weak, even after improving the instructions. Syntactic classification of terms was considerably poorer than classification of individual words, and the secondary classification of terms was clearly inadequate. The free-form word tags looked more promising, although those are inherently more subjective.\n\nThis changed the optimization question again.\n\nA cheap and stable step is probably not where more engineering effort should go. A step that is expensive and still produces poor results is a much better candidate for redesign.\n\nIn some cases, the right optimization is not a better prompt. It is a different process.\n\nThe original purpose of these trials was mainly to explore context management and caching, but the broader conclusion ended up being more useful.\n\nOptimizing an AI workflow is not simply about reducing token counts.\n\nIt means deciding which work genuinely requires a model, breaking that work into sufficiently small tasks, constraining the model's choices whenever possible, providing known information instead of making it infer it again, removing context that is no longer useful, avoiding unnecessary natural-language output, choosing the least expensive model that can perform each task reliably, and recording enough information to know where the cost is actually coming from.\n\nThe application should do everything it already knows how to do.\n\nThe model should be used for the uncertain parts.\n\nAnd both cost and result quality need to be measured together.\n\nThe video goes much deeper into all of this. It includes the context structure, input caching, function calling and structured responses, the complete workflow, execution logs, examples of actual contexts, detailed token and cost charts, some of the errors I found, and the preliminary quality review.\n\n[Watch the full video on YouTube](https://youtu.be/X3RT8wnQQRE)\n\nThis is still only the preliminary stage of the larger text-analysis experiment. The next work will focus much more heavily on improving and validating the linguistic results themselves.\n\nThanks for reading.", "url": "https://wpnews.pro/news/optimizing-ai-workflows-what-i-learned-from-four-text-analysis-trials", "canonical_source": "https://dev.to/miguel_diaz_k26/optimizing-ai-workflows-what-i-learned-from-four-text-analysis-trials-5f69", "published_at": "2026-09-21 14:31:44+00:00", "updated_at": "2026-09-21 14:55:08.679099+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "natural-language-processing", "ai-infrastructure", "mlops"], "entities": ["AIDBDeveloper", "GPT 5.6 Sol", "GPT 5.4 mini", "GPT 5.6 Terra", "Miguel Diaz"], "alternates": {"html": "https://wpnews.pro/news/optimizing-ai-workflows-what-i-learned-from-four-text-analysis-trials", "markdown": "https://wpnews.pro/news/optimizing-ai-workflows-what-i-learned-from-four-text-analysis-trials.md", "text": "https://wpnews.pro/news/optimizing-ai-workflows-what-i-learned-from-four-text-analysis-trials.txt", "jsonld": "https://wpnews.pro/news/optimizing-ai-workflows-what-i-learned-from-four-text-analysis-trials.jsonld"}}