SokoFlow AI Build Log, Week 4 of 6: what "no tool call" really means, a daily cap with no cron job, and a black box for every question
A shopkeeper asks: "What were my sales yesterday?"
It's a perfectly clear question. The model answered with nothing: no tool call, no error, no explanation.
And the same silence can also mean "hi, how are you?", "what's the weather tomorrow?", and "how did my sales do?" Same output, completely different right responses. That's what Week 4 was about: what SokoFlow does when it can't safely give the user what they asked for.
TL;DR
- A request that wasn't executed isn't necessarily a failed request, and a request that executed doesn't necessarily have data. Collapsing everything into one
ERRORthrows away the information you need to decide what to do.- Zero tool calls is a legitimate outcome, not an error, and the model's own explanatory text isn't trusted to classify it. The Orchestrator produces a structured outcome, and the app decides the response. - Retries are reserved for failures where a second model attempt has a credible chance of fixing things: an unknown tool, more than one tool call, or structurally invalid arguments.
- The per-shop daily cap is a Redis counter whose key changes every day (
shop:123:analytics_count:2026-10-31), so there's nothing to reset and no cron job.- For cost control I chose a query cap plus a per-call token ceiling over a full "reserve and reconcile" budget system. Cost enforcement stays a future design. - Every analytics question now produces one
InteractionLoggerrecord. A 25-phrasing script confirmed the defensive layer handled all of them safely.
SokoFlow is a headless WhatsApp ERP for Kenyan SMEs. I've been building it for about five months. There's no app and no UI to learn; for a shopkeeper, the conversation is the interface. Phase 3 adds one AI feature: natural-language shop analytics. A shopkeeper asks "which products are running low?" or "how does this week compare to last week?", and an LLM, using function calling against SokoFlow's existing services, figures out which query to run.
Last week I integrated analytics into the WhatsApp conversation flow. A message the intent resolver can't classify can now be routed to analytics through a routing policy (covered in last week's post). I also used the existing session to support follow-ups like "And yesterday?", by storing the last analytics call and its arguments and building a new call from that context.
So over three weeks SokoFlow learned to interpret questions, execute tools and keep context across follow-ups. This week was driven by one question:
What happens when the system cannot safely give the user what they asked for?
Failure handling is what separates a convincing demo from a dependable backend.
Picture a request moving through the architecture:
The trap I first fell into: a request that wasn't executed isn't necessarily a failed request, and a request that executed successfully doesn't necessarily have data to return.
Collapsing those cases into one generic ERROR risks losing the information needed to make the right decision. But I also didn't want a special class for every imaginable failure, which would be too much abstraction. The goal was the smallest contract that preserves the distinctions that actually change system behaviour.
I started at the provider boundary, the LLMClient, with a thought experiment. Consider these messages:
| User message | Model response |
|---|---|
| "Hi, how are you?" | No tool call |
| "What's the weather tomorrow?" | No tool call |
| "How did my sales do?" | No tool call |
| "What were my sales yesterday?" | No tool call, despite the request being clear |
All four produce the same observation: zero tool calls. But they potentially need different responses. That led to a decision from an earlier week: a model response with zero tool calls is a legitimate non-execution outcome, not automatically an error.
A second principle came out of it: the absence of a tool call tells us what the model proposed, not why it proposed it.
Among the fields the model returns is an optional reply text. When there's no tool call, that text can say something like "I'm not sure which sales period you mean. Could you clarify?"
Should that text decide the fallback behaviour, or should the application interpret a structured outcome? The decision was simple: model-generated prose isn't authoritative for classifying failures or choosing fallbacks. The Orchestrator detects a zero-tool-call response and produces a structured outcome.
This edge case almost caused feature creep. When the model returns no tool calls, the application can't safely conclude the request is unsupported, because it might be a legitimate analytics question. It also can't safely conclude that clarification is needed. That left two choices:
execute_tool, clarification_needed, unresolved, and so on), and the application validates and maps it to behaviour.
I leaned toward option 2 first. It gives a better user experience. The catch: it adds another contract to validate and another source of model errors, and it was tempting to go down that rabbit hole. It was premature optimization. For reliable, read-only analytics for Kenyan SMEs, option 1 was the better call.
Back in Week 1 I designed a bounded retry that the Orchestrator controls. Retries are useful, since a request can recover from a transient model hiccup. But they cost latency and money, and the second response might behave differently with no guarantee of being better.
So I designed the contract around one rule: retries are reserved for failures where a new model proposal has a credible chance of fixing the problem.
Repairable model proposals (the Orchestrator allows one additional attempt, with repair feedback where appropriate):
No automatic LLM retry:
Those outcomes don't justify automatically asking the model to reinterpret the same request.
SokoFlow has a special disambiguation branch, where the conversation parks in a waiting state while a clarification message goes back to the user. I had to decide how failures interact with it.
Last week I locked in that a pending analytics call exists for tool calls suspended while awaiting product disambiguation. The question this week: is that an ordinary analytics failure that returns a fallback, or does it hand control to the existing FSM disambiguation flow?
What I got wrong first: I had the Orchestrator notice the "ambiguous product" error in the execution result. That hid an assumption about who owns the meaning of product ambiguity.
What I settled on:
So the Orchestrator isn't independently deciding whether a product name is ambiguous. The backend's product lookup returns a domain outcome with multiple candidates, the Orchestrator preserves it, and the analytics handler connects it to the existing disambiguation flow.
Last week's follow-up memory has a catch. A user asks an analytics question, SokoFlow executes it and stores the tool name, validated arguments and timestamp in last_analytics_call. Then, much later, the user sends: "And yesterday?"
That's where the 600-second TTL from last week matters. The application checks last_analytics_call freshness. If it has expired, no prior context goes to the LLM, so the model reads "And yesterday?" as a fresh request. If it proposes no tool call, the Orchestrator returns the structured non-execution outcome and the analytics handler returns a recovery message.
No stale arguments get reused, and the backend never receives a fabricated interpretation.
While the fallbacks were taking shape, one edge case nearly pulled me into building a semantic interpretation system. A follow-up arrives after a sales query:
"What about the other one?"
The context is fresh, so the follow-up is allowed. But "the other one" doesn't clearly identify what should change, and the model proposes a tool call anyway. Should SokoFlow execute it if the arguments pass schema validation, or add another safeguard against semantically unsupported interpretations?
I chose to execute. The goal wasn't to eliminate every interpretation error in application code. A responsible system defines which risks it accepts and which it refuses. And semantic interpretation is exactly what the LLM is good at. Trying to rebuild every conversational reference in deterministic code would turn SokoFlow into a second, inferior language interpreter. I accepted that risk for this MVP.
The user shouldn't know everything that happens inside the system. Take the database being unavailable. Internally, we absolutely want to know: execution failed due to infrastructure. But telling the shopkeeper isn't helpful to them, and it leaks details that are a security risk. So the user gets a generic temporary-failure message.
The risk: LLM calls cost money per token. If a merchant, a bug or an abusive user gets into an endless chat loop, the API bill could spike.
The requirement: a per-shop daily query cap. My budget is realistically close to zero, since I chose a free provider in Week 1. But SokoFlow can change providers without touching the rest of the app, so this deserved real planning.
Two options:
I chose option 1, for two reasons:
I avoided the classic read-modify-write approach, which is vulnerable to race conditions because the modify happens in Python memory, between the read and the write. Instead I used Redis's atomic INCR. Redis runs each command to completion before starting the next, so two concurrent requests can't interleave inside an INCR. The order matters: increment first, then check the value that came back.
Next I had to choose between a fixed and a sliding window. I went with a calendar-aligned fixed window with a daily reset. A sliding window is more robust against bursts at the edge of a window, but our bounded analytics workload doesn't justify a more elaborate algorithm yet.
Why calendar-aligned? Because of a trick that spared me a cron job.
My first idea was a static key like shop:123:analytics_count. But how does it reset at the end of the day? A static key needs something running at midnight to delete it, like a cron job, and I didn't want that complexity.
The fix was to stop thinking about resetting. The key doesn't reset; the code changes which key it looks at every day. On October 31st the code generates:
shop:123:analytics_count:2026-10-31
Redis finds it, increments it, and tracks today's usage. When a merchant messages on November 1st, the date has changed, so the code generates a new key name:
shop:123:analytics_count:2026-11-01
When Redis receives INCR for a key that doesn't exist, it creates it and starts at 1.
One subtle decision. The shop's timezone still defines "today". So the date is in local timezone.
A new key every day raises an infrastructure question: what happens to yesterday's keys? Left alone, abandoned daily records pile up and fill Redis memory indefinitely. When a key is first created, setting a 48-hour TTL acts as an automatic garbage collector. A TTL is relative to the moment of creation—so a 24-hour TTL set at 8:00 AM outlives its active calendar day and expires at 8:00 AM the next morning. However, relying on a tight 24-hour window leaves zero safety margin for clock skew across application servers, Redis, and regional timezones, while also expiring data before you can audit yesterday's metrics during the next work day.
Extending the window to 48 hours ensures yesterday's final counter remains intact for full next-day reporting, logging checks, and debugging before it automatically purges itself from RAM. To ensure a deployment restart or process crash between key creation and TTL assignment doesn't leave an orphaned key with an indefinite lifespan, the initialization check must set or verify the expiration during execution
The number wasn't the hard part. The real question was a single limit shared by every shop, or one configurable per shop. I made it configurable from day one, with a default for the MVP baseline. The default value chosen was 100.
That broke my habit of avoiding extra design complexity, because this one costs almost nothing: fetching the value from the shop profile is cheap, and the Redis logic (current_count > limit) behaves identically. The real advantage is flexibility. If SokoFlow later has tiers, a hardcoded global cap would force a painful refactor of both the Redis middleware and the schema.
Suppose a shop has a daily AI budget of $0.05. Before a provider call, SokoFlow estimates the next call at $0.0001, and the shop has already accumulated $0.0499. Three options:
Option 1 eliminated itself. It fails under concurrency: if two messages hit the LLMClient at the same millisecond, both read the same remaining balance and both proceed, so the shop overspends before either records its usage.
Option 2 is the right design, with a caveat. It's a two-phase check, like how payment gateways handle authorizations:
It needs more Redis or database transaction logic, so I kept it as a future design.
Why option 3 works at this stage:
Because our architecture permits a single self-repair attempt when an initial model proposal fails schema validation, one logical user message can trigger up to two physical provider calls. With our production provider Groq running gpt-oss-20b and a configured max_tokens ceiling of 256, the absolute maximum token exposure per logical interaction is roughly cap × 2 × 256 tokens. For reasoning models like gpt-oss-20b, this max_tokens ceiling restricts both the internal chain-of-thought and the final tool-call payload. While setting this limit too low could theoretically truncate a model mid-reasoning before it outputs a valid tool call, a ceiling of 256 tokens provides ample runway for structured function calling while serving as a hard safety brake against runaway token consumption.
The last component this week is a dedicated logging component, SokoFlow's black box. Every time a merchant sends an analytics query, it captures and persists a snapshot of that transaction. Currently persisted in a JSON file store, with plans to create an actual database table schema for it.The record is written once at the end of the request.
It helps with post-mortem debugging when the LLM misbehaves, and it gives me historical telemetry on latency and cost.
Next week's job is an eval set that measures cost per query, among other metrics. A well-designed logger makes that easier: the raw messages and edge cases it captures can become the test cases for tuning the eval harness.
What exactly counts as one log record? Three options:
I chose Option A:
Option B would split one conversation turn into three or four rows, exploding log volume with verbose telemetry. Option C was appealing in theory, but conditional logging means branching and state inside the logging middleware, which makes error paths harder to test and maintain.
What I got wrong first: I included both correlation_id and interaction_id. That was redundant. A correlation ID is for distributed, async, multi-service systems where one user action spawns background jobs or crosses service boundaries, and SokoFlow isn't that here.
The final blueprint:
Identity: interaction_id, timestamp
Context: shop_ref, raw_query
Outcome: outcome, failure_category
Execution: tool_name, selected_arguments, provider, model,
attempt_count, repair_count
Performance: total_latency_ms
Usage: input_tokens, output_tokens, estimated_cost, usage_status
To check this week's target, I wrote a custom test script that loops through 25 edge-case phrasings, generated by an LLM to represent different ways of triggering a failure inside the system.
It feeds each query to the system, captures the InteractionLogger output, and programmatically checks whether the result was handled safely, meaning it triggered the fallback text or was blocked by a guardrail. It then computes: total safe responses ÷ total queries.
The goal wasn't to test the LLM's routing accuracy. It was to test the defensive code layer. The script hit a 100% success rate.
A fair reading of that number: it shows the defensive layer handles the failure modes I thought of, using phrasings generated by one model. It doesn't say anything about failure modes I didn't think of. Week 5's eval set is where messier, real-shaped input comes in.
Looking back, the same instinct showed up in almost every decision: I reached for the elaborate version first. A constrained model-decision contract for zero tool calls. A reserve-and-reconcile ledger for cost. A correlation_id for a system that doesn't need one. Each time, the simpler option won once I wrote down what the system actually needed. Next time I would start from "what behaviour does this change?" before I design anything.
SokoFlow is taking shape. Failures are diagnosable and the system is more observable. Week 5 is the eval set: 25–40 queries with expected tool and arguments, measured properly for the first time.