# Your Analytics Agent Needs a Harness. What Should Be in It? > Source: > Published: 2026-11-13 04:01:08+00:00 This article was drafted with AI assistance. The investigation below uses a fictional dataset and an illustrative event trace. The finance team asks a straightforward question: “Why did gross margin fall last week?” The analytics agent pulls the weekly totals, requests a breakdown by product, and starts a deeper query into supplier costs. That last call times out. The chat says the query stopped. Somewhere behind it, the warehouse may still be running the statement. The first two results present another problem. The weekly totals are complete, but the product tool returned only the first 50 rows. If the agent treats those rows as the entire breakdown, it can name a product as the biggest driver without having looked at every candidate. Both failures happen around the model. The system needs to track what the tools actually did, how much evidence came back, and which claims the answer can support. That is the job of the harness. An analytics harness assembles context, exposes tools, checks permissions, enforces limits, and records the outcome of each step. The model can propose the next query or explain a result. The runtime decides whether the call may proceed and what information about its outcome goes back to the model. A semantic layer supplies the governed definition of gross margin and the dimensions along which it can be analyzed. The harness applies that context to a specific request. It records which definition and periods were used, attaches the caller’s identity, and controls the lifetime of the queries. Cube describes a production analytics harness built around governed measures and tools that expose result limits. [Cube engineering write-up](https://cube.dev/blog/building-an-agentic-analytics-harness) One model and a handful of tools are enough to demonstrate the pattern. A separate evaluation harness runs prepared cases and scores outcomes. You can use it to test the production runtime, including conditions that a model may rarely encounter during a demo. Suppose the first tool returns complete, comparable weekly totals. Gross margin was 28% in the previous closed week and 24% last week. The difference is four percentage points. The product breakdown then returns 50 rows with an explicit truncation flag and no continuation token. The agent can report the aggregate decline. It has insufficient coverage to rank contributors across the full assortment. A useful result contract separates execution status from data coverage. A query can succeed and still produce a limited response. Give the runtime a result reference, scope, units, and coverage indicator. Where pagination exists, return a continuation token. When the tool cannot determine coverage, expose that uncertainty explicitly. The ranking also needs the right calculation. A product’s own margin rate does not measure its contribution to the change in company-wide margin. Product mix matters. A product with a steep decline and tiny sales volume can have less impact than a large product whose rate moved slightly. Before requesting the ranking, agree on a decomposition and reconcile its components to the aggregate. Otherwise, complete rows can still support the wrong conclusion. For this run, the harness can offer a narrower follow-up within the remaining budget. An approved server-side decomposition can return a small set of ranked contributors plus an aggregate remainder, provided the calculation covers the full authorized scope. If the tool cannot provide that, the final answer should preserve the distinction between the confirmed decline and the unfinished attribution. Imagine the cost drill-down starts with an application wait limit of eight seconds. When the deadline arrives, the application stops waiting for that call. The warehouse’s own statement lifetime is a separate concern. Configure an appropriate backend timeout where available and obtain the warehouse query ID as early as the driver allows. Persist the query ID alongside the run and step IDs. If the driver cannot return it before a connection fails, use a supported request identifier or query tag for reconciliation where possible. An ambiguous submission needs its own state. Retrying immediately can start a second expensive query while the first continues. A concrete cancellation mechanism is Snowflake’s query-ID-based cancellation function. Its documentation specifies which roles can cancel a query. The harness should use the cancellation mechanism supported by its backend and reconcile the outcome with a status source it can access. [Snowflake cancellation reference](https://docs.snowflake.com/en/sql-reference/functions/system_cancel_query) Illustrative eventRecorded stateWhat the interface can sayWarehouse accepts cost query and returns an IDRunning“Checking supplier costs.”Application wait expiresTimed out locally“The cost check exceeded its time limit.”Runtime submits a cancellation requestCancellation requested“Cancellation requested.”Backend confirms cancellationCancelled“The warehouse cancelled the cost query.”Status cannot be confirmed within the reconciliation windowOutcome unknown“Cancellation has not yet been confirmed.” A query can also complete before cancellation takes effect. Record that terminal outcome separately. Whether a late result can appear in the answer depends on the run’s deadline and the product’s policy for updating a response. For example, the runtime might retain a result reference for diagnosis and close the original answer without using the late data. Set budgets in the runtime. Start with a run deadline, a per-call wait limit, a maximum number of warehouse statements, and a cap on rows returned to the model. Use warehouse controls for execution time or scan limits where the backend supports them. A client row cap alone can leave the underlying scan just as expensive. Reserve time for a partial answer. If four seconds remain and a tool needs ten, the runtime can close with the verified weekly totals and name the missing evidence. Pass the remaining budget to the model so it can choose among permitted next steps. Enforce the same budget at dispatch so an optimistic suggestion cannot silently extend it. Persist enough state to resume safely after a worker restart. Store the run and step IDs, tool and definition versions, caller scope, budget, query ID, status, and a reference to any completed result. Keep result access bound to the caller and the relevant data snapshot or freshness policy. Avoid copying raw rows or credentials into general-purpose logs. A resumed worker should reconcile in-flight queries before submitting replacements. Reuse a completed result only if its scope and freshness still satisfy the run. Where the backend offers a deduplication key, use it. A logical step ID in your own database is valuable for tracing, but it does not by itself prevent a second warehouse execution. Prepare one scenario with a successful metric call, a truncated breakdown, and a timed-out cost drill-down. Run it against a test double first so you can force the order of events. Then check the cancellation path against a disposable warehouse workload using the actual driver and permissions. Inspect two outputs. In the user-facing answer, verify the metric, comparison period, units, and qualification of incomplete evidence. In the trace, verify the exact query ID targeted for cancellation, the final backend status when available, and whether any retry overlapped with the original call. Also exercise a completion racing with cancellation and a cancellation request denied by permissions. A November 2026 preprint on simulated banking agents provides an example of why this distinction matters. The authors observed episodes where an agent’s message denied an action recorded in its tool trace. The result comes from a banking simulation. For analytics, the corresponding test is to compare the chat’s claim about a stopped query with the warehouse’s recorded outcome. [Banking harness preprint](https://arxiv.org/abs/2611.06116) An acceptance checklist for the example is short. With cancellation still unconfirmed, the answer might read: “Gross margin fell from 28% to 24% across the two closed weeks. The product breakdown was incomplete, so I couldn’t establish the main drivers. The supplier-cost query timed out, and cancellation has not yet been confirmed.” That answer gives finance a number it can use and a clear account of the work still open. To build the first version of your harness, take one workflow your analysts already run. Define what counts as complete evidence, give every warehouse call a traceable identity, and test the path where the last query outlives the chat’s patience. [Your Analytics Agent Needs a Harness. What Should Be in It?](https://pub.towardsai.net/your-analytics-agent-needs-a-harness-what-should-be-in-it-203451b0c275) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.