See Where Your AI Agent’s Answer Came From

Atomic Agents added an optional OpenInference integration that lets developers inspect agent and tool operations in Arize Phoenix, demonstrated in a bill-splitting workflow released in the Atomic Agents repository at version v2.12.0. The example runs a planner that produces `(47 * 1.15) / 3`, a calculator that returns `18.0166666666667`, and an answer writer, producing a six-span trace (one CHAIN, two AGENT spans, one TOOL span and two LLM ChatCompletion spans) that shows the actual inputs and outputs at each handoff. The guide pins Phoenix version 19.8.0, run locally via `uvx --python 3.12 --from "arize-phoenix==19.8.0" phoenix serve` on port 6006, and requires Python 3.12, uv and a funded model API account.

See Where Your AI Agent’s Answer Came From A hands-on Atomic Agents and Phoenix guide: run a planner, calculator and answer writer, then inspect the real trace. Kenny Vaneetvelde https://eigenwise.io/about An AI workflow can return a readable answer while leaving you with a question: what happened between the request and that sentence? Which input reached the model? What did it pass to the tool? Did the next agent receive the result you expected? I’ve added an optional OpenInference integration to Atomic Agents https://eigenwise.github.io/atomic-agents/guides/observability.html so you can inspect those steps in Arize Phoenix. Atomic Agents records the agent and tool operations; a matching provider instrumentor records the model calls inside them. You can open a run and follow the actual values through your application. Let’s do that with a small, complete workflow. One agent turns a bill-splitting question into an expression, a calculator evaluates it, and another agent writes the answer. The example is in the Atomic Agents repository https://github.com/Eigenwise/atomic-agents/tree/v2.12.0/atomic-examples/phoenix-tracing . I ran it with a real model and captured the trace below. The Answer, and the Steps Behind It Here’s the question: If 3 friends split a 47 euro bill and add a 15 percent tip, how much does each pay? The planner produced 47 1.15 / 3 . The calculator returned 18.0166666666667 , and the answer writer responded: Including a 15% tip the €47 bill becomes €54.05, so each of the 3 friends pays about €18.02 precisely €18.0167 , or to split exactly have two pay €18.02 and one pay €18.01. That last detail matters: three payments of €18.02 would add up to €54.06. You can check this example independently, which makes it a useful first tracing task. There’s a language interpretation, an ordinary Python tool call, and a final explanation to inspect. A second run gave the rounded share of €18.02 without mentioning how to allocate the remaining cent. The calculation was the same. That gives you a concrete review question for the answer writer: does the wording handle the way the result will be used? The application controls the order of work. Python calls the planner, passes its expression to the calculator, then passes the result to the answer writer. There’s no automatic tool-calling loop in this example. You can set a breakpoint or add a check at each handoff. Phoenix calls the record of one workflow execution a trace . Each recorded operation is a span . This run contains six: CHAIN answer question |-- AGENT planner | -- LLM ChatCompletion |-- TOOL CalculatorTool -- AGENT answer writer -- LLM ChatCompletion I’ve shortened the two agent names here. In Phoenix, their full names include the input and output schema types. This is the application’s execution record: inputs, outputs, calls and timings. It doesn’t expose a model’s private thinking or establish that an answer is correct. Start Phoenix Locally You’ll need Git, Python 3.12, uv https://docs.astral.sh/uv/getting-started/installation/ , and a model API account with available credit. Phoenix runs locally for this guide; model requests still go to the configured provider and have their own costs. Start Phoenix in a separate terminal: uvx --python 3.12 --from "arize-phoenix==19.8.0" phoenix serve Open http://localhost:6006 . Keep that terminal running. This pins the Phoenix version used for the screenshots. Its server environment stays separate from the application’s dependencies. Now get the released example: git clone --depth 1 --branch v2.12.0 https://github.com/Eigenwise/atomic-agents.git aa-phoenix cd aa-phoenix/atomic-examples/phoenix-tracing uv venv --python 3.12 Activate the new environment. On macOS or Linux: source .venv/bin/activate In Windows PowerShell: .venv\Scripts\Activate.ps1 Install the application packages into that environment: uv pip install "atomic-agents observability ==2.12.0" "instructor==1.14.5" "openai==2.54.0" "arize-phoenix-otel==0.17.2" "openinference-instrumentation-openai==0.1.65" "python-dotenv==1.2.4" "sympy==1.14.0" These are the relevant package versions from the captured run. Connect the Model Provider The supplied example uses the OpenAI SDK. For the captured run I pointed that SDK at OpenRouter and requested openai/gpt-5-mini . In phoenix tracing/main.py , replace build client with: php def build client - instructor.Instructor: return instructor.from openai openai.OpenAI api key=os.environ "OPENROUTER API KEY" , base url="https://openrouter.ai/api/v1", max retries=0, timeout=45, Create a .env file in the example directory: OPENROUTER API KEY=your key here OPENAI MODEL=openai/gpt-5-mini PHOENIX COLLECTOR ENDPOINT=http://localhost:6006 Keep the real key out of Git and screenshots. max retries=0 disables retries inside this OpenAI SDK client; Instructor has its own retry behavior around structured output. Two agent calls don’t necessarily mean two provider attempts when something fails. If you use OpenAI directly, retain the original build client , set OPENAI API KEY and use OPENAI MODEL=gpt-5-mini . That is the released example’s default provider route. The screenshots in this guide come from the OpenRouter configuration above. Run the application: python -m phoenix tracing.main It prints the question and the generated answer. In Phoenix, open atomic-agents-example , then the newest answer question trace. For this question, expect a total of €54.05 and a share of about €18.0167 before rounding. The wording can vary between runs. Open the Calculator Step Click CalculatorTool in the trace tree. Its input should contain the expression the planner produced; its output should contain the evaluated result. In my run: {"expression": " 47 1.15 / 3"} {"result": "18.0166666666667"} Then open the first agent span. Its output contains the expression and a short explanation field defined by the example’s schema. Open the final agent span and check its input: the original question, the chosen expression and the calculator’s result should all be there. That gives you concrete places to investigate an unexpected answer. If the expression is wrong, inspect the planner’s input and output. If the expression is right but the computed value is wrong, inspect the tool. If both are right, check what reached the answer writer and how it described the result. The calculator is deliberately a small demonstration tool using SymPy’s sympify https://docs.sympy.org/latest/modules/core.html sympy.core.sympify.sympify . Its prompt asks for arithmetic, but a prompt doesn’t enforce a safe expression language. Don’t expose this evaluator to unrestricted user or model input. Use a restricted parser or a tool with a narrow validated operation schema when adapting the workflow. How the Calls Become One Trace The example enables instrumentation near the top of main.py : python from phoenix.otel import register tracer provider = register project name="atomic-agents-example", auto instrument=True, tracer = tracer provider.get tracer name auto instrument=True discovers the installed OpenInference instrumentors. Atomic Agents supplies the AGENT and TOOL spans. The separately installed OpenAI instrumentor supplies the nested LLM spans because this application uses the OpenAI SDK, including when its base URL points to OpenRouter. Use the instrumentor that matches the SDK your application calls. Installing the Instructor instrumentor as well as a provider instrumentor can create two layers of LLM spans. This setup installs only the OpenAI provider instrumentor. The parent span wraps the sequence of Python calls: with tracer.start as current span "answer question", openinference span kind="chain" as turn: turn.set input question plan = planner.run BasicChatInputSchema chat message=question calculation = calculator.run CalculatorToolInputSchema expression=plan.expression answer = answerer.run CalculationResultSchema question=question, expression=plan.expression, result=calculation.result, turn.set output answer.chat message Without that parent, these sequential agent and tool calls become separate traces. In a larger application, put the parent around the useful unit you want to inspect: answering one request, processing one document, or completing one job. Check the Model Call, Too Under either agent, open ChatCompletion . That span is where the provider instrumentation records the model request and response, model name and available token usage. The two calls in the captured successful trace report openai/gpt-5-mini . The agent span gives you a convenient view of your application’s typed inputs and outputs. The model span lets you inspect the messages and structured-output request sent through the SDK. If a field is missing from an agent’s result, you can compare the requested schema with the provider response instead of inferring what happened from the final sentence. For failures, inspect the span’s Events and status. Exceptions are recorded as errors. Instructor retries can appear as several model spans under one agent, so inspect the children before treating an agent’s duration or call count as a single successful request. Choose What You Record Before Using Real Data This example uses a harmless arithmetic question so the screenshots can show the full inputs and outputs. An application handling customer documents needs a deliberate decision about its trace payloads and where they are sent. The Atomic Agents instrumentor can hide agent and tool input/output values. On the automatic Phoenix path, add these to the same .env file before running: OPENINFERENCE HIDE INPUTS=true OPENINFERENCE HIDE OUTPUTS=true Hidden values are exported as REDACTED . Review your provider instrumentor’s settings too, along with other attributes and events; these switches aren’t a guarantee that every piece of application data has been removed. The observability guide https://eigenwise.github.io/atomic-agents/guides/observability.html covers manual configuration, streaming and the current instrumentation boundaries. For a service, initialize instrumentation once in each worker’s startup path before agent calls, and consider register batch=True so export is batched. Start by tracing one known request and confirm the operations you expect are present; the linked guide explains method overrides and other instrumentation boundaries. Bring the Pattern Into Your Application Pick a workflow you can check independently. Keep the application’s existing order of work, add a parent span around one request, instrument its agent and tool operations, and add the provider instrumentor for the SDK it uses. Run a known input and compare the values at each boundary with the expected result. In the bill example, that means checking €54.05 against the original question, 18.0166666666667 against the tool output, and the three rounded payments against the answer. In your application, those checks might concern which document was retrieved, which account identifier reached a tool, or which fields were passed to the next agent. I’m Kenny, the developer behind Atomic Agents and Eigenwise https://eigenwise.io . Through Eigenwise, I help teams integrate typed AI workflows, tool calls and Phoenix tracing into existing applications so they can inspect what reached each step. If you need help making an agent workflow inspectable in your own application, tell me what you’re building and where you’re getting stuck https://eigenwise.io/contact .