{"slug": "see-where-your-ai-agents-answer-came-from", "title": "See Where Your AI Agent’s Answer Came From", "summary": "Atomic Agents added an optional OpenInference integration that lets developers inspect agent and tool operations in Arize Phoenix, demonstrated in a bill-splitting workflow released in the Atomic Agents repository at version v2.12.0. The example runs a planner that produces `(47 * 1.15) / 3`, a calculator that returns `18.0166666666667`, and an answer writer, producing a six-span trace (one CHAIN, two AGENT spans, one TOOL span and two LLM ChatCompletion spans) that shows the actual inputs and outputs at each handoff. The guide pins Phoenix version 19.8.0, run locally via `uvx --python 3.12 --from \"arize-phoenix==19.8.0\" phoenix serve` on port 6006, and requires Python 3.12, uv and a funded model API account.", "body_md": "# See Where Your AI Agent’s Answer Came From\n\nA hands-on Atomic Agents and Phoenix guide: run a planner, calculator and answer writer, then inspect the real trace.\n[Kenny Vaneetvelde](https://eigenwise.io/about)\n\nAn AI workflow can return a readable answer while leaving you with a question: **what happened between the request and that sentence?** Which input reached the model? What did it pass to the tool? Did the next agent receive the result you expected?\n\nI’ve added an optional [OpenInference integration to Atomic Agents](https://eigenwise.github.io/atomic-agents/guides/observability.html) so you can inspect those steps in Arize Phoenix. Atomic Agents records the agent and tool operations; a matching provider instrumentor records the model calls inside them. You can open a run and follow the actual values through your application.\n\nLet’s do that with a small, complete workflow. One agent turns a bill-splitting question into an expression, a calculator evaluates it, and another agent writes the answer. The example is in the [Atomic Agents repository](https://github.com/Eigenwise/atomic-agents/tree/v2.12.0/atomic-examples/phoenix-tracing). I ran it with a real model and captured the trace below.\n\n## The Answer, and the Steps Behind It\n\nHere’s the question:\n\nIf 3 friends split a 47 euro bill and add a 15 percent tip, how much does each pay?\n\nThe planner produced `(47 * 1.15) / 3`. The calculator returned `18.0166666666667`, and the answer writer responded:\n\nIncluding a 15% tip the €47 bill becomes €54.05, so each of the 3 friends pays about €18.02 (precisely €18.0167), or to split exactly have two pay €18.02 and one pay €18.01.\n\nThat last detail matters: three payments of €18.02 would add up to €54.06. You can check this example independently, which makes it a useful first tracing task. There’s a language interpretation, an ordinary Python tool call, and a final explanation to inspect.\n\nA second run gave the rounded share of €18.02 without mentioning how to allocate the remaining cent. The calculation was the same. That gives you a concrete review question for the answer writer: does the wording handle the way the result will be used?\n\n**The application controls the order of work.** Python calls the planner, passes its expression to the calculator, then passes the result to the answer writer. There’s no automatic tool-calling loop in this example. You can set a breakpoint or add a check at each handoff.\n\nPhoenix calls the record of one workflow execution a **trace**. Each recorded operation is a **span**. This run contains six:\n\n```\nCHAIN  answer_question\n|-- AGENT  planner\n|   `-- LLM  ChatCompletion\n|-- TOOL   CalculatorTool\n`-- AGENT  answer writer\n    `-- LLM  ChatCompletion\n```\n\nI’ve shortened the two agent names here. In Phoenix, their full names include the input and output schema types. This is the application’s execution record: inputs, outputs, calls and timings. It doesn’t expose a model’s private thinking or establish that an answer is correct.\n\n## Start Phoenix Locally\n\nYou’ll need Git, Python 3.12, [uv](https://docs.astral.sh/uv/getting-started/installation/), and a model API account with available credit. Phoenix runs locally for this guide; model requests still go to the configured provider and have their own costs.\n\nStart Phoenix in a separate terminal:\n\n```\nuvx --python 3.12 --from \"arize-phoenix==19.8.0\" phoenix serve\n```\n\nOpen `http://localhost:6006`. Keep that terminal running. This pins the Phoenix version used for the screenshots. Its server environment stays separate from the application’s dependencies.\n\nNow get the released example:\n\n```\ngit clone --depth 1 --branch v2.12.0 https://github.com/Eigenwise/atomic-agents.git aa-phoenix\ncd aa-phoenix/atomic-examples/phoenix-tracing\nuv venv --python 3.12\n```\n\nActivate the new environment. On macOS or Linux:\n\n```\nsource .venv/bin/activate\n```\n\nIn Windows PowerShell:\n\n```\n.venv\\Scripts\\Activate.ps1\n```\n\nInstall the application packages into that environment:\n\n```\nuv pip install \"atomic-agents[observability]==2.12.0\" \"instructor==1.14.5\" \"openai==2.54.0\" \"arize-phoenix-otel==0.17.2\" \"openinference-instrumentation-openai==0.1.65\" \"python-dotenv==1.2.4\" \"sympy==1.14.0\"\n```\n\nThese are the relevant package versions from the captured run.\n\n## Connect the Model Provider\n\nThe supplied example uses the OpenAI SDK. For the captured run I pointed that SDK at OpenRouter and requested `openai/gpt-5-mini`.\n\nIn `phoenix_tracing/main.py`, replace `build_client()` with:\n\n``` php\ndef build_client() -> instructor.Instructor:\n    return instructor.from_openai(\n        openai.OpenAI(\n            api_key=os.environ[\"OPENROUTER_API_KEY\"],\n            base_url=\"https://openrouter.ai/api/v1\",\n            max_retries=0,\n            timeout=45,\n        )\n    )\n```\n\nCreate a `.env` file in the example directory:\n\n```\nOPENROUTER_API_KEY=your_key_here\nOPENAI_MODEL=openai/gpt-5-mini\nPHOENIX_COLLECTOR_ENDPOINT=http://localhost:6006\n```\n\nKeep the real key out of Git and screenshots. `max_retries=0` disables retries inside this OpenAI SDK client; Instructor has its own retry behavior around structured output. Two agent calls don’t necessarily mean two provider attempts when something fails.\n\nIf you use OpenAI directly, retain the original `build_client()`, set `OPENAI_API_KEY` and use `OPENAI_MODEL=gpt-5-mini`. That is the released example’s default provider route. The screenshots in this guide come from the OpenRouter configuration above.\n\nRun the application:\n\n```\npython -m phoenix_tracing.main\n```\n\nIt prints the question and the generated answer. In Phoenix, open **atomic-agents-example**, then the newest `answer_question` trace. For this question, expect a total of €54.05 and a share of about €18.0167 before rounding. The wording can vary between runs.\n\n## Open the Calculator Step\n\nClick **CalculatorTool** in the trace tree. Its input should contain the expression the planner produced; its output should contain the evaluated result. In my run:\n\n```\n{\"expression\": \"(47 * 1.15) / 3\"}\n{\"result\": \"18.0166666666667\"}\n```\n\nThen open the first agent span. Its output contains the expression and a short explanation field defined by the example’s schema. Open the final agent span and check its input: the original question, the chosen expression and the calculator’s result should all be there.\n\nThat gives you concrete places to investigate an unexpected answer. If the expression is wrong, inspect the planner’s input and output. If the expression is right but the computed value is wrong, inspect the tool. If both are right, check what reached the answer writer and how it described the result.\n\nThe calculator is deliberately a small demonstration tool using SymPy’s [`sympify`](https://docs.sympy.org/latest/modules/core.html#sympy.core.sympify.sympify). Its prompt asks for arithmetic, but a prompt doesn’t enforce a safe expression language. **Don’t expose this evaluator to unrestricted user or model input.** Use a restricted parser or a tool with a narrow validated operation schema when adapting the workflow.\n\n## How the Calls Become One Trace\n\nThe example enables instrumentation near the top of `main.py`:\n\n``` python\nfrom phoenix.otel import register\ntracer_provider = register(\n    project_name=\"atomic-agents-example\",\n    auto_instrument=True,\n)\ntracer = tracer_provider.get_tracer(__name__)\n```\n\n`auto_instrument=True` discovers the installed OpenInference instrumentors. Atomic Agents supplies the **AGENT** and **TOOL** spans. The separately installed OpenAI instrumentor supplies the nested **LLM** spans because this application uses the OpenAI SDK, including when its base URL points to OpenRouter.\n\nUse the instrumentor that matches the SDK your application calls. Installing the Instructor instrumentor as well as a provider instrumentor can create two layers of LLM spans. This setup installs only the OpenAI provider instrumentor.\n\nThe parent span wraps the sequence of Python calls:\n\n```\nwith tracer.start_as_current_span(\n    \"answer_question\", openinference_span_kind=\"chain\"\n) as turn:\n    turn.set_input(question)\n    plan = planner.run(BasicChatInputSchema(chat_message=question))\n    calculation = calculator.run(\n        CalculatorToolInputSchema(expression=plan.expression)\n    )\n    answer = answerer.run(\n        CalculationResultSchema(\n            question=question,\n            expression=plan.expression,\n            result=calculation.result,\n        )\n    )\n    turn.set_output(answer.chat_message)\n```\n\nWithout that parent, these sequential agent and tool calls become separate traces. In a larger application, put the parent around the useful unit you want to inspect: answering one request, processing one document, or completing one job.\n\n## Check the Model Call, Too\n\nUnder either agent, open **ChatCompletion**. That span is where the provider instrumentation records the model request and response, model name and available token usage. The two calls in the captured successful trace report `openai/gpt-5-mini`.\n\nThe agent span gives you a convenient view of your application’s typed inputs and outputs. The model span lets you inspect the messages and structured-output request sent through the SDK. If a field is missing from an agent’s result, you can compare the requested schema with the provider response instead of inferring what happened from the final sentence.\n\nFor failures, inspect the span’s **Events** and status. Exceptions are recorded as errors. Instructor retries can appear as several model spans under one agent, so inspect the children before treating an agent’s duration or call count as a single successful request.\n\n## Choose What You Record Before Using Real Data\n\nThis example uses a harmless arithmetic question so the screenshots can show the full inputs and outputs. An application handling customer documents needs a deliberate decision about its trace payloads and where they are sent.\n\nThe Atomic Agents instrumentor can hide agent and tool input/output values. On the automatic Phoenix path, add these to the same `.env` file before running:\n\n```\nOPENINFERENCE_HIDE_INPUTS=true\nOPENINFERENCE_HIDE_OUTPUTS=true\n```\n\nHidden values are exported as `__REDACTED__`. Review your provider instrumentor’s settings too, along with other attributes and events; these switches aren’t a guarantee that every piece of application data has been removed. The [observability guide](https://eigenwise.github.io/atomic-agents/guides/observability.html) covers manual configuration, streaming and the current instrumentation boundaries.\n\nFor a service, initialize instrumentation once in each worker’s startup path before agent calls, and consider `register(batch=True)` so export is batched. Start by tracing one known request and confirm the operations you expect are present; the linked guide explains method overrides and other instrumentation boundaries.\n\n## Bring the Pattern Into Your Application\n\nPick a workflow you can check independently. Keep the application’s existing order of work, add a parent span around one request, instrument its agent and tool operations, and add the provider instrumentor for the SDK it uses. Run a known input and compare the values at each boundary with the expected result.\n\nIn the bill example, that means checking €54.05 against the original question, `18.0166666666667` against the tool output, and the three rounded payments against the answer. In your application, those checks might concern which document was retrieved, which account identifier reached a tool, or which fields were passed to the next agent.\n\nI’m Kenny, the developer behind **Atomic Agents** and **[Eigenwise](https://eigenwise.io)**. Through Eigenwise, I help teams integrate typed AI workflows, tool calls and Phoenix tracing into existing applications so they can inspect what reached each step. If you need help making an agent workflow inspectable in your own application, [tell me what you’re building and where you’re getting stuck](https://eigenwise.io/contact).", "url": "https://wpnews.pro/news/see-where-your-ai-agents-answer-came-from", "canonical_source": "https://eigenwise.io/writing/see-where-your-ai-agents-answer-came-from", "published_at": "2026-10-19 00:00:00+00:00", "updated_at": "2026-10-19 17:15:19.308927+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "mlops", "large-language-models"], "entities": ["Atomic Agents", "Arize Phoenix", "OpenInference", "Kenny Vaneetvelde", "Eigenwise", "Python"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/see-where-your-ai-agents-answer-came-from", "markdown": "https://wpnews.pro/news/see-where-your-ai-agents-answer-came-from.md", "text": "https://wpnews.pro/news/see-where-your-ai-agents-answer-came-from.txt", "jsonld": "https://wpnews.pro/news/see-where-your-ai-agents-answer-came-from.jsonld"}}