Code Interpreter Internals in Microsoft Foundry: What Actually Happens Inside That Sandbox

Day 8 of Microsoft Foundry: 100 Days / 100 Blogs — a daily deep dive into the Foundry ecosystem for developers building production AI systems.

Your agent just told a user "I calculated the standard deviation of your Q3 revenue and it's $42,318." How? It didn't call a math library you wrote. It didn't hallucinate a number and hope. Somewhere between the model's token stream and that answer, a real Python interpreter spun up in a container you don't manage, ran actual code, and returned a real result.

That's Code Interpreter — one of the most misunderstood tools in Microsoft Foundry's Agent Service. Most tutorials show you a five-line snippet that uploads a CSV and gets a bar chart back. What they don't show you is what's actually happening in between: container provisioning, session lifecycle, file staging through Azure Storage, execution isolation, and the failure modes that will bite you the first time you put this in front of real users at real scale.

This post goes under the hood.

LLMs are terrible at arithmetic, terrible at exact string manipulation over large datasets, and terrible at anything requiring deterministic, verifiable execution. Ask GPT-5-class models to compute the factorial of 100 by "reasoning it out" in tokens and you'll get a plausible-looking but wrong number more often than you'd like. Ask them to filter a 50,000-row CSV by three conditions and aggregate a column, and you're rolling dice on hallucinated row values.

The fix predates Foundry — OpenAI shipped it as "Code Interpreter" for ChatGPT, and the same pattern shows up as "the local sandbox tool" in Anthropic's Claude, in Gemini, and now natively wired into Microsoft Foundry's Agent Service. The idea is simple in principle and hard in implementation: give the model a real, isolated Python runtime it can write to, execute in, read output from, and iterate against — without giving it network access to your infrastructure or persistent state across unrelated conversations.

What makes the Foundry implementation worth understanding at a systems level is how it wires that sandbox into the rest of the agent runtime — the toolbox model, the container lifecycle, the annotation-based file citation mechanism, and the divergent paths for prompt agents versus hosted agents built on the Microsoft Agent Framework.

Three primitives matter here, and conflating them is the source of most confusion:

CodeInterpreterTool in the Python SDK, CodeInterpreterToolboxTool when attached via a toolbox) that you attach to an agent definition. This is a declaration of capability, not a running process. This distinction matters because it dictates cost and correctness. If you're building a data-analysis agent that a user comes back to three separate times over an hour with follow-up questions ("now filter by region," "now compute the median"), you want those calls landing on the same container session so previously-loaded dataframes and generated intermediate files persist. If you're not managing the container reference deliberately, you will silently get a fresh, empty sandbox on every call, and the agent will look inexplicably forgetful about data it "just" analyzed.

At a high level, the request path looks like this:

code_interpreter tool in its tool list, decides code execution is the right move and emits a tool call containing Python source.container_file_citation annotations pointing at any output files, which your client resolves via the containers API to download the actual bytes. The important architectural detail: the model doesn't see raw bytes of generated files. It sees a citation — a container_id and file_id pair — embedded as an annotation on the output text. Your application code is responsible for walking the response's annotations and calling the containers/files retrieval endpoint to actually pull the PNG or CSV down. This indirection exists because file payloads (a rendered chart, a multi-megabyte CSV) don't belong inline in a token stream; they belong in blob-backed storage with a stable reference.

Foundry gives you two container management strategies:

Automatic (AutoCodeInterpreterToolParam) — you hand Foundry a list of file_ids at agent-definition time (or per-request via structured inputs) and it manages container creation, file staging, and teardown for you. This is what almost every quickstart shows. It's the right default for stateless, single-shot analysis tasks: "here's a CSV, make me a chart," done.

Explicit container management — you create and reference a container ID directly, controlling exactly when it's provisioned and reused across multiple turns or multiple agent invocations. This is what you want for multi-turn analytical sessions where a user iterates on the same dataset ("now group by region," "now export that as JSON") and you need the dataframe state, intermediate variables, or previously-generated files to persist between calls without re-up everything each time.

The trade-off is exactly what you'd expect from any resource-lifecycle decision: automatic mode is simpler and harder to misuse, but you pay per-call container spin-up costs and lose continuity. Explicit mode gives you continuity and can be cheaper for chatty sessions, but now you own cleanup — an orphaned container that nobody deletes keeps its billable session alive until the one-hour ceiling regardless of whether anyone's using it.

This is the part that trips people up in production because it spans three different storage boundaries:

openai.files.create(purpose="assistants", file=...) against the project's OpenAI-compatible endpoint. This lands the file in Foundry-managed storage, independent of any container — it's a durable, reusable file object referenced by file_id. open() it like a normal local path.filename, and container_id. Two failure classes live here. First, forgetting step 5 — treating the citation as if it were the file, then wondering why your downstream pipeline received a JSON blob instead of PNG bytes. Second, container lifetime mismatches — if you try to retrieve a file after the container's session has expired (past the 30-minute idle window or the one-hour hard ceiling), the file is gone. There's no persistent, container-independent storage of generated outputs unless you explicitly copy them out during the active session — only uploaded inputs survive as durable file objects.

Foundry supports Code Interpreter through two structurally different agent shapes, and picking the wrong one for your use case creates unnecessary complexity.

Prompt agents are server-side declarative agents you define with PromptAgentDefinition and register via project.agents.create_version(...). You attach CodeInterpreterTool directly to the definition. Foundry owns the entire execution loop — you send a message, Foundry orchestrates model calls, tool calls, and sandbox execution server-side, and you get a finished response. This is the simpler path and the right default for most agentic data-analysis features.

Hosted agents, built with the Microsoft Agent Framework (Agent/ FoundryChatClient), run your orchestration code in-process — you own the agent loop, the framework just gives you a chat client abstraction over the Foundry-hosted model. For these, Code Interpreter isn't attached directly to the agent; it's exposed through a toolbox — a versioned, reusable collection of tools published behind an MCP-compatible endpoint ({project_endpoint}/toolboxes/{name}/versions/{version}/mcp). Your hosted agent connects to that MCP endpoint via FoundryToolbox, and the code-execution capability is negotiated over MCP just like any other remote tool (see Day 6 of this series on the Toolbox/MCP pattern).

Why does this split exist? Toolboxes decouple tool curation from agent code. A platform team can define a code-interpreter-enabled toolbox once, version it, apply governance (allow-lists, credential scoping) at the toolbox layer, and let a dozen different hosted agents — written by different teams, in different languages, using different orchestration frameworks — consume the exact same governed capability without re-implementing container management logic themselves.

import os
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import (
    PromptAgentDefinition,
    CodeInterpreterTool,
    AutoCodeInterpreterToolParam,
)

PROJECT_ENDPOINT = os.environ["FOUNDRY_PROJECT_ENDPOINT"]

project = AIProjectClient(endpoint=PROJECT_ENDPOINT, credential=DefaultAzureCredential())
openai = project.get_openai_client()

with open("quarterly_results.csv", "rb") as f:
    uploaded = openai.files.create(purpose="assistants", file=f)

agent = project.agents.create_version(
    agent_name="finance-analyst",
    definition=PromptAgentDefinition(
        model="gpt-5-mini",
        instructions=(
            "You are a financial analyst. Use Python to compute exact figures — "
            "never estimate arithmetic mentally. Show your work."
        ),
        tools=[
            CodeInterpreterTool(
                container=AutoCodeInterpreterToolParam(file_ids=[uploaded.id])
            )
        ],
    ),
    description="Analyst agent with sandboxed Python execution.",
)

conversation = openai.conversations.create()

response = openai.responses.create(
    conversation=conversation.id,
    input="What's the standard deviation of the operating_profit column?",
    extra_body={"agent_reference": {"name": agent.name, "type": "agent_reference"}},
)

print(response.output_text)

for item in response.output:
    if item.type == "message":
        for part in item.content:
            for ann in getattr(part, "annotations", []) or []:
                if ann.type == "container_file_citation":
                    data = openai.containers.files.content.retrieve(
                        file_id=ann.file_id, container_id=ann.container_id
                    )
                    with open(ann.filename, "wb") as out:
                        out.write(data.read())
                    print(f"Downloaded artifact: {ann.filename}")

Note the instruction line: "never estimate arithmetic mentally." This isn't decoration — it's a real behavioral lever. Without an explicit nudge, models frequently answer numeric questions directly from context rather than routing through the tool, especially for "simple-looking" arithmetic. If correctness matters, say so in the system instructions.

import asyncio
import os
from agent_framework import Agent
from agent_framework.foundry import FoundryChatClient, FoundryToolbox
from azure.identity import AzureCliCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import CodeInterpreterToolboxTool, AutoCodeInterpreterToolParam

PROJECT_ENDPOINT = os.environ["FOUNDRY_PROJECT_ENDPOINT"]

async def main() -> None:
    credential = AzureCliCredential()
    project = AIProjectClient(endpoint=PROJECT_ENDPOINT, credential=credential)
    openai = project.get_openai_client()

    with open("quarterly_results.csv", "rb") as f:
        uploaded = openai.files.create(purpose="assistants", file=f)

    toolbox = project.toolboxes.create_version(
        name="analyst-toolbox",
        description="Sandboxed Python execution for the finance team's agents.",
        tools=[
            CodeInterpreterToolboxTool(
                container=AutoCodeInterpreterToolParam(file_ids=[uploaded.id])
            )
        ],
    )

    mcp_url = (
        f"{PROJECT_ENDPOINT}/toolboxes/{toolbox.name}"
        f"/versions/{toolbox.version}/mcp?api-version=v1"
    )
    toolbox_tool = FoundryToolbox(credential, url=mcp_url)

    agent = Agent(
        client=FoundryChatClient(credential=credential),
        instructions="You can write and execute Python to answer quantitative questions precisely.",
        tools=[toolbox_tool],
    )

    result = await agent.run(
        "Load the uploaded CSV and tell me the standard deviation of operating_profit."
    )
    print(result.text)

asyncio.run(main())

Same underlying sandbox, same billing model — different ownership boundary. The hosted-agent path is the one to reach for when your orchestration logic (retries, branching, multi-agent handoff) needs to live in your own process rather than inside a Foundry-managed prompt-agent definition.

Walking through the factorial of 100 example from Microsoft's own hosted-agent sample is instructive because the answer (a 158-digit integer) is unambiguously either correct or wrong — no room for a model to fudge it:

import math; print(math.factorial(100)). If the code throws an exception — a KeyError because the CSV column name doesn't match what the model assumed, for instance — the traceback comes back as the tool result too, and a capable model will often self-correct on the next turn by inspecting column names first (df.columns.tolist()) before retrying the original computation. This iterative repair loop is one of Code Interpreter's most valuable properties in practice, and it's also why you should budget for multiple tool-call round trips per user question, not just one, when estimating latency and token cost.

The sandbox boundary is the whole point, so treat it as a real security control, not a formality:

file_ids are visible inside the container. The model cannot browse your Foundry project's other files, other conversations' uploads, or the host filesystem."; import os; os.system(...), the Three numbers matter for capacity planning:

Design implication: if your product's traffic pattern is bursty (e.g., a monthly reporting rush), expect provisioning latency variance during bursts, and don't assume container spin-up time is constant across load levels.

Code Interpreter is billed separately from token usage — it's a session-based charge on top of whatever Azure OpenAI/Foundry model tokens the surrounding conversation consumes (verify current per-session pricing in the Azure pricing calculator before budgeting, as this is a distinct SKU from model inference). The practical cost drivers are:

Code Interpreter isn't the only way to get code execution in front of a model, and it isn't always the right one:

Code Interpreter is Foundry's answer to a problem every serious agent eventually hits: language models are fluent but not reliably correct at exact computation. The sandbox — with its explicit container lifecycle, annotation-based file citation model, and split path between prompt agents and MCP-exposed toolboxes for hosted agents — is a genuinely well-thought-out piece of infrastructure once you understand the primitives underneath the five-line quickstart. Get the container lifecycle, file lifecycle, and session economics right, and you get an agent that can be trusted with real arithmetic, real data transformations, and real charts — not just plausible-sounding ones.

If your agent currently answers numeric or data-heavy questions purely by "reasoning" in tokens, that's the tell it's time to wire this in.

This is Day 8 of Microsoft Foundry: 100 Days / 100 Blogs — a daily series covering the breadth of Microsoft Foundry for developers building real production AI systems. Follow along for the next 92 days.