Code Interpreter Internals in Microsoft Foundry: What Actually Happens Inside That Sandbox

A developer's deep dive into Microsoft Foundry's Code Interpreter explains how the Agent Service provisions isolated Python containers, stages files through Azure Storage, and returns results via container_file_citation annotations rather than raw file bytes. The post details three distinct primitives — the CodeInterpreterTool declaration, the running container session, and the toolbox attachment — and warns that failing to manage container references deliberately causes agents to silently lose previously loaded dataframes between calls. It also contrasts the execution paths for prompt agents versus hosted agents built on the Microsoft Agent Framework.

Day 8 of Microsoft Foundry: 100 Days / 100 Blogs — a daily deep dive into the Foundry ecosystem for developers building production AI systems. Your agent just told a user "I calculated the standard deviation of your Q3 revenue and it's $42,318." How? It didn't call a math library you wrote. It didn't hallucinate a number and hope. Somewhere between the model's token stream and that answer, a real Python interpreter spun up in a container you don't manage, ran actual code, and returned a real result. That's Code Interpreter — one of the most misunderstood tools in Microsoft Foundry's Agent Service. Most tutorials show you a five-line snippet that uploads a CSV and gets a bar chart back. What they don't show you is what's actually happening in between: container provisioning, session lifecycle, file staging through Azure Storage, execution isolation, and the failure modes that will bite you the first time you put this in front of real users at real scale. This post goes under the hood. LLMs are terrible at arithmetic, terrible at exact string manipulation over large datasets, and terrible at anything requiring deterministic, verifiable execution. Ask GPT-5-class models to compute the factorial of 100 by "reasoning it out" in tokens and you'll get a plausible-looking but wrong number more often than you'd like. Ask them to filter a 50,000-row CSV by three conditions and aggregate a column, and you're rolling dice on hallucinated row values. The fix predates Foundry — OpenAI shipped it as "Code Interpreter" for ChatGPT, and the same pattern shows up as "the local sandbox tool" in Anthropic's Claude, in Gemini, and now natively wired into Microsoft Foundry's Agent Service. The idea is simple in principle and hard in implementation: give the model a real, isolated Python runtime it can write to, execute in, read output from, and iterate against — without giving it network access to your infrastructure or persistent state across unrelated conversations. What makes the Foundry implementation worth understanding at a systems level is how it wires that sandbox into the rest of the agent runtime — the toolbox model, the container lifecycle, the annotation-based file citation mechanism, and the divergent paths for prompt agents versus hosted agents built on the Microsoft Agent Framework. Three primitives matter here, and conflating them is the source of most confusion: CodeInterpreterTool in the Python SDK, CodeInterpreterToolboxTool when attached via a toolbox that you attach to an agent definition. This is a declaration of capability, not a running process. This distinction matters because it dictates cost and correctness. If you're building a data-analysis agent that a user comes back to three separate times over an hour with follow-up questions "now filter by region," "now compute the median" , you want those calls landing on the same container session so previously-loaded dataframes and generated intermediate files persist. If you're not managing the container reference deliberately, you will silently get a fresh, empty sandbox on every call, and the agent will look inexplicably forgetful about data it "just" analyzed. At a high level, the request path looks like this: code interpreter tool in its tool list, decides code execution is the right move and emits a tool call containing Python source. container file citation annotations pointing at any output files, which your client resolves via the containers API to download the actual bytes. The important architectural detail: the model doesn't see raw bytes of generated files . It sees a citation — a container id and file id pair — embedded as an annotation on the output text. Your application code is responsible for walking the response's annotations and calling the containers/files retrieval endpoint to actually pull the PNG or CSV down. This indirection exists because file payloads a rendered chart, a multi-megabyte CSV don't belong inline in a token stream; they belong in blob-backed storage with a stable reference. Foundry gives you two container management strategies: Automatic AutoCodeInterpreterToolParam — you hand Foundry a list of file ids at agent-definition time or per-request via structured inputs and it manages container creation, file staging, and teardown for you. This is what almost every quickstart shows. It's the right default for stateless, single-shot analysis tasks: "here's a CSV, make me a chart," done. Explicit container management — you create and reference a container ID directly, controlling exactly when it's provisioned and reused across multiple turns or multiple agent invocations. This is what you want for multi-turn analytical sessions where a user iterates on the same dataset "now group by region," "now export that as JSON" and you need the dataframe state, intermediate variables, or previously-generated files to persist between calls without re-uploading everything each time. The trade-off is exactly what you'd expect from any resource-lifecycle decision: automatic mode is simpler and harder to misuse, but you pay per-call container spin-up costs and lose continuity. Explicit mode gives you continuity and can be cheaper for chatty sessions, but now you own cleanup — an orphaned container that nobody deletes keeps its billable session alive until the one-hour ceiling regardless of whether anyone's using it. This is the part that trips people up in production because it spans three different storage boundaries: openai.files.create purpose="assistants", file=... against the project's OpenAI-compatible endpoint. This lands the file in Foundry-managed storage, independent of any container — it's a durable, reusable file object referenced by file id . open it like a normal local path. filename , and container id . Two failure classes live here. First, forgetting step 5 — treating the citation as if it were the file, then wondering why your downstream pipeline received a JSON blob instead of PNG bytes. Second, container lifetime mismatches — if you try to retrieve a file after the container's session has expired past the 30-minute idle window or the one-hour hard ceiling , the file is gone. There's no persistent, container-independent storage of generated outputs unless you explicitly copy them out during the active session — only uploaded inputs survive as durable file objects. Foundry supports Code Interpreter through two structurally different agent shapes, and picking the wrong one for your use case creates unnecessary complexity. Prompt agents are server-side declarative agents you define with PromptAgentDefinition and register via project.agents.create version ... . You attach CodeInterpreterTool directly to the definition. Foundry owns the entire execution loop — you send a message, Foundry orchestrates model calls, tool calls, and sandbox execution server-side, and you get a finished response. This is the simpler path and the right default for most agentic data-analysis features. Hosted agents , built with the Microsoft Agent Framework Agent / FoundryChatClient , run your orchestration code in-process — you own the agent loop, the framework just gives you a chat client abstraction over the Foundry-hosted model. For these, Code Interpreter isn't attached directly to the agent; it's exposed through a toolbox — a versioned, reusable collection of tools published behind an MCP-compatible endpoint {project endpoint}/toolboxes/{name}/versions/{version}/mcp . Your hosted agent connects to that MCP endpoint via FoundryToolbox , and the code-execution capability is negotiated over MCP just like any other remote tool see Day 6 of this series on the Toolbox/MCP pattern . Why does this split exist? Toolboxes decouple tool curation from agent code . A platform team can define a code-interpreter-enabled toolbox once, version it, apply governance allow-lists, credential scoping at the toolbox layer, and let a dozen different hosted agents — written by different teams, in different languages, using different orchestration frameworks — consume the exact same governed capability without re-implementing container management logic themselves. python import os from azure.identity import DefaultAzureCredential from azure.ai.projects import AIProjectClient from azure.ai.projects.models import PromptAgentDefinition, CodeInterpreterTool, AutoCodeInterpreterToolParam, PROJECT ENDPOINT = os.environ "FOUNDRY PROJECT ENDPOINT" project = AIProjectClient endpoint=PROJECT ENDPOINT, credential=DefaultAzureCredential openai = project.get openai client Step 1: upload the input file as a durable, container-independent file object with open "quarterly results.csv", "rb" as f: uploaded = openai.files.create purpose="assistants", file=f Step 2: declare the agent with Code Interpreter, auto-managed container, and the uploaded file pre-staged for the sandbox agent = project.agents.create version agent name="finance-analyst", definition=PromptAgentDefinition model="gpt-5-mini", instructions= "You are a financial analyst. Use Python to compute exact figures — " "never estimate arithmetic mentally. Show your work." , tools= CodeInterpreterTool container=AutoCodeInterpreterToolParam file ids= uploaded.id , , description="Analyst agent with sandboxed Python execution.", conversation = openai.conversations.create response = openai.responses.create conversation=conversation.id, input="What's the standard deviation of the operating profit column?", extra body={"agent reference": {"name": agent.name, "type": "agent reference"}}, print response.output text Step 3: walk annotations for any generated artifacts charts, exports for item in response.output: if item.type == "message": for part in item.content: for ann in getattr part, "annotations", or : if ann.type == "container file citation": data = openai.containers.files.content.retrieve file id=ann.file id, container id=ann.container id with open ann.filename, "wb" as out: out.write data.read print f"Downloaded artifact: {ann.filename}" Note the instruction line: "never estimate arithmetic mentally." This isn't decoration — it's a real behavioral lever. Without an explicit nudge, models frequently answer numeric questions directly from context rather than routing through the tool, especially for "simple-looking" arithmetic. If correctness matters, say so in the system instructions. python import asyncio import os from agent framework import Agent from agent framework.foundry import FoundryChatClient, FoundryToolbox from azure.identity import AzureCliCredential from azure.ai.projects import AIProjectClient from azure.ai.projects.models import CodeInterpreterToolboxTool, AutoCodeInterpreterToolParam PROJECT ENDPOINT = os.environ "FOUNDRY PROJECT ENDPOINT" async def main - None: credential = AzureCliCredential project = AIProjectClient endpoint=PROJECT ENDPOINT, credential=credential openai = project.get openai client with open "quarterly results.csv", "rb" as f: uploaded = openai.files.create purpose="assistants", file=f Curate the tool once, as a versioned, governable toolbox toolbox = project.toolboxes.create version name="analyst-toolbox", description="Sandboxed Python execution for the finance team's agents.", tools= CodeInterpreterToolboxTool container=AutoCodeInterpreterToolParam file ids= uploaded.id , mcp url = f"{PROJECT ENDPOINT}/toolboxes/{toolbox.name}" f"/versions/{toolbox.version}/mcp?api-version=v1" toolbox tool = FoundryToolbox credential, url=mcp url agent = Agent client=FoundryChatClient credential=credential , instructions="You can write and execute Python to answer quantitative questions precisely.", tools= toolbox tool , result = await agent.run "Load the uploaded CSV and tell me the standard deviation of operating profit." print result.text asyncio.run main Same underlying sandbox, same billing model — different ownership boundary. The hosted-agent path is the one to reach for when your orchestration logic retries, branching, multi-agent handoff needs to live in your own process rather than inside a Foundry-managed prompt-agent definition. Walking through the factorial of 100 example from Microsoft's own hosted-agent sample is instructive because the answer a 158-digit integer is unambiguously either correct or wrong — no room for a model to fudge it: import math; print math.factorial 100 . If the code throws an exception — a KeyError because the CSV column name doesn't match what the model assumed, for instance — the traceback comes back as the tool result too, and a capable model will often self-correct on the next turn by inspecting column names first df.columns.tolist before retrying the original computation. This iterative repair loop is one of Code Interpreter's most valuable properties in practice, and it's also why you should budget for multiple tool-call round trips per user question , not just one, when estimating latency and token cost. The sandbox boundary is the whole point, so treat it as a real security control, not a formality: file ids are visible inside the container. The model cannot browse your Foundry project's other files, other conversations' uploads, or the host filesystem. "; import os; os.system ... , the Three numbers matter for capacity planning: Design implication: if your product's traffic pattern is bursty e.g., a monthly reporting rush , expect provisioning latency variance during bursts, and don't assume container spin-up time is constant across load levels. Code Interpreter is billed separately from token usage — it's a session-based charge on top of whatever Azure OpenAI/Foundry model tokens the surrounding conversation consumes verify current per-session pricing in the Azure pricing calculator before budgeting, as this is a distinct SKU from model inference . The practical cost drivers are: Code Interpreter isn't the only way to get code execution in front of a model, and it isn't always the right one: Code Interpreter is Foundry's answer to a problem every serious agent eventually hits: language models are fluent but not reliably correct at exact computation. The sandbox — with its explicit container lifecycle, annotation-based file citation model, and split path between prompt agents and MCP-exposed toolboxes for hosted agents — is a genuinely well-thought-out piece of infrastructure once you understand the primitives underneath the five-line quickstart. Get the container lifecycle, file lifecycle, and session economics right, and you get an agent that can be trusted with real arithmetic, real data transformations, and real charts — not just plausible-sounding ones. If your agent currently answers numeric or data-heavy questions purely by "reasoning" in tokens, that's the tell it's time to wire this in. This is Day 8 of Microsoft Foundry: 100 Days / 100 Blogs — a daily series covering the breadth of Microsoft Foundry for developers building real production AI systems. Follow along for the next 92 days.