{"slug": "code-interpreter-internals-in-microsoft-foundry-what-actually-happens-inside", "title": "Code Interpreter Internals in Microsoft Foundry: What Actually Happens Inside That Sandbox", "summary": "A developer's deep dive into Microsoft Foundry's Code Interpreter explains how the Agent Service provisions isolated Python containers, stages files through Azure Storage, and returns results via container_file_citation annotations rather than raw file bytes. The post details three distinct primitives — the CodeInterpreterTool declaration, the running container session, and the toolbox attachment — and warns that failing to manage container references deliberately causes agents to silently lose previously loaded dataframes between calls. It also contrasts the execution paths for prompt agents versus hosted agents built on the Microsoft Agent Framework.", "body_md": "Day 8 of **Microsoft Foundry: 100 Days / 100 Blogs** — a daily deep dive into the Foundry ecosystem for developers building production AI systems.\n\nYour agent just told a user \"I calculated the standard deviation of your Q3 revenue and it's $42,318.\" How? It didn't call a math library you wrote. It didn't hallucinate a number and hope. Somewhere between the model's token stream and that answer, a real Python interpreter spun up in a container you don't manage, ran actual code, and returned a real result.\n\nThat's Code Interpreter — one of the most misunderstood tools in Microsoft Foundry's Agent Service. Most tutorials show you a five-line snippet that uploads a CSV and gets a bar chart back. What they don't show you is what's actually happening in between: container provisioning, session lifecycle, file staging through Azure Storage, execution isolation, and the failure modes that will bite you the first time you put this in front of real users at real scale.\n\nThis post goes under the hood.\n\nLLMs are terrible at arithmetic, terrible at exact string manipulation over large datasets, and terrible at anything requiring deterministic, verifiable execution. Ask GPT-5-class models to compute the factorial of 100 by \"reasoning it out\" in tokens and you'll get a plausible-looking but wrong number more often than you'd like. Ask them to filter a 50,000-row CSV by three conditions and aggregate a column, and you're rolling dice on hallucinated row values.\n\nThe fix predates Foundry — OpenAI shipped it as \"Code Interpreter\" for ChatGPT, and the same pattern shows up as \"the local sandbox tool\" in Anthropic's Claude, in Gemini, and now natively wired into Microsoft Foundry's Agent Service. The idea is simple in principle and hard in implementation: give the model a real, isolated Python runtime it can write to, execute in, read output from, and iterate against — without giving it network access to your infrastructure or persistent state across unrelated conversations.\n\nWhat makes the Foundry implementation worth understanding at a systems level is *how* it wires that sandbox into the rest of the agent runtime — the toolbox model, the container lifecycle, the annotation-based file citation mechanism, and the divergent paths for prompt agents versus hosted agents built on the Microsoft Agent Framework.\n\nThree primitives matter here, and conflating them is the source of most confusion:\n\n`CodeInterpreterTool` in the Python SDK, `CodeInterpreterToolboxTool` when attached via a toolbox) that you attach to an agent definition. This is a declaration of capability, not a running process.\nThis distinction matters because it dictates cost and correctness. If you're building a data-analysis agent that a user comes back to three separate times over an hour with follow-up questions (\"now filter by region,\" \"now compute the median\"), you want those calls landing on the *same* container session so previously-loaded dataframes and generated intermediate files persist. If you're not managing the container reference deliberately, you will silently get a fresh, empty sandbox on every call, and the agent will look inexplicably forgetful about data it \"just\" analyzed.\n\nAt a high level, the request path looks like this:\n\n`code_interpreter` tool in its tool list, decides code execution is the right move and emits a tool call containing Python source.`container_file_citation` annotations pointing at any output files, which your client resolves via the containers API to download the actual bytes.\nThe important architectural detail: **the model doesn't see raw bytes of generated files**. It sees a citation — a `container_id` and `file_id` pair — embedded as an annotation on the output text. Your application code is responsible for walking the response's annotations and calling the containers/files retrieval endpoint to actually pull the PNG or CSV down. This indirection exists because file payloads (a rendered chart, a multi-megabyte CSV) don't belong inline in a token stream; they belong in blob-backed storage with a stable reference.\n\nFoundry gives you two container management strategies:\n\n**Automatic (`AutoCodeInterpreterToolParam`)** — you hand Foundry a list of `file_ids` at agent-definition time (or per-request via structured inputs) and it manages container creation, file staging, and teardown for you. This is what almost every quickstart shows. It's the right default for stateless, single-shot analysis tasks: \"here's a CSV, make me a chart,\" done.\n\n**Explicit container management** — you create and reference a container ID directly, controlling exactly when it's provisioned and reused across multiple turns or multiple agent invocations. This is what you want for multi-turn analytical sessions where a user iterates on the same dataset (\"now group by region,\" \"now export that as JSON\") and you need the dataframe state, intermediate variables, or previously-generated files to persist between calls without re-uploading everything each time.\n\nThe trade-off is exactly what you'd expect from any resource-lifecycle decision: automatic mode is simpler and harder to misuse, but you pay per-call container spin-up costs and lose continuity. Explicit mode gives you continuity and can be cheaper for chatty sessions, but now you own cleanup — an orphaned container that nobody deletes keeps its billable session alive until the one-hour ceiling regardless of whether anyone's using it.\n\nThis is the part that trips people up in production because it spans three different storage boundaries:\n\n`openai.files.create(purpose=\"assistants\", file=...)` against the project's OpenAI-compatible endpoint. This lands the file in Foundry-managed storage, independent of any container — it's a durable, reusable file object referenced by `file_id`.` open()` it like a normal local path.`filename`, and `container_id`.\nTwo failure classes live here. First, forgetting step 5 — treating the citation as if it *were* the file, then wondering why your downstream pipeline received a JSON blob instead of PNG bytes. Second, container lifetime mismatches — if you try to retrieve a file after the container's session has expired (past the 30-minute idle window or the one-hour hard ceiling), the file is gone. There's no persistent, container-independent storage of *generated* outputs unless you explicitly copy them out during the active session — only uploaded *inputs* survive as durable file objects.\n\nFoundry supports Code Interpreter through two structurally different agent shapes, and picking the wrong one for your use case creates unnecessary complexity.\n\n**Prompt agents** are server-side declarative agents you define with `PromptAgentDefinition` and register via `project.agents.create_version(...)`. You attach `CodeInterpreterTool` directly to the definition. Foundry owns the entire execution loop — you send a message, Foundry orchestrates model calls, tool calls, and sandbox execution server-side, and you get a finished response. This is the simpler path and the right default for most agentic data-analysis features.\n\n**Hosted agents**, built with the Microsoft Agent Framework (`Agent`/` FoundryChatClient`), run your orchestration code in-process — you own the agent loop, the framework just gives you a chat client abstraction over the Foundry-hosted model. For these, Code Interpreter isn't attached directly to the agent; it's exposed through a **toolbox** — a versioned, reusable collection of tools published behind an **MCP-compatible endpoint** (`{project_endpoint}/toolboxes/{name}/versions/{version}/mcp`). Your hosted agent connects to that MCP endpoint via `FoundryToolbox`, and the code-execution capability is negotiated over MCP just like any other remote tool (see Day 6 of this series on the Toolbox/MCP pattern).\n\nWhy does this split exist? Toolboxes decouple *tool curation* from *agent code*. A platform team can define a code-interpreter-enabled toolbox once, version it, apply governance (allow-lists, credential scoping) at the toolbox layer, and let a dozen different hosted agents — written by different teams, in different languages, using different orchestration frameworks — consume the exact same governed capability without re-implementing container management logic themselves.\n\n``` python\nimport os\nfrom azure.identity import DefaultAzureCredential\nfrom azure.ai.projects import AIProjectClient\nfrom azure.ai.projects.models import (\n    PromptAgentDefinition,\n    CodeInterpreterTool,\n    AutoCodeInterpreterToolParam,\n)\n\nPROJECT_ENDPOINT = os.environ[\"FOUNDRY_PROJECT_ENDPOINT\"]\n\nproject = AIProjectClient(endpoint=PROJECT_ENDPOINT, credential=DefaultAzureCredential())\nopenai = project.get_openai_client()\n\n# Step 1: upload the input file as a durable, container-independent file object\nwith open(\"quarterly_results.csv\", \"rb\") as f:\n    uploaded = openai.files.create(purpose=\"assistants\", file=f)\n\n# Step 2: declare the agent with Code Interpreter, auto-managed container,\n# and the uploaded file pre-staged for the sandbox\nagent = project.agents.create_version(\n    agent_name=\"finance-analyst\",\n    definition=PromptAgentDefinition(\n        model=\"gpt-5-mini\",\n        instructions=(\n            \"You are a financial analyst. Use Python to compute exact figures — \"\n            \"never estimate arithmetic mentally. Show your work.\"\n        ),\n        tools=[\n            CodeInterpreterTool(\n                container=AutoCodeInterpreterToolParam(file_ids=[uploaded.id])\n            )\n        ],\n    ),\n    description=\"Analyst agent with sandboxed Python execution.\",\n)\n\nconversation = openai.conversations.create()\n\nresponse = openai.responses.create(\n    conversation=conversation.id,\n    input=\"What's the standard deviation of the operating_profit column?\",\n    extra_body={\"agent_reference\": {\"name\": agent.name, \"type\": \"agent_reference\"}},\n)\n\nprint(response.output_text)\n\n# Step 3: walk annotations for any generated artifacts (charts, exports)\nfor item in response.output:\n    if item.type == \"message\":\n        for part in item.content:\n            for ann in getattr(part, \"annotations\", []) or []:\n                if ann.type == \"container_file_citation\":\n                    data = openai.containers.files.content.retrieve(\n                        file_id=ann.file_id, container_id=ann.container_id\n                    )\n                    with open(ann.filename, \"wb\") as out:\n                        out.write(data.read())\n                    print(f\"Downloaded artifact: {ann.filename}\")\n```\n\nNote the instruction line: `\"never estimate arithmetic mentally.\"` This isn't decoration — it's a real behavioral lever. Without an explicit nudge, models frequently answer numeric questions directly from context rather than routing through the tool, especially for \"simple-looking\" arithmetic. If correctness matters, say so in the system instructions.\n\n``` python\nimport asyncio\nimport os\nfrom agent_framework import Agent\nfrom agent_framework.foundry import FoundryChatClient, FoundryToolbox\nfrom azure.identity import AzureCliCredential\nfrom azure.ai.projects import AIProjectClient\nfrom azure.ai.projects.models import CodeInterpreterToolboxTool, AutoCodeInterpreterToolParam\n\nPROJECT_ENDPOINT = os.environ[\"FOUNDRY_PROJECT_ENDPOINT\"]\n\nasync def main() -> None:\n    credential = AzureCliCredential()\n    project = AIProjectClient(endpoint=PROJECT_ENDPOINT, credential=credential)\n    openai = project.get_openai_client()\n\n    with open(\"quarterly_results.csv\", \"rb\") as f:\n        uploaded = openai.files.create(purpose=\"assistants\", file=f)\n\n    # Curate the tool once, as a versioned, governable toolbox\n    toolbox = project.toolboxes.create_version(\n        name=\"analyst-toolbox\",\n        description=\"Sandboxed Python execution for the finance team's agents.\",\n        tools=[\n            CodeInterpreterToolboxTool(\n                container=AutoCodeInterpreterToolParam(file_ids=[uploaded.id])\n            )\n        ],\n    )\n\n    mcp_url = (\n        f\"{PROJECT_ENDPOINT}/toolboxes/{toolbox.name}\"\n        f\"/versions/{toolbox.version}/mcp?api-version=v1\"\n    )\n    toolbox_tool = FoundryToolbox(credential, url=mcp_url)\n\n    agent = Agent(\n        client=FoundryChatClient(credential=credential),\n        instructions=\"You can write and execute Python to answer quantitative questions precisely.\",\n        tools=[toolbox_tool],\n    )\n\n    result = await agent.run(\n        \"Load the uploaded CSV and tell me the standard deviation of operating_profit.\"\n    )\n    print(result.text)\n\nasyncio.run(main())\n```\n\nSame underlying sandbox, same billing model — different ownership boundary. The hosted-agent path is the one to reach for when your orchestration logic (retries, branching, multi-agent handoff) needs to live in your own process rather than inside a Foundry-managed prompt-agent definition.\n\nWalking through the `factorial of 100` example from Microsoft's own hosted-agent sample is instructive because the answer (a 158-digit integer) is unambiguously either correct or wrong — no room for a model to fudge it:\n\n`import math; print(math.factorial(100))`.\nIf the code throws an exception — a `KeyError` because the CSV column name doesn't match what the model assumed, for instance — the traceback comes back as the tool result too, and a capable model will often self-correct on the next turn by inspecting column names first (`df.columns.tolist()`) before retrying the original computation. This iterative repair loop is one of Code Interpreter's most valuable properties in practice, and it's also why you should budget for **multiple tool-call round trips per user question**, not just one, when estimating latency and token cost.\n\nThe sandbox boundary is the whole point, so treat it as a real security control, not a formality:\n\n`file_ids` are visible inside the container. The model cannot browse your Foundry project's other files, other conversations' uploads, or the host filesystem.`\"; import os; os.system(...)`, the Three numbers matter for capacity planning:\n\nDesign implication: if your product's traffic pattern is bursty (e.g., a monthly reporting rush), expect provisioning latency variance during bursts, and don't assume container spin-up time is constant across load levels.\n\nCode Interpreter is billed **separately from token usage** — it's a session-based charge on top of whatever Azure OpenAI/Foundry model tokens the surrounding conversation consumes *(verify current per-session pricing in the Azure pricing calculator before budgeting, as this is a distinct SKU from model inference)*. The practical cost drivers are:\n\nCode Interpreter isn't the only way to get code execution in front of a model, and it isn't always the right one:\n\nCode Interpreter is Foundry's answer to a problem every serious agent eventually hits: language models are fluent but not reliably correct at exact computation. The sandbox — with its explicit container lifecycle, annotation-based file citation model, and split path between prompt agents and MCP-exposed toolboxes for hosted agents — is a genuinely well-thought-out piece of infrastructure once you understand the primitives underneath the five-line quickstart. Get the container lifecycle, file lifecycle, and session economics right, and you get an agent that can be trusted with real arithmetic, real data transformations, and real charts — not just plausible-sounding ones.\n\nIf your agent currently answers numeric or data-heavy questions purely by \"reasoning\" in tokens, that's the tell it's time to wire this in.\n\n*This is Day 8 of Microsoft Foundry: 100 Days / 100 Blogs — a daily series covering the breadth of Microsoft Foundry for developers building real production AI systems. Follow along for the next 92 days.*", "url": "https://wpnews.pro/news/code-interpreter-internals-in-microsoft-foundry-what-actually-happens-inside", "canonical_source": "https://dev.to/monuminu/code-interpreter-internals-in-microsoft-foundry-what-actually-happens-inside-that-sandbox-3ih", "published_at": "2026-09-21 05:35:47+00:00", "updated_at": "2026-09-21 05:52:58.825972+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-infrastructure", "developer-tools", "ai-products"], "entities": ["Microsoft", "Microsoft Foundry", "Agent Service", "Microsoft Agent Framework", "OpenAI", "ChatGPT", "Anthropic", "Claude"], "alternates": {"html": "https://wpnews.pro/news/code-interpreter-internals-in-microsoft-foundry-what-actually-happens-inside", "markdown": "https://wpnews.pro/news/code-interpreter-internals-in-microsoft-foundry-what-actually-happens-inside.md", "text": "https://wpnews.pro/news/code-interpreter-internals-in-microsoft-foundry-what-actually-happens-inside.txt", "jsonld": "https://wpnews.pro/news/code-interpreter-internals-in-microsoft-foundry-what-actually-happens-inside.jsonld"}}