Programmatic Tool Calling

OpenAI introduced Programmatic Tool Calling, a feature that lets a model write and run JavaScript in a fresh, isolated V8 runtime to coordinate its tools in parallel, with loops and conditions, while keeping intermediate results in the hosted runtime. In the Responses API, applications add the programmatic_tool_calling hosted tool and set allowed_callers on each eligible tool, while the Agents API enables the feature by default and manages the agent loop. OpenAI says the runtime supports JavaScript with top-level await but provides no Node.js, package installation, direct network access, general-purpose filesystem, subprocess execution, console, or persistent state between executions, and that Responses API requests support Zero Data Retention workflows without a persistent code-execution container when ZDR is enabled for the organization or project.

Programmatic Tool Calling lets a model write and run JavaScript that coordinates its tools. A program can call tools in parallel, use loops and conditions, and keep intermediate results in the hosted runtime. This is useful when a task needs a sequence of related tool calls or needs to process large tool outputs before returning a result. In the Responses API, your application decides whether Programmatic Tool Calling is available and which eligible tools the model can call directly, from a program, or either way. It continues to run any client-owned tool calls. The Agents API agents-api enables Programmatic Tool Calling by default and manages the agent loop for you. Check the model page https://developers.openai.com/api/docs/models before enabling Programmatic Tool Calling. Understand the runtime environment OpenAI runs each generated program in a fresh, isolated V8 runtime. The runtime supports JavaScript with top-level await , but it does not provide Node.js, package installation, direct network access, a general-purpose filesystem, subprocess execution, a console, or persistent JavaScript state between program executions. Programs can interact with external systems only through tools enabled in the request and can emit output with text ... or image ... . For Responses API requests, Programmatic Tool Calling supports Zero Data Retention ZDR workflows without requiring a persistent code-execution container. ZDR must be enabled for the organization or project; setting store: false enables stateless continuation but does not enable ZDR by itself. Eligibility and retention depend on the complete request, including its model, tools, and third-party services; see data controls https://developers.openai.com/api/docs/guides/your-data . Choose when to use Programmatic Tool Calling Use Programmatic Tool Calling when a stage has predictable control flow and code can return a smaller structured result. Use direct tool calling when one call is sufficient, each result requires fresh model judgment, or the work requires approval or preservation of citations or native artifacts. | Task shape | Recommended mode | |---|---| | A single lookup or action | Use direct tool calling. | | Several results that code can filter, join, rank, remove duplicates from, aggregate, or validate | Use Programmatic Tool Calling when the program can return a smaller structured result. | | Dependent calls with predictable data flow | Use Programmatic Tool Calling when code can derive later arguments and the limits and failure behavior are explicit. | | Adaptive search or semantic evaluation | Use direct tool calling when each result should influence the model’s next decision. | | Writes or approval-sensitive actions | Use direct tool calling by default to preserve a clear authorization boundary. | | Final citation or native artifact validation | Use direct tool calling unless the program preserves the native output and validates every required item. | Configure Programmatic Tool Calling For the Responses API, add the programmatic tool calling hosted tool to the request. Then set allowed callers on each eligible tool that the program can invoke. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 { "type": "function", "name": "get inventory", "description": "Return an object with sku string and available units number .", "parameters": { "type": "object", "properties": { "sku": { "type": "string" } }, "required": "sku" , "additionalProperties": false }, "output schema": { "type": "object", "properties": { "sku": { "type": "string" }, "available units": { "type": "number" } }, "required": "sku", "available units" , "additionalProperties": false }, "allowed callers": "programmatic" }, { "type": "programmatic tool calling" } allowed callers controls how the model can invoke a tool: | Value | Behavior | |---|---| | Omitted or "direct" | The model can call the tool directly. | | "programmatic" | Only code in a program item can call the tool. | | "direct", "programmatic" | The model can call the tool directly or from a program. | parameters describes the function arguments. When a function returns predictable structured data, output schema describes the JSON object encoded in its function call output.output string. Define both so generated JavaScript can use the returned fields reliably. Supported tools The following tool types support allowed callers: "programmatic" : - function and custom - mcp - apply patch - Local and hosted shell - code interpreter For MCP tools, the tool’s require approval policy can pause the program until you approve the call. For OpenAI-hosted tools, review the tool’s data-retention and security guidance before enabling it in a program. Combine with tool search Tool search https://developers.openai.com/api/docs/guides/tools-tool-search runs as a top-level Responses API tool, not from inside generated JavaScript. Function, custom, and MCP tools with defer loading: true are not initially available to a program. After the model loads a matching tool, a later program can invoke it through tools. when its allowed callers includes "programmatic" . An already-running program cannot invoke tool search, so the model must load deferred tools before starting a program that needs them. Guide routing when both modes are available When your application lets the model call a function directly or from a program, assign each route to a specific workflow stage. Generic instructions such as “use Programmatic Tool Calling efficiently” don’t identify the intended boundary. For example: <tool orchestration Use Programmatic Tool Calling for bounded stage using only eligible tools . Run independent calls concurrently when safe. Use only documented tool input and output fields. Process and reduce the intermediate results, then emit exactly program result shape , including the evidence needed for the final answer. Stop when condition is met. Retry transient failures at most R times. Do not repeat completed calls or perform side-effecting actions. If a required result is still missing, return a clear structured failure. Use direct tool calls for semantic judgment, approval, or final validation . </tool orchestration Here is an example of how to use this template: <tool orchestration Use Programmatic Tool Calling to compare inventory with demand for sku 123 using only get inventory and get demand. Run both calls concurrently. Use only documented tool input and output fields. Process and reduce the intermediate results, then emit exactly one JSON object with sku, available units, requested units, and shortage units, where shortage units is max requested units - available units, 0 . Include available units and requested units as evidence for the calculation. Stop when both tool results contain the required fields. Retry transient failures at most 1 time. Do not repeat completed calls or perform side-effecting actions. If a required result is still missing, return a clear structured failure. Use direct tool calls only for approval before any inventory-changing action. </tool orchestration For workflows that need both modes, define one handoff and avoid switching routes or repeating work. If a safe fallback exists, define it once and limit its retries. Understand program response items Each API call still returns the standard Responses API object https://developers.openai.com/api/reference/resources/responses/methods/create . Programmatic Tool Calling doesn’t introduce a separate response envelope. When the model uses Programmatic Tool Calling, the response’s output array can contain: - A program item containing the generated JavaScript, a call id , and an opaque fingerprint used to resume or replay the program. - A function call item made by the program. It has its own call id , which your application uses to return the function result. Its caller.caller id matches the program’s call id . - A program output item containing the program’s final result and status. Its call id matches the program’s call id , and its status is completed or incomplete . These are separate top-level items in response.output ; the caller field records their execution relationship. For example, a program can pause while your application runs get inventory and get demand : 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 { "type": "program", "id": "prog 123", "call id": "call prog 123", "code": "const stock, demand = await Promise.all tools.get inventory { sku: 'sku 123' } , tools.get demand { sku: 'sku 123' } ; text JSON.stringify { sku: stock.sku, available units: stock.available units, requested units: demand.requested units, shortage units: Math.max demand.requested units - stock.available units, 0 } ;", "fingerprint": "opaque replay state" }, { "type": "function call", "id": "fc 123", "call id": "call inventory 123", "name": "get inventory", "arguments": "{\"sku\":\"sku 123\"}", "caller": { "type": "program", "caller id": "call prog 123" } }, { "type": "function call", "id": "fc 456", "call id": "call demand 123", "name": "get demand", "arguments": "{\"sku\":\"sku 123\"}", "caller": { "type": "program", "caller id": "call prog 123" } } These examples show only the relevant items from response.output ; they omit the surrounding standard Responses object. After your application returns the nested function results, a later response can contain the complete program output item: 1 2 3 4 5 6 7{ "type": "program output", "id": "prog out 123", "call id": "call prog 123", "result": "{\"sku\":\"sku 123\",\"available units\":42,\"requested units\":31,\"shortage units\":0}", "status": "completed" } The JSON string in program output.result follows the program result shape from your instructions. The surrounding program output item follows the API contract shown above. These are separate contracts. A final message can arrive with the program output or in a later response, so continue until you receive that message. OpenAI runs the model-generated JavaScript in the hosted runtime. Your application executes returned client-owned function calls; it does not execute the generated JavaScript. Return the function result as a function call output . Copy caller from the function call without changing it. The service uses that value to resume the correct program. Continue after client-owned function calls A program can pause more than once as it reaches client-owned tools. Continue until the response contains a final assistant message: 1. Send the request with the hosted tool and functions that allow programmatic calls. 2. Run every returned client-owned function call. 3. Return each function result with the original call id and caller . 4. Handle an incomplete response before continuing. 5. If the response contains no pending function call items and no final message item, continue from that response. With store: false , replay its output items; for a stored response, use previous response id . 6. Stop when the response contains a final message item. Read response.output text or the message’s refusal content. The following example uses store: false , preserves every response item, and returns each function result to the program: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117import json from openai import OpenAI client = OpenAI model = "gpt-6-astra" def get inventory sku : return {"sku": sku, "available units": 42} def get demand sku : return {"sku": sku, "requested units": 31} implementations = { "get inventory": get inventory, "get demand": get demand, } tools = { "type": "function", "name": "get inventory", "description": "Return an object with sku string and available units number .", "parameters": { "type": "object", "properties": {"sku": {"type": "string"}}, "required": "sku" , "additionalProperties": False, }, "output schema": { "type": "object", "properties": { "sku": {"type": "string"}, "available units": {"type": "number"}, }, "required": "sku", "available units" , "additionalProperties": False, }, "allowed callers": "programmatic" , }, { "type": "function", "name": "get demand", "description": "Return an object with sku string and requested units number .", "parameters": { "type": "object", "properties": {"sku": {"type": "string"}}, "required": "sku" , "additionalProperties": False, }, "output schema": { "type": "object", "properties": { "sku": {"type": "string"}, "requested units": {"type": "number"}, }, "required": "sku", "requested units" , "additionalProperties": False, }, "allowed callers": "programmatic" , }, {"type": "programmatic tool calling"}, input items = { "role": "user", "content": "Compare inventory with demand for sku 123.", } while True: response = client.responses.create model=model, store=False, input=input items, tools=tools, if response.status = "completed": raise RuntimeError f"Response ended with status {response.status}" Preserve every output item, including program and reasoning items. input items.extend item.model dump exclude none=True for item in response.output calls = item for item in response.output if item.type == "function call" if not calls: message = next item for item in response.output if item.type == "message" , None if message: refusal = next part.refusal for part in message.content if part.type == "refusal" , "", print response.output text or refusal break continue for call in calls: run = implementations.get call.name if run is None: raise ValueError f"Unknown tool: {call.name}" result = run json.loads call.arguments input items.append { "type": "function call output", "call id": call.call id, "output": json.dumps result , Preserve caller so the runtime can resume the correct program. "caller": call.caller.model dump if call.caller else None, } When you store responses, you can continue from previous response id instead of resending all earlier response items. Send the new function call output items as the next input. With store: false , replay the complete sequence in order, including every program , reasoning, function-call, function-call-output, and program output item. For stateless reasoning-model requests, replay every returned reasoning item. Each item includes encrypted content by default. See conversation state https://developers.openai.com/api/docs/guides/conversation-state manually-manage-conversation-state for the general stateless pattern. Design tools for programs - Return structured, compact data that JavaScript can inspect without parsing prose. - Use output schema to define each tool’s expected return fields and types, and document its error behavior. If the return shape isn’t known in advance, keep the tool direct so the model can inspect the result. - Define the exact program result shape and required evidence. Return a clear structured failure when the program can’t produce a valid result. - Make function calls idempotent when possible. A retry or replay shouldn’t repeat an unsafe side effect. - Check arguments and permissions for each call in your application, even when it comes from a hosted program. - Give tools specific names and descriptions so the model can compose them correctly. - Require application-level approval before high-impact actions, regardless of the caller. Evaluate Programmatic Tool Calling Programmatic Tool Calling can reduce the amount of intermediate tool output added to model context, but the effect depends on the task and tool responses. Start with direct tool calling as a baseline, then compare both approaches on representative tasks. Define the final-answer quality bar and required evidence before measuring efficiency. Evaluate token use and tool calls alongside correctness, completeness, and evidence coverage, and make any accepted quality tradeoff explicit. Measure: - Final-answer correctness, completeness, and evidence coverage. - Input and total tokens, end-to-end latency, and cost. - Model turns, tool calls, retries, and recovery behavior. - Safety outcomes, especially for side effects and approval requirements. - Whether the route that ran matched the intended workflow stage. Agents API In the Agents API https://developers.openai.com/api/docs/guides/agents-api/overview , Programmatic Tool Calling runs in the OpenAI-managed agent harness and is enabled by default. The harness gives the agent an exec tool and makes its existing tools available inside generated JavaScript. You don’t need to wrap those tools as command-line programs or install them in the sandbox. To disable Programmatic Tool Calling, include this entry in agent.tools : { "type": "programmatic tool calling", "enabled": false } Omitting the entry or its enabled field leaves Programmatic Tool Calling enabled. A type-only entry, { "type": "programmatic tool calling" } , also keeps it enabled. The allowed callers configuration and Responses continuation loop above describe the Responses API integration. Programmatic Tool Calling also works in conversation-only sessions with environment.type set to none . Bash, executor MCPs, and other tools that run in a sandbox still require an execution environment https://developers.openai.com/api/docs/guides/agents-api/environments/self-hosted . Orchestrating a tool in JavaScript doesn’t change where the tool runs. A shell call runs commands in the sandbox; the JavaScript runtime doesn’t start system processes itself. Executor MCPs still use the sandbox, and function tools still call your application server. The agent processes their results before deciding what enters model context. Use the routing guidance above to define which workflow stages should use code. Follow Functions https://developers.openai.com/api/docs/guides/agents-api/tools/functions and MCP connections https://developers.openai.com/api/docs/guides/agents-api/tools/mcp for Agents API configuration and call handling. Related guides - Use function calling https://developers.openai.com/api/docs/guides/function-calling to define client-owned functions. - Use tool search https://developers.openai.com/api/docs/guides/tools-tool-search to defer large tool definitions until a model needs them. - Use conversation state https://developers.openai.com/api/docs/guides/conversation-state to continue stored or stateless Responses API requests. - Review data controls https://developers.openai.com/api/docs/guides/your-data before choosing a storage mode.