cd /news/ai-agents/build-tool-calling-parsers-in-rust-w… · home › topics › ai-agents › article
[ARTICLE · art-172663] src=developer.nvidia.com ↗ pub= topic=ai-agents verified=true sentiment=· neutral

Build Tool-Calling Parsers in Rust with AI Agents

NVIDIA Dynamo's frontend-crates project publishes a language-neutral Conformance Corpus of YAML fixtures that pair fixed model output with expected parser results, letting Rust parsers and Python references run identical cases for complete, malformed, truncated, multiple and interleaved tool calls, plus streaming boundaries and end-of-input behavior. The reusable Rust components behind NVIDIA Dynamo's tool-calling parsers consume token IDs or decoded text and separate reasoning, visible text and tool calls in order, with a protocol adapter converting those events into OpenAI-compatible batch JSON and streaming responses. The guidance distinguishes syntax errors, such as an unclosed get_weather call truncated mid-city, from semantic errors, such as a valid frame naming an unknown_tool absent from the request, and assigns validation of tool availability, authorization, schema and call policy to the serving layer rather than the parser.

recommendation

Run the shared YAML Conformance Corpus cases against any new parser, adapting inputs to the model's grammar, and let the serving layer—not the parser—validate tool availability, authorization and schema.

more in playbook: Designing agent systems
by read11 min views5 publishedNov 19, 2026
Build Tool-Calling Parsers in Rust with AI Agents
Image: NVIDIA Developer Blog

A tool call can arrive in pieces, an argument can be cut short, or model-specific markup can leak into the answer. A frontend parser turns that output, received as token IDs or decoded text, into visible text, reasoning, and tool calls an application can use.

In this post, you’ll learn how tool-call parsing works and use an AI agent to build a parser for a new model. You’ll start with a weather request and a human-reviewed test set, using frontend-crates, the reusable Rust components behind NVIDIA Dynamo tool-calling parsers.

Suppose you ask for the weather in Paris. The model may generate reasoning followed by a get_weather call with a city argument. The frontend prepares and tokenizes the prompt, then an inference engine runs the LLM to generate tokens. The parser consumes token IDs or decoded text and separates reasoning, visible text, and tool calls in order. A protocol adapter turns those events into an OpenAI-compatible response, using the same parsing path for batch JSON and streaming updates.

The tricky part is recognizing where each piece starts and ends. A call can be malformed or incomplete, a marker can appear inside an argument, or ordinary text can surround the call. The parser has to handle these boundaries without losing data, duplicating calls, or leaking model-specific markup.

Returning to the weather request: the model might stop halfway through a city name or call a tool that wasn’t offered. Either can disrupt the agent’s next action. The following NVIDIA Nemotron examples show these two failures.

Syntax error. The argument and tool-call frame never close, so no complete call can be parsed.

</think><tool_call>
<function=get_weather><parameter=city>Par

Semantic error. The frame is valid, but the named tool is absent from the request.

</think><tool_call>
<function=unknown_tool><parameter=city>Paris</parameter></function>
</tool_call>

These failures require different handling. If the model names an unavailable tool, the parser reports the call it received, and the serving layer validates tool availability, authorization, schema, and call policy. Changing the tool name would hide what the model actually generated.

An incomplete call is harder to judge while the stream is open. The rest of the city name may arrive in the next chunk, and a marker can split across chunks, too. Emit too early and syntax leaks into the answer; wait too long and ordinary text gets delayed. The parser needs consistent rules for what to emit, what to buffer, and how to handle the end of the stream.

A parser can encounter an unbounded range of inputs. With invalid input, it must follow documented recovery rules rather than guess the model’s intent. Even a syntactically valid call that conforms to the schema can be the wrong choice for the user’s request. The parser needs a consistent grammar and recovery policy so it can preserve valid data without inventing calls or leaking model-specific markup.

To check those decisions, frontend-crates pairs model outputs with expected results in a shared Conformance Corpus.

Using the Conformance Corpus #

Use the Conformance Corpus to check how applicable parsers handle behavior first observed in one model. Its language-neutral YAML fixtures pair fixed input with expected output, so Rust parsers and Python references can run the same cases. A regression test for leaked markup, duplicate calls, or lost arguments can cover other families when adapted to their grammars.

For a new parser, start with all UNIFIED.* cases across model families. They cover complete, malformed, truncated, multiple, and interleaved calls, plus streaming boundaries and end-of-input behavior. Adapt inputs to your model’s grammar and request settings, add model-specific edge cases, and explain exclusions. You can’t test every possible output, but you can reuse cases from other formats.

Table 1 shows the UNIFIED.1-tool_only case from the Parser Conformance Table. In this example, the GOLDEN test set pairs four model-specific inputs with the same expected result: get_weather with city set to Paris. The markers differ, but each case tests the same behavior. Adapting the input format makes the same behavioral test reusable across models.

Model Native input
Beam <|start|>`` assistant to=get_weather``<|channel|>`` analysis <|constrain|>`` xml``<|message|>``<rfl:parameter name="city">`` Paris``</rfl:parameter>``<|call|>
GPT-OSS <|start|>`` assistant``<|channel|>`` commentary to=functions.get_weather <|constrain|>`` json``<|message|>``{"city": "Paris"}``<|call|>
Muse Spark <|start|>`` assistant to=get_weather``<|message|><atem:function_calls>``<atem:invoke name="get_weather">``<atem:parameter name="city">`` Paris``</atem:parameter>``</atem:invoke>``</atem:function_calls><|eom|>
NVIDIA Nemotron 3 </think>``<tool_call>``<function=`` get_weather``>``<parameter=city>`` Paris``</parameter>``</function>``</tool_call>

Table 1. Model-specific inputs with the same expected GOLDEN output: get_weather({“city”:”Paris”})

Tutorial: Create a new parser #

Now you can create a parser for a model that frontend-crates doesn’t support yet. Ask your AI agent to read this post and work through the tutorial, starting with the prompt in step 1. Your model’s specification defines the grammar; the shared cases provide behaviors to test.

You’ll need a frontend-crates checkout, Rust and Cargo, and the conformance dependencies. The parser exercise runs on a CPU using fixed test fixtures, so you don’t need a GPU. End-to-end testing also requires a working Dynamo runtime, discovery services, and hardware for your model. When you’re ready to test end-to-end, launch the tests with Dynamo runtime on the actual model and GPU(s).

The workflow in Figure 2 starts with the model’s grammar, recovery rules, and reviewed GOLDEN expectations. The agent writes or fixes the parser, runs tests, and uses failures to guide the next change. If it proposes changing the expected behavior, review that decision before accepting it.

Test your parser end to end with real models before claiming deployment support. Plan time to allocate a machine, load the model, initialize the runtime, and run your own test cases. Turn failures into unit tests for the agent, then rerun the end-to-end test to check the fix.

1. Start with a prompt

Give the agent this prompt: In the frontend-crates checkout, follow .agents/skills/tool-parser-generator/SKILL.md to create a v2 Unified parser for [MODEL_NAME]. Research its grammar and prepare the GOLDEN test set, adapting applicable UNIFIED.* cases. for human review. After approval, implement, validate, and provide the local conformance report URL for human inspection.

2. Define the GOLDEN test set

The GOLDEN test set defines the behavior your parser will be tested against. Use the prompt in step 1 to have the agent generate an initial test set in conformance/fixtures-unified-v2/families/<name>/inputs_and_golden.yaml, independently of parser results. Each case pairs an input with its expected output, called GOLDEN. If those expectations are wrong, the parser can pass the tests and still behave incorrectly.

The following example pairs an input with its expected GOLDEN output. It omits request definitions, tools, initialization, and chunk settings; the existing nemotron_v3/inputs_and_golden.yaml shows the full format.

family: nemotron_v3
cases:
  UNIFIED.tool_only.nemotron_v3:
    input: |-
      </think><tool_call>
      <function=get_weather>
      <parameter=city>Paris</parameter>
      </function></tool_call>
    golden:
      - kind: tool_call
        name: get_weather
        arguments: {city: Paris}

Ask the agent for an adversarial review, then go through the set manually, case by case. The initial test set may contain errors because expected behavior depends on context the agent may not have. Review it against the following sources and requirements:

  • The model format and API specification.
  • The reference implementation.
  • Other parsers, including vLLM and SGLang.
  • The grammar, tool schema, and request settings.
  • Relevant forum discussions and existing tickets.
  • Caller needs and recovery costs in latency and complexity.

Where sources disagree, you may need to make a judgment call. Write down the recovery rule and why you chose it, and document the reason for any GOLDEN change. Expectations should follow that decision, not the parser’s output.

3. Implement the parser and inspect the results

After reviewing the GOLDEN test set, ask the agent to continue: build the parser, run all applicable UNIFIED.* cases and shared regression tests, and fix the failures. Have it recapture results, generate conformance/CONFORMANCE_v2.html locally, and provide the local report URL so you can inspect its work.

Open that URL in your browser. The local report uses the published Parser Conformance Table‘s interface with your local results. Inspect each case’s input, request settings, GOLDEN, and recorded output. Investigate regressions and missing captures. Green means the output matches GOLDEN, so the table depends on the expectations you reviewed. Figure 3 shows the report interface.

Guided-decoding support

The agent’s implementation step also covers guided decoding, which can reduce malformed calls by constraining generation to the model’s markers and argument schema. Ask your agent to follow “Add and validate structural tags” in the parser README and “Implement and validate” in the agent skill. These instructions cover builder registration, grammar tests, and backend qualification. Guided decoding can’t ensure that the model chooses the right city or prevent truncation, so keep the parser’s recovery checks.

Common gotchas: speed, streaming, and memory #

Passing correctness tests is only a starting point for an agent-generated parser. The implementation may still be slow or use too much memory. Try plain text, tool calls, long arguments, and malformed input with large and tiny chunks. Measure throughput, time to the first event, and memory use on the same build and hardware. Double the input length to spot code that keeps rescanning or copying what it already read.

With streaming, watch what the caller receives as each chunk arrives. A parser can return the right final result and still make the caller wait until the end. Replay short fixtures at every split point, then try one-character and mixed chunks, including splits inside markers, escaped quotes, and arguments. Check that events arrive in order and arguments stay intact. If events are queued, test a slow consumer and cancellation too.

Also try cutting the stream off halfway through a marker or argument. Check that recovery follows the documented rule without leaking markup, duplicating calls, or losing valid text. The parser should handle the input it received without guessing the missing pieces.

Test text containing examples of the model’s format as well as tool calls against the documented grammar and quote-handling policy. The current v2 default treats recognized markers as structural even inside quoted prose, so quotation marks alone do not keep a complete tool-call example in ordinary text. Record the configured policy and test its expected behavior.

Give the agent this prompt to check for common gotchas:

Check this parser for common gotchas. Benchmark speed and memory with short, long, and malformed inputs using different chunk sizes. Look for repeated scanning, copying, or growing buffers. Verify that output arrives incrementally and stays correct across chunk boundaries and truncation. Where queues exist, test a slow consumer, backpressure, and cancellation. Check literal format examples and real tool calls against the documented grammar and configured quote-handling policy. Report measurements, reproducible failures, and anything you could not verify.

How to use the parser crate in your own Rust project #

Choose a parser whose grammar matches your model. This example uses the existing NVIDIA Nemotron parser and splits both a marker and the city name across chunks.

This example was tested with Rust 1.96.1 and dynamo-parsers-v2 0.10.3. Run cargo new parser-demo, then cd parser-demo. Add the following entries under [dependencies] in Cargo.toml:

anyhow = "1"
dynamo-parsers-v2 = "=0.10.3"
serde_json = "1"

Replace src/main.rs with:

use dynamo_parsers_v2::{
    Tool, UnifiedParserInit, UnifiedParserOutput, create_unified_parser_for_family,
};
use serde_json::json;

fn main() -> anyhow::Result<()> {
    let tools = [Tool {
        name: "get_weather".into(),
        description: None,
        parameters: json!({"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}),
        strict: None,
    }];
    let mut parser = create_unified_parser_for_family("nemotron_v3", &tools)?;
    parser.initialize_request(UnifiedParserInit::default())?;
    let mut output = UnifiedParserOutput::default();
    for chunk in [
        "Check the forecast.</think><tool_",
        "call><function=get_weather><parameter=city>Par",
        "is</parameter></function></tool_call>",
    ] {
        let mut delta = UnifiedParserOutput::default();
        parser.parse_into(chunk, &mut delta)?;
        output.append(&mut delta);
    }
    output.append(&mut parser.finish()?);
    println!("{}", serde_json::to_string(&output.assembled())?);
    Ok(())
}

Run cargo generate-lockfile, then cargo run --locked. The result contains reasoning followed by a get_weather call with city set to Paris:

[{"kind":"reasoning","text":"Check the forecast."},
 {"kind":"tool_call","name":"get_weather","arguments":{"city":"Paris"}}]

Keep one initialized parser for each request choice.

  • create_unified_parser_for_family selects the grammar.
  • initialize_request sets the starting state.
  • parse_into consumes each chunk.
  • Finalize once: use finish() for truncation, cancellation, or the end of the transport stream. Usefinish_at_eos() only for a backend-confirmed grammar-ending end-of-sequence (EOS) signal.
  • Use assembled() for the ordered results.

In a streaming application, forward the events in delta.events after each parse_into call and the events returned by finish(). This example collects the events to print the final output. Before executing tools, validate completed calls for tool availability, authorization, schema, and call policy.

Get started #

To build a parser for your model, start with the source, fixtures, and instructions in frontend-crates and adapt an existing test case. Give your agent the model’s format specification and the repository instructions, then use the Parser Conformance Table to check the parser’s recorded output against GOLDEN. When you open an issue or pull request, include the exact failing input, case identifier, and test results so reviewers can reproduce the behavior.

You can use these parsers in your own Rust project or as part of NVIDIA Dynamo, an inference framework that uses frontend-crates. Before you begin, check the latest repository instructions so you and your agent follow the current steps.

Acknowledgements #

Jesse Gu, Alec Flowers, Dong Xu, Qi Wang, and the many partners and proofreaders who helped with this blog and frontend-crates.

── more in #ai-agents 4 stories · sorted by recency
── more on @nvidia 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/build-tool-calling-p…] indexed:0 read:11min 2026-11-19 · —