# Build Tool-Calling Parsers in Rust with AI Agents

> Source: <https://developer.nvidia.com/blog/build-tool-calling-parsers-in-rust-with-ai-agents/>
> Published: 2026-11-19 16:30:46+00:00

**Recommendation:** Run the shared YAML Conformance Corpus cases against any new parser, adapting inputs to the model's grammar, and let the serving layer—not the parser—validate tool availability, authorization and schema.
More in the playbook "Designing agent systems": <https://wpnews.pro/playbooks/agent-architecture>

A tool call can arrive in pieces, an argument can be cut short, or model-specific markup can leak into the answer. A frontend parser turns that output, received as token IDs or decoded text, into visible text, reasoning, and tool calls an application can use.

In this post, you’ll learn how tool-call parsing works and use an AI agent to build a parser for a new model. You’ll start with a weather request and a human-reviewed test set, using [frontend-crates](https://github.com/ai-dynamo/frontend-crates), the reusable Rust components behind [NVIDIA Dynamo](https://github.com/ai-dynamo/dynamo) tool-calling parsers.

Suppose you ask for the weather in Paris. The model may generate reasoning followed by a `get_weather` call with a city argument. The frontend prepares and tokenizes the prompt, then an inference engine runs the LLM to generate tokens. The parser consumes token IDs or decoded text and separates reasoning, visible text, and tool calls in order. A protocol adapter turns those events into an OpenAI-compatible response, using the same parsing path for batch JSON and streaming updates.

The tricky part is recognizing where each piece starts and ends. A call can be malformed or incomplete, a marker can appear inside an argument, or ordinary text can surround the call. The parser has to handle these boundaries without losing data, duplicating calls, or leaking model-specific markup.

Returning to the weather request: the model might stop halfway through a city name or call a tool that wasn’t offered. Either can disrupt the agent’s next action. The following NVIDIA Nemotron examples show these two failures.

**Syntax error.** The argument and tool-call frame never close, so no complete call can be parsed.

```
</think><tool_call>
<function=get_weather><parameter=city>Par
```

**Semantic error.** The frame is valid, but the named tool is absent from the request.

```
</think><tool_call>
<function=unknown_tool><parameter=city>Paris</parameter></function>
</tool_call>
```

These failures require different handling. If the model names an unavailable tool, the parser reports the call it received, and the serving layer validates tool availability, authorization, schema, and call policy. Changing the tool name would hide what the model actually generated.

An incomplete call is harder to judge while the stream is open. The rest of the city name may arrive in the next chunk, and a marker can split across chunks, too. Emit too early and syntax leaks into the answer; wait too long and ordinary text gets delayed. The parser needs consistent rules for what to emit, what to buffer, and how to handle the end of the stream.

A parser can encounter an unbounded range of inputs. With invalid input, it must follow documented recovery rules rather than guess the model’s intent. Even a syntactically valid call that conforms to the schema can be the wrong choice for the user’s request. The parser needs a consistent grammar and recovery policy so it can preserve valid data without inventing calls or leaking model-specific markup.

To check those decisions, frontend-crates pairs model outputs with expected results in a shared Conformance Corpus.

## Using the Conformance Corpus

Use the Conformance Corpus to check how applicable parsers handle behavior first observed in one model. Its language-neutral YAML fixtures pair fixed input with expected output, so Rust parsers and Python references can run the same cases. A regression test for leaked markup, duplicate calls, or lost arguments can cover other families when adapted to their grammars.

For a new parser, start with all [`UNIFIED.*`](https://github.com/ai-dynamo/frontend-crates/blob/main/conformance/utils/lib/parsers/UNIFIED_CASES.md) cases across model families. They cover complete, malformed, truncated, multiple, and interleaved calls, plus streaming boundaries and end-of-input behavior. Adapt inputs to your model’s grammar and request settings, add model-specific edge cases, and explain exclusions. You can’t test every possible output, but you can reuse cases from other formats.

Table 1 shows the `UNIFIED.1-tool_only` case from the [Parser Conformance Table](https://ai-dynamo.org/frontend-crates/?search_tab-unified=parser%3Abeam%20OR%20parser%3Agpt_oss%20OR%20parser%3Amuse_spark%20OR%20parser%3Anemotron_v3&vertical=0). In this example, the GOLDEN test set pairs four model-specific inputs with the same expected result: get_weather with city set to Paris. The markers differ, but each case tests the same behavior. Adapting the input format makes the same behavioral test reusable across models.

| **Model** | **Native input** | 
|---|---|
| **Beam** | `<\|start\|>`` assistant to=get_weather``<\|channel\|>`` analysis` `<\|constrain\|>`` xml``<\|message\|>``<rfl:parameter name="city">`` Paris``</rfl:parameter>``<\|call\|>` | 
| **GPT-OSS** | `<\|start\|>`` assistant``<\|channel\|>`` commentary to=functions.get_weather` `<\|constrain\|>`` json``<\|message\|>``{"city": "Paris"}``<\|call\|>` | 
| **Muse Spark** | `<\|start\|>`` assistant to=get_weather``<\|message\|><atem:function_calls>``<atem:invoke name="get_weather">``<atem:parameter name="city">`` Paris``</atem:parameter>``</atem:invoke>``</atem:function_calls><\|eom\|>` | 
| **NVIDIA Nemotron 3** | `</think>``<tool_call>``<function=`` get_weather``>``<parameter=city>`` Paris``</parameter>``</function>``</tool_call>` | 

*Table 1. Model-specific inputs with the same expected GOLDEN output: get_weather({“city”:”Paris”})*

## Tutorial: Create a new parser

Now you can create a parser for a model that frontend-crates doesn’t support yet. Ask your AI agent to read this post and work through the tutorial, starting with the prompt in step 1. Your model’s specification defines the grammar; the shared cases provide behaviors to test.

You’ll need a frontend-crates checkout, Rust and Cargo, and the conformance dependencies. The parser exercise runs on a CPU using fixed test fixtures, so you don’t need a GPU. End-to-end testing also requires a working [Dynamo](https://www.nvidia.com/en-us/ai/dynamo/) runtime, discovery services, and hardware for your model. When you’re ready to test end-to-end, launch the tests with Dynamo runtime on the actual model and GPU(s).

The workflow in Figure 2 starts with the model’s grammar, recovery rules, and reviewed GOLDEN expectations. The agent writes or fixes the parser, runs tests, and uses failures to guide the next change. If it proposes changing the expected behavior, review that decision before accepting it.

Test your parser end to end with real models before claiming deployment support. Plan time to allocate a machine, load the model, initialize the runtime, and run your own test cases. Turn failures into unit tests for the agent, then rerun the end-to-end test to check the fix.

### 1. Start with a prompt

Give the agent this prompt: In the frontend-crates checkout, follow .agents/skills/tool-parser-generator/SKILL.md to create a v2 Unified parser for [MODEL_NAME]. Research its grammar and prepare the GOLDEN test set, adapting applicable UNIFIED.* cases. Pause for human review. After approval, implement, validate, and provide the local conformance report URL for human inspection.

### 2. Define the GOLDEN test set

The GOLDEN test set defines the behavior your parser will be tested against. Use the prompt in step 1 to have the agent generate an initial test set in `conformance/fixtures-unified-v2/families/<name>/inputs_and_golden.yaml`, independently of parser results. Each case pairs an input with its expected output, called GOLDEN. If those expectations are wrong, the parser can pass the tests and still behave incorrectly.

The following example pairs an input with its expected GOLDEN output. It omits request definitions, tools, initialization, and chunk settings; the existing [`nemotron_v3/inputs_and_golden.yaml`](https://github.com/ai-dynamo/frontend-crates/blob/main/conformance/fixtures-unified-v2/families/nemotron_v3/inputs_and_golden.yaml) shows the full format.

```
family: nemotron_v3
cases:
  UNIFIED.tool_only.nemotron_v3:
    input: |-
      </think><tool_call>
      <function=get_weather>
      <parameter=city>Paris</parameter>
      </function></tool_call>
    golden:
      - kind: tool_call
        name: get_weather
        arguments: {city: Paris}
```

Ask the agent for an adversarial review, then go through the set manually, case by case. The initial test set may contain errors because expected behavior depends on context the agent may not have. Review it against the following sources and requirements:

- The model format and API specification.
- The reference implementation.
- Other parsers, including vLLM and SGLang.
- The grammar, tool schema, and request settings.
- Relevant forum discussions and existing tickets.
- Caller needs and recovery costs in latency and complexity.

Where sources disagree, you may need to make a judgment call. Write down the recovery rule and why you chose it, and document the reason for any GOLDEN change. Expectations should follow that decision, not the parser’s output.

### 3. Implement the parser and inspect the results

After reviewing the GOLDEN test set, ask the agent to continue: build the parser, run all applicable [UNIFIED.*](https://github.com/ai-dynamo/frontend-crates/blob/main/conformance/utils/lib/parsers/UNIFIED_CASES.md) cases and shared regression tests, and fix the failures. Have it recapture results, generate `conformance/CONFORMANCE_v2.html` locally, and provide the local report URL so you can inspect its work.

Open that URL in your browser. The local report uses the published [Parser Conformance Table](https://ai-dynamo.org/frontend-crates/?search_tab-unified=parser%3Abeam%20OR%20parser%3Agpt_oss%20OR%20parser%3Amuse_spark%20OR%20parser%3Anemotron_v3&vertical=0)‘s interface with your local results. Inspect each case’s input, request settings, GOLDEN, and recorded output. Investigate regressions and missing captures. Green means the output matches GOLDEN, so the table depends on the expectations you reviewed. Figure 3 shows the report interface.

### Guided-decoding support

The agent’s implementation step also covers guided decoding, which can reduce malformed calls by constraining generation to the model’s markers and argument schema. Ask your agent to follow “[Add and validate structural tags](https://github.com/ai-dynamo/frontend-crates/blob/main/parsers/v2/README.md#add-and-validate-structural-tags)” in the parser README and “[Implement and validate](https://github.com/ai-dynamo/frontend-crates/blob/main/.agents/skills/tool-parser-generator/SKILL.md#2-implement-and-validate)” in the agent skill. These instructions cover builder registration, grammar tests, and backend qualification. Guided decoding can’t ensure that the model chooses the right city or prevent truncation, so keep the parser’s recovery checks.

## Common gotchas: speed, streaming, and memory

Passing correctness tests is only a starting point for an agent-generated parser. The implementation may still be slow or use too much memory. Try plain text, tool calls, long arguments, and malformed input with large and tiny chunks. Measure throughput, time to the first event, and memory use on the same build and hardware. Double the input length to spot code that keeps rescanning or copying what it already read.

With streaming, watch what the caller receives as each chunk arrives. A parser can return the right final result and still make the caller wait until the end. Replay short fixtures at every split point, then try one-character and mixed chunks, including splits inside markers, escaped quotes, and arguments. Check that events arrive in order and arguments stay intact. If events are queued, test a slow consumer and cancellation too.

Also try cutting the stream off halfway through a marker or argument. Check that recovery follows the documented rule without leaking markup, duplicating calls, or losing valid text. The parser should handle the input it received without guessing the missing pieces.

Test text containing examples of the model’s format as well as tool calls against the documented grammar and quote-handling policy. The current v2 default treats recognized markers as structural even inside quoted prose, so quotation marks alone do not keep a complete tool-call example in ordinary text. Record the configured policy and test its expected behavior.

Give the agent this prompt to check for common gotchas:

Check this parser for common gotchas. Benchmark speed and memory with short, long, and malformed inputs using different chunk sizes. Look for repeated scanning, copying, or growing buffers. Verify that output arrives incrementally and stays correct across chunk boundaries and truncation. Where queues exist, test a slow consumer, backpressure, and cancellation. Check literal format examples and real tool calls against the documented grammar and configured quote-handling policy. Report measurements, reproducible failures, and anything you could not verify.

## How to use the parser crate in your own Rust project

Choose a parser whose grammar matches your model. This example uses the existing NVIDIA Nemotron parser and splits both a marker and the city name across chunks.

This example was tested with Rust 1.96.1 and dynamo-parsers-v2 0.10.3. Run `cargo new parser-demo`, then `cd parser-demo`. Add the following entries under `[dependencies]` in `Cargo.toml`:

```
anyhow = "1"
dynamo-parsers-v2 = "=0.10.3"
serde_json = "1"
```

Replace `src/main.rs` with:

```
use dynamo_parsers_v2::{
    Tool, UnifiedParserInit, UnifiedParserOutput, create_unified_parser_for_family,
};
use serde_json::json;

fn main() -> anyhow::Result<()> {
    let tools = [Tool {
        name: "get_weather".into(),
        description: None,
        parameters: json!({"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}),
        strict: None,
    }];
    let mut parser = create_unified_parser_for_family("nemotron_v3", &tools)?;
    parser.initialize_request(UnifiedParserInit::default())?;
    let mut output = UnifiedParserOutput::default();
    for chunk in [
        "Check the forecast.</think><tool_",
        "call><function=get_weather><parameter=city>Par",
        "is</parameter></function></tool_call>",
    ] {
        let mut delta = UnifiedParserOutput::default();
        parser.parse_into(chunk, &mut delta)?;
        output.append(&mut delta);
    }
    output.append(&mut parser.finish()?);
    println!("{}", serde_json::to_string(&output.assembled())?);
    Ok(())
}
```

Run `cargo generate-lockfile`, then `cargo run --locked`. The result contains reasoning followed by a get_weather call with city set to Paris:

```
[{"kind":"reasoning","text":"Check the forecast."},
 {"kind":"tool_call","name":"get_weather","arguments":{"city":"Paris"}}]
```

Keep one initialized parser for each request choice.

- `create_unified_parser_for_family` selects the grammar.
- `initialize_request` sets the starting state.
- `parse_into` consumes each chunk.
- Finalize once: use `finish()` for truncation, cancellation, or the end of the transport stream. Use`finish_at_eos()` only for a backend-confirmed grammar-ending end-of-sequence (EOS) signal.
- Use `assembled()` for the ordered results.

In a streaming application, forward the events in `delta.events` after each `parse_into` call and the events returned by `finish()`. This example collects the events to print the final output. Before executing tools, validate completed calls for tool availability, authorization, schema, and call policy.

## Get started

To build a parser for your model, start with the source, fixtures, and instructions in [frontend-crates](https://github.com/ai-dynamo/frontend-crates) and adapt an existing test case. Give your agent the model’s format specification and the repository instructions, then use the [Parser Conformance Table](https://ai-dynamo.org/frontend-crates/?search_tab-unified=parser%3Abeam%20OR%20parser%3Agpt_oss%20OR%20parser%3Amuse_spark%20OR%20parser%3Anemotron_v3&vertical=0) to check the parser’s recorded output against GOLDEN. When you open an issue or pull request, include the exact failing input, case identifier, and test results so reviewers can reproduce the behavior.

You can use these parsers in your own Rust project or as part of [NVIDIA Dynamo](https://github.com/ai-dynamo/dynamo), an inference framework that uses frontend-crates. Before you begin, check the [latest repository instructions](https://github.com/ai-dynamo/frontend-crates/blob/main/parsers/v2/README.md#adding-a-day-0-tool-calling-parser) so you and your agent follow the current steps.

## Acknowledgements

Jesse Gu, [Alec Flowers](https://developer.nvidia.com/blog/author/alecflowers/), Dong Xu, Qi Wang, and the many partners and proofreaders who helped with this blog and frontend-crates.
