Teaching PacketSnitch to Use Its Own Documentation: Build a Small AI Tool Router PacketSnitch developers built a small AI tool router that has the model classify a user's natural-language request into an intent category and then look up pregenerated documentation for the matching action, rather than loading every application function into the prompt. The documentation index is generated at build time with the command `npm run docs:build-index`, and the router emits an intent-shaped object (for example `{"intent": "query", "query": "transport.proto: tcp"}`) that is then mapped to an action-shaped call such as `filter-query` with a `query` argument. The approach is intended to avoid context limits, inconsistent tool selection and the need for a large model, dedicated GPU or cloud provider when running locally. The Problem One of the more interesting problems I’ve run into while developing PacketSnitch is figuring out how to let a user interact with the application using natural language, and then translate that request into an actual operation that the application knows how to perform. This sounds relatively straightforward at first. Give an AI model access to a collection of tools, describe what those tools do, and let it decide which one to call. Right? Well, not exactly. The problem is that as an application grows, so does the number of things you can ask it to do. You end up with an increasingly large collection of functions, arguments, parameters, and very specific requirements for how each operation needs to be executed. You can throw all of that into a prompt, but eventually you’re going to run into context limitations, inconsistent tool selection, and models that simply aren’t capable enough to reliably figure out what you’re asking them to do. And if you’re running everything locally, you also have to consider the hardware you’re expecting your users to have. I didn’t particularly want to require a massive language model, a dedicated GPU, or a cloud provider just to let somebody ask PacketSnitch a question about their capture. So I started looking at the problem a little differently. A Search Engine Is Already Doing Something Similar Interestingly, the solution I came up with is based on something I had already implemented elsewhere in PacketSnitch: the search functionality on the website. The way the website search works is that it takes a user’s search terms, identifies relevant keywords, and uses those keywords to find the appropriate documentation and information from an indexed collection. I realized I could apply essentially the same concept to the AI integration. Instead of teaching an AI model how to use every individual function in PacketSnitch, I could teach it how to recognize what the user wants to accomplish, categorize that intent, and then look up the appropriate instructions for carrying out the task. In other words, the AI doesn’t actually need to know how to do everything. It needs to know where to look. Building a Corpus of Actions The first part of this was creating a structured collection of actions that the AI could reference. These actions are associated with documentation and instructions explaining exactly how they should be carried out. These documentation indexes are pregenerated at build time using: npm run docs:build-index This produces the structured documentation and examples that the AI can reference when determining how to fulfill a request. Rather than dumping the entire application’s documentation into the model’s context, I use keywords passed into the prompt to generate a relevant collection of action categories on demand. The model then interprets the user’s request and determines which category of actions is most appropriate. There is an important distinction here between what the router is expected to output and what the actual application needs to execute. For example, the router might be given documentation explaining that a request to find matching packets should use the query intent, and that applying a filter requires the filter-query action with an argument called query . A router-shaped example looks like this: { "intent": "query", "query": "transport.proto: tcp", "rationale": "The request asks for matching packets, so it is a filter query.", "ratCount": 0 } The corresponding action-shaped representation looks like this: { "action": "filter-query", "args": { "query": "transport.proto: tcp" } } These are two different representations serving two different purposes. The first describes what the user wants to accomplish. The second describes the operation the application needs to perform. The model uses the documentation as a reference to understand the relationship between the two, but it isn’t necessarily just copying one JSON structure into another. It needs to account for arbitrary user-supplied data, application context, and sometimes additional planning before it can construct the final action. This is particularly important when dealing with more complicated requests, where a user might want to combine several operations into one task. For example, a capability can be represented as a plan containing a capability step: { "intent": "plan", "steps": { "action": "capability", "id": "stats.places.search", "args": { "query": "Hangzhou" } } , "rationale": "The request asks for one registered capability.", "ratCount": 0 } The documentation provides the model with the information it needs to construct these operations, while the model itself is responsible for interpreting the request and generating the appropriate JSON. This is an important distinction because the model isn’t expected to memorize the application’s entire API, understand every function in the codebase, or invent its own implementation of a requested operation. It’s effectively acting as a router and categorizer that connects the user’s intent to a collection of documented capabilities. The application handles the actual execution. A Small Model With an Application-Controlled Escalation Path The other interesting part of this implementation is that I don’t actually need a particularly large model to accomplish any of this. I’m using a 0.5B parameter model as the initial router. This handles straightforward requests, where the intent is relatively obvious and the appropriate action can be identified without much additional reasoning. For requests that are more complicated, there’s an escalation path to a 1.5B parameter model. There is a subtle but important distinction in how this works: the 0.5B model isn’t deciding whether it’s smart enough to handle a request. That decision is made by the surrounding application logic. PacketSnitch evaluates the intent identified by the smaller model and determines whether that request falls within the capabilities of the fast routing system. For example, asking PacketSnitch to show TCP packets is a relatively straightforward operation. The router can identify the intent, look up the appropriate action, and generate the necessary JSON. Asking PacketSnitch to pull up a specific location on the heatmap and display only its interactions with another host that is already filtered is a different story. That involves multiple pieces of application context, potentially several operations, and a little more planning than I’d reasonably expect from a 0.5B model. In that situation, the application escalates the request to the more capable model. There is also an additional validation mechanism. The models are trained to produce valid JSON, and if the smaller model produces output that doesn’t meet the expected requirements, it gets rejected and automatically escalated. This means I’m not relying on the smaller model to recognize its own limitations or determine whether it should ask for help. The application controls that process. The 1.5B model is then able to handle more complicated requests, including planning operations, incorporating application context, and chaining multiple intents together to accomplish something that would be difficult to represent as a single straightforward tool call. This gives me two advantages. First, the majority of requests can be handled by an extremely small model, which is considerably easier to run locally. Second, the more complicated requests still have access to a model with a little more capacity to interpret the user’s intent and work through the available documentation. I don’t need to throw a large model at every single request just because a small percentage of requests might require more reasoning. Training the Router Getting the architecture working was one thing. Getting a model this small to actually recognize enough different user intentions to reliably select the correct action was another. I ended up going through five training runs to get the initial 0.5B router where I wanted it, followed by six retraining iterations while refining the setup and training data. The base models I used were: - Qwen/Qwen2.5-0.5B-Instruct - unsloth/Qwen2.5-1.5B-Instruct One of the more frustrating parts of this process was discovering that an earlier training setup I had experimented with was fundamentally broken. The resulting models were terrible, and it wasn’t immediately obvious that the problem was the training stack itself rather than the data or the model. Once I identified that, I pinned down the exact configuration I was using and stopped changing variables unnecessarily. The training environment ended up looking like this: torch==2.11.0+cu128 unsloth==2026.11.4 unsloth zoo==2026.11.4 transformers==4.57.6 trl==0.29.1 peft==0.21.3 bitsandbytes==0.50.2 datasets==4.8.5 accelerate==1.15.0 huggingface-hub==0.36.2 safetensors==0.8.0 fsspec==2026.2.0 sentencepiece==0.2.3 protobuf==4.25.9 gguf==0.19.0 pyyaml==6.0.3 xformers==0.0.36 I also force-reinstalled the appropriate PyTorch build against the CUDA 12.8 index after installing the requirements to make sure the environment was using the exact version I intended. From there, most of the work was in the training data. With a model this small, you really don’t have much room for ambiguity. It needs to recognize the difference between requests that might sound similar to a larger model, understand which category they belong to, and reliably produce the correct structured response. I spent quite a bit of time modifying and tweaking the dataset, making sure the model was learning the distinctions between different types of intent rather than simply memorizing keywords. That distinction matters quite a bit. A user might ask PacketSnitch to find something, analyze something, summarize something, or retrieve information about a particular host. Depending on the wording, these requests can overlap considerably. The router needs to recognize what the user actually intends to accomplish, not just match a couple of words and blindly select the first tool that happens to contain them. The output also needs to be predictable. I’m not asking the model to write an essay or produce a conversational response. I’m asking it to identify an operation and construct a small JSON representation that the application can actually use. Eventually I got the router to a point where it was reliably identifying the appropriate intent and selecting the correct tool without needing a much larger model. What the Results Look Like One of the things I wanted to be able to do during development was measure whether the router was actually producing the expected tool calls, rather than just looking at its responses and deciding that they seemed reasonable. I built test scenarios that compare the expected action against the model’s prediction. For example, here’s an actual working test from the mischief-500h router: Request: Show TCP packets Expected output: { "action": "filter-query", "args": { "query": "transport.proto: tcp" } } Model prediction: { "action": "filter-query", "args": { "query": "transport.proto: tcp" } } The test output: { "scenario": "fast-route:filter:transport.proto: tcp", "request": "Show TCP packets", "expected": "{\"action\":\"filter-query\",\"args\":{\"query\":\"transport.proto: tcp\"}}", "prediction": "{\"action\":\"filter-query\",\"args\":{\"query\":\"transport.proto: tcp\"}}", "ok": true, "latencyMs": 153 } That’s a 153ms response for a small model identifying the user’s intent and producing the correct structured action. Obviously, that’s one test case, not a claim that every request will perform identically. But it demonstrates what this architecture is capable of when the request falls within the smaller model’s responsibilities. The important thing is that the model isn’t spending its time figuring out how to implement a filter. It already has documentation explaining how to do that. Its job is to understand what the user wants and produce the appropriate action. The Cost of Doing It The entire process took a little over a week. For the actual training, I used datacenter-mounted L40S graphics cards, running on a 90-core Xeon system. The total cost for the compute was only around $40. That’s not a typo. For roughly forty dollars in training compute, I was able to get a small language model trained and refined enough to perform a fairly specialized routing task that would otherwise be tempting to throw a much larger model at. And the resulting model is small enough to run on a reasonably equipped laptop with 32GB of memory, without consuming everything available to it. I think that’s a pretty good tradeoff. Why This Approach Works I think one of the bigger misconceptions surrounding AI integration is that you need a particularly intelligent model to accomplish something useful. You don’t necessarily need one. A lot of application-level AI problems aren’t really problems that require a model to know everything. They’re problems that require it to identify what somebody wants, determine which capability is relevant, and follow a known procedure. By separating those responsibilities, you can dramatically reduce what the model actually needs to know. In this case, the model is essentially a translation layer between natural language and the application’s existing functionality. The documentation becomes the knowledge base, the JSON definitions and examples become the instructions, and the model becomes the component responsible for interpreting intent and navigating that information. It’s also a much more maintainable arrangement. If I add another capability to PacketSnitch, I can document the action and make it available to the router without having to retrain the model every time I add another tool. The documentation index is pregenerated, the relevant information is retrieved when needed, and the model uses that information to construct the operation. There are obviously limitations. A small model can still misinterpret ambiguous requests, select the wrong category, or produce an incorrect argument structure. That’s why the application still needs to validate the resulting calls and handle failures appropriately. The model isn’t being given unrestricted authority over the application simply because it can construct a JSON response. But for what I’m trying to accomplish, this approach has worked surprisingly well. Conclusion What’s interesting to me is that I essentially ended up using the same philosophy for AI that I was already using for search. Rather than trying to make the system know everything, I made it good at finding the information it needs to accomplish a particular task. The AI identifies what the user wants, finds the appropriate instructions, and follows them. The surrounding application determines whether the request is appropriate for the smaller model or needs to be escalated to something more capable. It doesn’t need a massive parameter count, an expensive GPU, or a cloud-based inference service to do something genuinely useful. I think there’s quite a bit of potential in this approach, particularly for applications with large collections of documented tools and functionality. Instead of trying to train a model to understand an entire application, you can train a relatively small model to navigate the application’s own knowledge base. It’s a clever way to teach an AI to work with the tools it has available, without having to teach it how to build every one of those tools itself. And perhaps more importantly, it makes local AI integration a considerably more practical proposition. Happy hacking