Function Calling With a Local LLM to Drive Foundry: Fuzz, Read, Repeat

A developer built an agent loop that lets a local LLM (qwen2.5-coder:7b) drive Foundry to fuzz-test Solidity contracts, using Ollama's function calling to run forge tests, read files, and write fuzz tests. The tool, which keeps everything on the user's machine, iterates until tests compile and reveal counterexamples, though the model sometimes wastes turns renaming functions. The developer emphasizes keeping the toolset minimal and truncating test output to avoid overwhelming the small model.

The first time I let a local model drive Foundry unsupervised, it spent eleven turns trying to fix a fuzz test by renaming the test function. Not changing the logic. Renaming it. testFuzz withdraw , then test fuzz withdraw , then testWithdrawFuzz , each time running the suite and reading the same compiler error with fresh optimism. That experiment still turned into one of the more useful tools in my workflow, once I accepted what a small local model can and cannot do in an agent loop. The idea is simple: give qwen2.5-coder three tools run forge tests, read a contract file, write a fuzz test and point it at a target contract. It reads the code, writes property tests, runs them, reads the failures, and iterates. When it works, it surfaces broken invariants I would have gotten to eventually, but it gets there while I make coffee, and everything stays on my machine. Ollama supports function calling through its chat API. You describe tools in JSON schema, the model returns tool calls , you execute and feed results back. Keep the toolset minimal, every extra tool is another way for a 7b model to get confused. js const tools = { type: "function", function: { name: "read file", description: "Read a Solidity source file from the project.", parameters: { type: "object", properties: { path: { type: "string", description: "Path relative to project root" }, }, required: "path" , }, }, }, { type: "function", function: { name: "write fuzz test", description: "Write the complete contents of the fuzz test file test/Fuzz.t.sol. " + "Always write the FULL file, never a fragment.", parameters: { type: "object", properties: { content: { type: "string", description: "Full Solidity file content" }, }, required: "content" , }, }, }, { type: "function", function: { name: "run forge test", description: "Compile and run the test suite. Returns compiler errors or test results.", parameters: { type: "object", properties: {}, required: }, }, }, ; Two design decisions hiding in there. First, write fuzz test takes no path: the model writes exactly one file, test/Fuzz.t.sol , always in full. Letting a small model choose file paths or emit diffs is asking for contracts/test/../test/Fuzz.sol and patches against lines that don't exist. Second, run forge test takes no arguments at all, so the model can't invent flags. The executors are thin wrappers. The only interesting one is the test runner, which truncates aggressively: js import { execFileSync } from "node:child process"; function runForgeTest : string { try { const out = execFileSync "forge", "test", "--fuzz-runs", "256" , { cwd: PROJECT ROOT, timeout: 120 000, encoding: "utf8", } ; return tail out, 3000 ; } catch e: unknown { const err = e as { stdout?: string; stderr?: string }; return "FAILED\n" + tail err.stdout ?? "" + err.stderr ?? "" , 3000 ; } } Truncation matters because a forge trace on a failing fuzz test can be enormous, and dumping 40k tokens of trace into a small model's context doesn't inform it, it lobotomizes it. I keep the tail, which is where Foundry puts the counterexample and the summary. js const messages: Message = { role: "system", content: SYSTEM PROMPT }, { role: "user", content: Target contract: src/${target}. Read it, identify invariants, write fuzz tests, run them, iterate until they compile and either pass or reveal a real counterexample. }, ; for let turn = 0; turn < MAX TURNS; turn++ { const res = await ollama.chat { model: "qwen2.5-coder:7b", messages, tools } ; messages.push res.message ; if res.message.tool calls?.length break; // model thinks it's done for const call of res.message.tool calls { const result = dispatch call.function.name, call.function.arguments ; messages.push { role: "tool", content: result } ; } } The system prompt is where most of the iteration went. The parts that earned their place: You write Foundry fuzz tests for one target contract. Rules: - ALWAYS read the target contract before writing any test. - Test properties and invariants, not specific values. Good properties: balance accounting sums correctly, access control cannot be bypassed, state transitions preserve solvency, no operation mints value from nothing. - Use vm.assume or bound to constrain inputs, never require statements. - After every write, run the tests. Read the ACTUAL error before editing. - If the same error appears twice in a row, change your approach entirely. - A failing fuzz test with a counterexample is a SUCCESS. Report it and stop. That last line is important and counterintuitive. The model's instinct is to "fix" a failing test by weakening the assertion until it passes, which is the exact opposite of what a security tool should do. Telling it explicitly that a legitimate counterexample is the win condition mostly not always suppresses that reflex. I hit the same failure mode building spectr-ai: small models are trained to make errors go away, and in security work the error is frequently the product. Small models lose the plot in long agent loops. Around eight to twelve turns in, qwen2.5-coder:7b starts forgetting constraints from the system prompt, re-reading files it already read, or cycling between two broken versions of the same test. The 1.5b model is worse, it rarely survives past turn four before emitting a tool call with malformed arguments. This isn't a bug I can prompt away, it's what limited context handling and small parameter counts look like in practice. So I stopped fighting it and bounded the loop instead: Hard turn cap. MAX TURNS = 12 . Past that, whatever's in test/Fuzz.t.sol is the output, and I take over. Loop detection. I hash each write fuzz test payload. Same hash twice means the model is cycling, and the harness injects a user message saying so, one time. Second repeat kills the run. Compile-first gating. The most common death spiral is chasing compiler errors wrong import path, wrong pragma . The harness pre-seeds the file with a known-good skeleton correct imports, a deployed instance of the target in setUp , so the model starts from something that compiles and only has to fill in properties. This single change roughly doubled how often runs end usefully. One target, one file. I don't ask it to fuzz a protocol. I ask it to fuzz one contract, sometimes one function. Small scope is the difference between a focused property test and turn-eleven function renaming. With those bounds, a run takes a few minutes on my machine WSL2, Ollama, 7b model and ends usefully most of the time: either compiling property tests I edit and keep, or a genuine counterexample. It found an unchecked rounding case in one of my own vault experiments that my hand-written tests missed, mostly because the model, having no idea what the code was "supposed" to do, tested the boring invariant I had skipped. That's the honest pitch. It's not an autonomous auditor, it's a tireless, slightly dim intern with a Foundry license, and bounded correctly that's genuinely worth having. Have you tried giving a local model tools yet, and if so, at what turn count does yours start eating its own tail?