{"slug": "function-calling-with-a-local-llm-to-drive-foundry-fuzz-read-repeat", "title": "Function Calling With a Local LLM to Drive Foundry: Fuzz, Read, Repeat", "summary": "A developer built an agent loop that lets a local LLM (qwen2.5-coder:7b) drive Foundry to fuzz-test Solidity contracts, using Ollama's function calling to run forge tests, read files, and write fuzz tests. The tool, which keeps everything on the user's machine, iterates until tests compile and reveal counterexamples, though the model sometimes wastes turns renaming functions. The developer emphasizes keeping the toolset minimal and truncating test output to avoid overwhelming the small model.", "body_md": "The first time I let a local model drive Foundry unsupervised, it spent eleven turns trying to fix a fuzz test by renaming the test function. Not changing the logic. Renaming it. `testFuzz_withdraw`\n\n, then `test_fuzz_withdraw`\n\n, then `testWithdrawFuzz`\n\n, each time running the suite and reading the same compiler error with fresh optimism.\n\nThat experiment still turned into one of the more useful tools in my workflow, once I accepted what a small local model can and cannot do in an agent loop. The idea is simple: give qwen2.5-coder three tools (run forge tests, read a contract file, write a fuzz test) and point it at a target contract. It reads the code, writes property tests, runs them, reads the failures, and iterates. When it works, it surfaces broken invariants I would have gotten to eventually, but it gets there while I make coffee, and everything stays on my machine.\n\nOllama supports function calling through its chat API. You describe tools in JSON schema, the model returns `tool_calls`\n\n, you execute and feed results back. Keep the toolset minimal, every extra tool is another way for a 7b model to get confused.\n\n``` js\nconst tools = [\n  {\n    type: \"function\",\n    function: {\n      name: \"read_file\",\n      description: \"Read a Solidity source file from the project.\",\n      parameters: {\n        type: \"object\",\n        properties: {\n          path: { type: \"string\", description: \"Path relative to project root\" },\n        },\n        required: [\"path\"],\n      },\n    },\n  },\n  {\n    type: \"function\",\n    function: {\n      name: \"write_fuzz_test\",\n      description:\n        \"Write the complete contents of the fuzz test file test/Fuzz.t.sol. \" +\n        \"Always write the FULL file, never a fragment.\",\n      parameters: {\n        type: \"object\",\n        properties: {\n          content: { type: \"string\", description: \"Full Solidity file content\" },\n        },\n        required: [\"content\"],\n      },\n    },\n  },\n  {\n    type: \"function\",\n    function: {\n      name: \"run_forge_test\",\n      description: \"Compile and run the test suite. Returns compiler errors or test results.\",\n      parameters: { type: \"object\", properties: {}, required: [] },\n    },\n  },\n];\n```\n\nTwo design decisions hiding in there. First, `write_fuzz_test`\n\ntakes no path: the model writes exactly one file, `test/Fuzz.t.sol`\n\n, always in full. Letting a small model choose file paths or emit diffs is asking for `contracts/test/../test/Fuzz.sol`\n\nand patches against lines that don't exist. Second, `run_forge_test`\n\ntakes no arguments at all, so the model can't invent flags.\n\nThe executors are thin wrappers. The only interesting one is the test runner, which truncates aggressively:\n\n``` js\nimport { execFileSync } from \"node:child_process\";\n\nfunction runForgeTest(): string {\n  try {\n    const out = execFileSync(\"forge\", [\"test\", \"--fuzz-runs\", \"256\"], {\n      cwd: PROJECT_ROOT,\n      timeout: 120_000,\n      encoding: \"utf8\",\n    });\n    return tail(out, 3000);\n  } catch (e: unknown) {\n    const err = e as { stdout?: string; stderr?: string };\n    return \"FAILED\\n\" + tail((err.stdout ?? \"\") + (err.stderr ?? \"\"), 3000);\n  }\n}\n```\n\nTruncation matters because a forge trace on a failing fuzz test can be enormous, and dumping 40k tokens of trace into a small model's context doesn't inform it, it lobotomizes it. I keep the tail, which is where Foundry puts the counterexample and the summary.\n\n``` js\nconst messages: Message[] = [\n  { role: \"system\", content: SYSTEM_PROMPT },\n  { role: \"user\", content: `Target contract: src/${target}. Read it, identify invariants, write fuzz tests, run them, iterate until they compile and either pass or reveal a real counterexample.` },\n];\n\nfor (let turn = 0; turn < MAX_TURNS; turn++) {\n  const res = await ollama.chat({ model: \"qwen2.5-coder:7b\", messages, tools });\n  messages.push(res.message);\n\n  if (!res.message.tool_calls?.length) break; // model thinks it's done\n\n  for (const call of res.message.tool_calls) {\n    const result = dispatch(call.function.name, call.function.arguments);\n    messages.push({ role: \"tool\", content: result });\n  }\n}\n```\n\nThe system prompt is where most of the iteration went. The parts that earned their place:\n\n```\nYou write Foundry fuzz tests for one target contract.\nRules:\n- ALWAYS read the target contract before writing any test.\n- Test properties and invariants, not specific values. Good properties:\n  balance accounting sums correctly, access control cannot be bypassed,\n  state transitions preserve solvency, no operation mints value from nothing.\n- Use vm.assume or bound() to constrain inputs, never require statements.\n- After every write, run the tests. Read the ACTUAL error before editing.\n- If the same error appears twice in a row, change your approach entirely.\n- A failing fuzz test with a counterexample is a SUCCESS. Report it and stop.\n```\n\nThat last line is important and counterintuitive. The model's instinct is to \"fix\" a failing test by weakening the assertion until it passes, which is the exact opposite of what a security tool should do. Telling it explicitly that a legitimate counterexample is the win condition mostly (not always) suppresses that reflex. I hit the same failure mode building spectr-ai: small models are trained to make errors go away, and in security work the error is frequently the product.\n\nSmall models lose the plot in long agent loops. Around eight to twelve turns in, qwen2.5-coder:7b starts forgetting constraints from the system prompt, re-reading files it already read, or cycling between two broken versions of the same test. The 1.5b model is worse, it rarely survives past turn four before emitting a tool call with malformed arguments. This isn't a bug I can prompt away, it's what limited context handling and small parameter counts look like in practice.\n\nSo I stopped fighting it and bounded the loop instead:\n\n**Hard turn cap.** `MAX_TURNS = 12`\n\n. Past that, whatever's in `test/Fuzz.t.sol`\n\nis the output, and I take over.\n\n**Loop detection.** I hash each `write_fuzz_test`\n\npayload. Same hash twice means the model is cycling, and the harness injects a user message saying so, one time. Second repeat kills the run.\n\n**Compile-first gating.** The most common death spiral is chasing compiler errors (wrong import path, wrong pragma). The harness pre-seeds the file with a known-good skeleton (correct imports, a deployed instance of the target in `setUp`\n\n), so the model starts from something that compiles and only has to fill in properties. This single change roughly doubled how often runs end usefully.\n\n**One target, one file.** I don't ask it to fuzz a protocol. I ask it to fuzz one contract, sometimes one function. Small scope is the difference between a focused property test and turn-eleven function renaming.\n\nWith those bounds, a run takes a few minutes on my machine (WSL2, Ollama, 7b model) and ends usefully most of the time: either compiling property tests I edit and keep, or a genuine counterexample. It found an unchecked rounding case in one of my own vault experiments that my hand-written tests missed, mostly because the model, having no idea what the code was \"supposed\" to do, tested the boring invariant I had skipped.\n\nThat's the honest pitch. It's not an autonomous auditor, it's a tireless, slightly dim intern with a Foundry license, and bounded correctly that's genuinely worth having.\n\nHave you tried giving a local model tools yet, and if so, at what turn count does yours start eating its own tail?", "url": "https://wpnews.pro/news/function-calling-with-a-local-llm-to-drive-foundry-fuzz-read-repeat", "canonical_source": "https://dev.to/pavelespitia/function-calling-with-a-local-llm-to-drive-foundry-fuzz-read-repeat-5el3", "published_at": "2026-08-01 15:09:05+00:00", "updated_at": "2026-08-01 15:50:49.742993+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence", "large-language-models", "ai-agents"], "entities": ["Foundry", "Ollama", "qwen2.5-coder"], "alternates": {"html": "https://wpnews.pro/news/function-calling-with-a-local-llm-to-drive-foundry-fuzz-read-repeat", "markdown": "https://wpnews.pro/news/function-calling-with-a-local-llm-to-drive-foundry-fuzz-read-repeat.md", "text": "https://wpnews.pro/news/function-calling-with-a-local-llm-to-drive-foundry-fuzz-read-repeat.txt", "jsonld": "https://wpnews.pro/news/function-calling-with-a-local-llm-to-drive-foundry-fuzz-read-repeat.jsonld"}}