Wrote a Local MCP Server

A developer created a local MCP server to extend llama-cpp's tool capabilities, finding that models like Gemma 4 12B reliably call MCP-provided tools. The server, written first in Python then Rust, offers streamlined tool calls that are more reliable than multi-step instructions, though CORS issues may require server-side fixes.

LLMs cannot actually do anything except generate text—unless you give them access to tools . In llama-cpp there are some built-in tools that you can enable with the "tools all" argument. However more tools can be used from MCP servers . An MCP server Model Context Protocol is an HTTP server that returns a JSON list of tools that a model can call, along with some details about when to call each tool. When a prompt such as those entered into the llama-cpp chat UI is determined by the LLM to require a tool, it may use the MCP server tools. After some prompting, I developed an MCP server in Python and then Rust. I found that: • Models like Gemma 4 12B will call into the tools provided by the MCP server, and this is fairly reliable. • MCP servers can be run locally, although some CORS headers issues may arise—these can be fixed with some additional server-side logic. • MCP tools calls are just like built-in ones, but they can be streamlined a bit and may be more reliable because of this. For example, if I have an MCP server provide a create archive tool call, the model can create an archive by just using this tool call. But if I have a Markdown file with instructions on creating the archive, this may be more likely to fail as it involves more steps. So having an MCP server for commonly-needed tasks may be worthwhile.