LLMs cannot actually do anything except generate text—unless you give them access to tools. In llama-cpp
there are some built-in tools that you can enable with the "tools all" argument. However more tools can be used from MCP servers.
An MCP server (Model Context Protocol) is an HTTP server that returns a JSON list of tools that a model can call, along with some details about when to call each tool. When a prompt (such as those entered into the llama-cpp
chat UI) is determined by the LLM to require a tool, it may use the MCP server tools.
After some prompting, I developed an MCP server in Python and then Rust. I found that:
• Models like Gemma 4 12B will call into the tools provided by the MCP server, and this is fairly reliable.
• MCP servers can be run locally, although some CORS headers issues may arise—these can be fixed with some additional server-side logic.
• MCP tools calls are just like built-in ones, but they can be streamlined a bit and may be more reliable because of this.
For example, if I have an MCP server provide a create_archive
tool call, the model can create an archive by just using this tool call. But if I have a Markdown file with instructions on creating the archive, this may be more likely to fail as it involves more steps. So having an MCP server for commonly-needed tasks may be worthwhile.