I have been compiling llama-cpp to run LLMs locally for about a year. This took about 20 minutes each time because I could not run it on multiple threads because my GPU does not have enough memory. Today I decided to download the pre-built Llama package for version 1.0.0, which was released a few days ago.
This installs the llama command in the terminal, so you can run llama serve and then the arguments for the inference server. Everything worked perfectly the first time, and I had Ternary-Bonsai-2-2B-Q2_0.gguf running right away, complete with my MCP server.
I feel like Llama (llama-cpp) will eventually replace AI subscriptions and companies like OpenAI and Anthropic. It will take a few years for hardware to improve, and become more affordable (I don't recommend buying any expensive computer equipment right now). With local inference, you get privacy, and save a network round-trip—so it will eventually be faster to run Llama than to use an online service.