A developer created a local MCP server to extend llama-cpp's tool capabilities, finding that models like Gemma 4 12B reliably call MCP-provided tools. The server, written first in Python then Rust, of…
Laguna XS 2.1, used with OpenCode, helped a developer replace narrowly-focused unit tests in a Rust project with a single, more general test that covers multiple functions at once, resulting in fewer …
Poolside AI's Laguna XS model, a small local LLM, performed well in refactoring Rust code via OpenCode, making relatively few errors and handling tool calls correctly except for escaped quotes. The 20…
A developer proposes replacing traditional Bash scripts with Markdown files processed by local AI agents, arguing that code is a maintenance burden. Using llama-cpp's tool-use feature, an AI agent lik…
Kimi K3, a new AI model released this week and claimed to be near-frontier quality similar to Fable and GPT, successfully optimized a Rust program by implementing a SWAR (SIMD Within a Register) optim…
A developer used AI agents Gemma 4 12B and Qwen 3.6 35B to rewrite Rust code without doc-comments, but found direct rewriting unreliable. Instead, the developer had Qwen 3.6 generate a Python script t…
A developer is using llama-cpp's tool-calling feature to let large language models execute system commands such as reading files and editing code, comparing the approach to bash scripting but easier t…
A developer reports that speculative decoding techniques MTP, EAGLE-3, and DFlash can significantly speed up local inference of large language models in llama-cpp. Testing on an NVidia 3060 RTX 12 GB …
Z-Lab's DFlash technique uses diffusion models to accelerate LLM token generation through speculative decoding, achieving up to 123 tokens per second for code generation in tests with Qwen 3 8B on lla…
Local LLMs are necessary for privacy and freedom, as data center subscriptions risk data misuse and account lockouts. Smaller models are becoming more capable, making local AI viable for software deve…
A developer spent an hour trying to optimize a Rust function but failed, then used a local LLM agent (Qwen 3.6 35B) in OpenCode to suggest and implement a faster, clearer solution. The LLM's alternati…
Intel's AutoRound quantization algorithm for large language models claims state-of-the-art performance, enabling LLMs like Qwen 3.6 to run on consumer GPUs. In testing, AutoRound produced consistent o…