A developer benchmarked four caching strategies — exact-match, semantic, retrieval, and prompt (prefix) caching — on the same RAG pipeline running both a local Qwen3 4B model on a MacBook Air and Clau…
A new arXiv study (2608.15693v1) finds that no single model compression technique wins across edge AI tasks, reporting that Qwen3.5 0.8B reaches 93.85 SQuAD F1 and 92 EM under Q5_K_M GGUF quantization…
A new arXiv preprint (2608.00585v1) shows that per-chunk verification fails on multi-hop retrieval-augmented generation, with entailment scoring reaching only 0.643, 0.523, and 0.560 AUC on HotpotQA, …
A developer analyzed the open-source ecosystem for reducing LLM costs, focusing on three cost lines: cached input, uncached/written input, and output. The key insight is that tools must preserve the m…
Headroom, a new macOS menu bar app, reduces Claude Code token usage by approximately 50% by locally trimming prompt bloat and boilerplate before inputs reach the model. The tool processes all data on …