16:09
2026-08-27
forum.level1techs.com
large-language-models
So, what local inference models are we using?
A user reports running Gemma4:26b on a GPU at 2000+ tokens/s pre-fill and 80+ tokens/s evaluation, and Laguna S 2.1 118B on a CPU server at 13-14 tokens/s, expressing frustration at the lack of models…