A homelab test on an i5-12600K with 64 GB DDR5 and an RTX 4070 12GB found the narrow inference runtime Strata generated 512 tokens at 53.2 tok/s with 60,000 tokens of context on the 125B-parameter Qwe…
A user running Qwen 3.8 27B on a 3090 reports that switching to ninfer, a Qwen-focused inference engine, doubled their token generation speed from 35-40 to 55-60 tokens per second. The user is conside…
A user running ninfer's Qwen 3.8 27B model on an RTX 3090 is seeking guidance on creating an abliterated version of the model, noting that ninfer uses a custom file format that prevents simple configu…
A user running Qwen3.8-27B on an RTX 5090 with the ninfer runtime reported a 1.72× speedup over LM Studio/llama.cpp, achieving 124.08 tok/s decode and 8,143.82 tok/s prompt eval with mixed NVFP4/FP8 q…