{"slug": "how-to-speed-up-inference-on-dgx-spark", "title": "How to speed up inference on DGX Spark?", "summary": "A DGX Spark user running the granite-4.2-30b-fp8 model reported getting only 4-6 output tokens per second with llama.cpp and vLLM, and reached 40 tokens per second after a forum respondent identified as lambda advised switching to a mixture-of-experts model, specifically Qwen 3.8 Flash Next. The exchange, posted on the Level1Techs forum, illustrates that model choice rather than inference engine tuning drove the roughly 7-10x throughput gain on the hardware.", "body_md": "I have tried llama.cpp and vllm but so far only getting 4-6 output tokens/seconds with granite-4.2-30b-fp8\n\nAre there any known tips to get faster output? different models, options, bespoke patches?\n\n \n1 Like\n\n \n[lambda](https://forum.level1techs.com/u/lambda)\n2\n \nPick a different model? Ideally MoE.\n\nCheck out Qwen 3.8 Flash Next -\n\n \n2 Likes\n\n \nThank you, I was able to get 40 tokens/seconds with this", "url": "https://wpnews.pro/news/how-to-speed-up-inference-on-dgx-spark", "canonical_source": "https://forum.level1techs.com/t/how-to-speed-up-inference-on-dgx-spark/259911#post_3", "published_at": "2026-11-03 03:58:33+00:00", "updated_at": "2026-11-03 04:21:28.309838+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools"], "entities": ["DGX Spark", "llama.cpp", "vLLM", "granite-4.2-30b-fp8", "Qwen 3.8 Flash Next", "Level1Techs", "lambda"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-to-speed-up-inference-on-dgx-spark", "markdown": "https://wpnews.pro/news/how-to-speed-up-inference-on-dgx-spark.md", "text": "https://wpnews.pro/news/how-to-speed-up-inference-on-dgx-spark.txt", "jsonld": "https://wpnews.pro/news/how-to-speed-up-inference-on-dgx-spark.jsonld"}}