cd /news/large-language-models/how-to-speed-up-inference-on-dgx-spa… · home › topics › large-language-models › article
[ARTICLE · art-162900] src=forum.level1techs.com ↗ pub= topic=large-language-models verified=true sentiment=↑ positive

How to speed up inference on DGX Spark?

A DGX Spark user running the granite-4.2-30b-fp8 model reported getting only 4-6 output tokens per second with llama.cpp and vLLM, and reached 40 tokens per second after a forum respondent identified as lambda advised switching to a mixture-of-experts model, specifically Qwen 3.8 Flash Next. The exchange, posted on the Level1Techs forum, illustrates that model choice rather than inference engine tuning drove the roughly 7-10x throughput gain on the hardware.

read1 min views1 publishedNov 3, 2026
How to speed up inference on DGX Spark?
Image: Forum (auto-discovered)

I have tried llama.cpp and vllm but so far only getting 4-6 output tokens/seconds with granite-4.2-30b-fp8

Are there any known tips to get faster output? different models, options, bespoke patches?

1 Like

lambda 2

Pick a different model? Ideally MoE.

Check out Qwen 3.8 Flash Next -

2 Likes

Thank you, I was able to get 40 tokens/seconds with this

── more in #large-language-models 4 stories · sorted by recency
── more on @dgx spark 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/how-to-speed-up-infe…] indexed:0 read:1min 2026-11-03 · —