# Speed Up LLM Inference with DSpark Speculative Decoding > Source: > Published: 2026-08-31 14:00:00+00:00 # Speed Up LLM Inference with DSpark Speculative Decoding Learn how DSpark speculative decoding can improve local LLM generation speed using the same GPU, with Qwen3-8B, llama.cpp, and CUDA. There are many ways to get more from the models and GPU infrastructure you already have. Quantization, optimized kernels, and better inference engines can all help, but **speculative decoding** is especially useful because it can increase generation speed without simply adding more GPUs. There are now several approaches to speculative decoding. Traditional methods use a smaller draft model, while **Multi-Token Prediction (MTP)** predicts several future tokens at once. Methods such as **Medusa** and **EAGLE** improve how those drafts are produced, while **DFlash** generates blocks of candidate tokens in parallel. **DSpark** takes another approach by combining parallel drafting with a lightweight sequential component. This helps later draft tokens use information from earlier predictions while keeping much of the speed advantage of parallel generation. In this guide, we will test DSpark with **Qwen3-8B and llama.cpp**. We will benchmark the model normally, enable DSpark with a matching draft model, and compare the generation speeds to see how much performance we can gain from the same GPU. ## How DSpark Works DeepSeek's ** DSpark** improves the drafting part of speculative decoding. Parallel draft models can predict a whole block of tokens in one pass, which is fast, but later predictions can become less accurate because they do not fully depend on the tokens predicted earlier in the block. DSpark combines a **parallel backbone with a lightweight sequential component**, allowing later draft positions to incorporate information from earlier predicted tokens while retaining much of the speed of parallel generation. In simplified terms: DSpark can also estimate how likely draft tokens are to survive verification, allowing low-confidence parts of a block to be dropped instead of wasting verification compute. **llama.cpp** exposes this through its DSpark implementation and optional confidence threshold. DeepSeek reports that DSpark improved per-user generation speed by **60–85%** compared with its previous MTP-1 production baseline when deployed with DeepSeek-V4. Those numbers should not be treated as expected results for our small local model, so we are going to measure the difference ourselves. ## 1. Building llama.cpp and Downloading the Models We will build the latest ** llama.cpp** from source so we can use its current DSpark implementation with CUDA acceleration. Install the required tools: ``` apt-get update apt-get install -y git cmake build-essential ``` Clone the official llama.cpp repository: ``` cd /workspace git clone https://github.com/ggml-org/llama.cpp ``` Build it with CUDA support enabled: ``` cmake llama.cpp -B llama.cpp/build \ -DBUILD_SHARED_LIBS=OFF \ -DGGML_CUDA=ON cmake --build llama.cpp/build \ --config Release \ -j \ --clean-first \ --target llama-cli llama-mtmd-cli llama-server llama-gguf-split ``` This creates the binaries we need while allowing the models to run on the GPU. Next, create a directory for the model files: ``` mkdir -p /workspace/models ``` We will download the GGUF files manually using the Hugging Face CLI so the download time does not affect our benchmarks. Install the CLI: ``` pip install -U huggingface_hub ``` If your Hugging Face token is stored in `HF_TOKEN` , authenticate with: ``` hf auth login --token "$HF_TOKEN" ``` Download the **Qwen3-8B Q4_K_M** target model: ``` hf download \ Qwen/Qwen3-8B-GGUF \ Qwen3-8B-Q4_K_M.gguf \ --local-dir /workspace/models ``` Then download the matching **DSpark Q8_0** draft model: ``` hf download \ ggml-org/Qwen3-8B-GGUF \ dspark-Qwen3-8B-Q8_0.gguf \ --local-dir /workspace/models ``` The first file is the main model that generates the final output. The smaller DSpark model will generate speculative draft tokens for the target model to verify. Confirm that both files are available: ``` ls -lh /workspace/models ``` You should see something similar to: ``` 4.7G Qwen3-8B-Q4_K_M.gguf 1.2G dspark-Qwen3-8B-Q8_0.gguf ``` With **llama.cpp** built and both models downloaded, we can first measure the normal Qwen3-8B generation speed before enabling speculative decoding. ## 2. Measuring the Baseline Speed Before enabling DSpark, we need a baseline. We will run **Qwen3-8B normally** and record its generation speed so we can compare it against the speculative-decoding run. Move into the llama.cpp directory: ``` cd /workspace/llama.cpp ``` Run Qwen3-8B without speculative decoding: ``` ./build/bin/llama-cli \ -m /workspace/models/Qwen3-8B-Q4_K_M.gguf \ -ngl all \ -fa on \ --temp 0 \ --top-k 1 \ -n 512 \ -st \ -p "Write a complete Python implementation of merge sort. Explain how it works and include its time and space complexity. /no_think" ``` Here, `-ngl all` offloads all model layers to the GPU, while `-fa on` enables Flash Attention. We also use deterministic decoding: ``` --temp 0 --top-k 1 ``` This is important because we will use the **same prompt, token limit, and decoding settings** when testing DSpark, giving us a cleaner apples-to-apples comparison. When generation finishes, look for the benchmark summary printed by **llama.cpp**: ``` [ Prompt: 294.6 t/s | Generation: 95.0 t/s ] ``` For this guide, the important number is **Generation: 95.0 tokens/s**. We will use this as our baseline when measuring the DSpark speedup. ## 3. Running the Same Test With DSpark Now we will repeat the benchmark with **DSpark enabled**. The goal is to keep the target model, prompt, token limit, and decoding settings the same so we can directly measure the effect of speculative decoding. Run the same Qwen3-8B model, this time with the DSpark draft model attached: ``` ./build/bin/llama-cli \ -m /workspace/models/Qwen3-8B-Q4_K_M.gguf \ -md /workspace/models/dspark-Qwen3-8B-Q8_0.gguf \ --spec-type draft-dspark \ --spec-draft-n-max 3 \ -ngl all \ -ngld all \ -fa on \ --temp 0 \ --top-k 1 \ -n 512 \ -st \ -p "Write a complete Python implementation of merge sort. Explain how it works and include its time and space complexity. /no_think" ``` Here, `-md` loads the DSpark draft model, while `--spec-type draft-dspark` enables DSpark speculative decoding. `--spec-draft-n-max 3` allows DSpark to draft up to three tokens at a time, and `-ngld all` offloads the draft model to the GPU. When the run finishes, record the generation speed: ``` [ Prompt: 88.0 t/s | Generation: 124.9 t/s ] ``` Now compare it with our baseline: | Configuration | Prompt Speed | Generation Speed | |---|---|---| | Qwen3-8B baseline | 294.6 t/s | 95.0 t/s | | Qwen3-8B + DSpark | 88.0 t/s | 124.9 t/s | DSpark increases generation throughput from **95.0 to 124.9 tokens/s**. That is about a **1.31× speedup**, or roughly **31.5% faster generation**, using the same target model and GPU. The prompt-processing speed is lower in the DSpark run, but the main benefit we are measuring is **autoregressive generation speed**. For workloads that generate longer responses, the higher token-generation throughput can have a much larger impact on overall inference time. ## Final Thoughts For local LLM acceleration, I still think **MTP is often the more practical option**, especially because it is simpler and available across a wider range of models. However, **DSpark can have an edge over basic multi-token prediction** in cases where better draft quality leads to more accepted speculative tokens. The good thing is that DSpark is very easy to set up in **llama.cpp**. The bigger limitation is **model support**: only a small number of models currently have compatible DSpark draft models available. Support in **llama.cpp** is also still relatively new, so you may run into bugs or instability depending on the model and build you are using. For now, DSpark is an interesting acceleration technique to experiment with, but **MTP remains the more broadly useful option for local inference**. ( [Abid Ali Awan](https://abid.work) [@1abidaliawan](https://www.linkedin.com/in/1abidaliawan)) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master's degree in technology management and a bachelor's degree in telecommunication engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.