{"slug": "llama-cpp-1-0-qwen3-8-27b-vulkan-vs-sycl-on-intel-arc-b390", "title": "llama.cpp 1.0 + Qwen3.8-27B: Vulkan vs SYCL on Intel Arc B390", "summary": "A benchmark of llama.cpp build 1386 (commit a956a852c) on an Intel Arc B390 GPU found the Vulkan backend processed a 512-token prompt at 271.35 tokens per second versus 98.21 t/s for SYCL, making Vulkan 2.76× faster for prompt processing. In token generation over 128 tokens, SYCL reached 6.06 t/s against Vulkan's 5.89 t/s, a roughly 3% edge the author called essentially a tie. The test ran the Qwen3.8-27B Q4_K_M model (15.65 GiB, 27.32B parameters) fully offloaded to the GPU on a Xiaomi Book Pro 14 with an Intel Core Ultra X7 358H and 30.7 GB shared memory, using Mesa 24.3 for Vulkan and Intel oneAPI 2026.0 for SYCL.", "body_md": "# llama.cpp 1.0 + Qwen3.8-27B: Vulkan vs SYCL on Intel Arc B390\n\n**llama.cpp version:** `a956a852c` (commit: \"Fix ctz64 on 32bit (#32273)\", build 1386)**Model:** Qwen3.8-27B-thinkingcap-abliterated-mtp-Q4_K_M (15.65 GiB, 27.32B params)**Hardware:** Xiaomi Book Pro 14 — Intel Core Ultra X7 358H + **Intel Arc B390 GPU** (96 compute units, 30.7 GB shared memory)**OS:** CachyOS (Arch-based), Linux 6.12+, Mesa 24.3+, Intel oneAPI 2026.0\n\n## Executive Summary\n\n| Test | **Vulkan** | **SYCL** | **Winner** | \n|---|---|---|---|\n| **Prompt Processing (pp512)** | **271.35 t/s** | 98.21 t/s | **Vulkan 2.76× faster** | \n| **Token Generation (tg128)** | 5.89 t/s | **6.06 t/s** | SYCL ~3% faster | \n\n**Vulkan dominates prompt processing** — critical for long-context workloads, RAG, and batch inference.**SYCL edges out token generation** — marginal advantage for interactive chat.\n\n## Hardware & Software Stack\n\n| Component | Version | \n|---|---|\n| **GPU** | Intel Arc B390 (Battlemage, 96 EUs) | \n| **CPU** | Intel Core Ultra X7 358H (16c/16t, Panther Lake) | \n| **RAM** | 30 GB LPDDR5x + zram swap | \n| **Vulkan Driver** | Mesa 24.3 (open-source, `intel_hasvk` ) | \n| **SYCL Runtime** | Intel oneAPI 2026.0 (Level Zero 1.34, oneDNN 2026.0) | \n| **Compiler (Vulkan)** | Clang 23.1.4 | \n| **Compiler (SYCL)** | Intel icx/icpx 2026.0 | \n| **llama.cpp** | a956a852c (v1.0.0-dev, build 1386) | \n\n## Build Configurations\n\n### Vulkan (Mesa / Open Source)\n\n```\ncmake -B build-vulkan \\\n  -DCMAKE_BUILD_TYPE=Release \\\n  -DGGML_VULKAN=ON \\\n  -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_SYCL=OFF -DGGML_METAL=OFF -DGGML_OPENCL=OFF \\\n  -DBUILD_SHARED_LIBS=ON -DGGML_NATIVE=ON -DGGML_OPENMP=ON \\\n  -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++\n```\n\n- Uses **Mesa's `hasvk` driver** with`KHR_cooperative_matrix` (matrix cores)\n- No proprietary Intel stack required\n- Supports fp16, bf16, int8 dot products\n\n### SYCL (Intel oneAPI / Proprietary)\n\n```\nsource /opt/intel/oneapi/setvars.sh\ncmake -B build-sycl-linux \\\n  -DCMAKE_BUILD_TYPE=Release \\\n  -DGGML_SYCL=ON -DGGML_SYCL_DNN=ON -DGGML_SYCL_GRAPH=ON \\\n  -DGGML_SYCL_TARGET=INTEL -DGGML_SYCL_F16=OFF \\\n  -DGGML_SYCL_SUPPORT_LEVEL_ZERO_API=ON -DGGML_SYCL_SUPPORT_XMX=ON \\\n  -DGGML_SYCL_HOST_MEM_FALLBACK=ON \\\n  -DGGML_OPENCL=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=OFF -DGGML_METAL=OFF \\\n  -DBUILD_SHARED_LIBS=ON -DGGML_NATIVE=ON -DGGML_OPENMP=ON \\\n  -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx\n```\n\n- Uses **Intel Level Zero** +**oneDNN** +**XMX (DPAS) matrix instructions**\n- Requires proprietary oneAPI toolkit\n- Enables: flash attention, graph capture, XMX ESIMD kernels, host-pinned memory\n\n## Benchmark Methodology\n\n```\n# SYCL (requires oneAPI env)\nsource /opt/intel/oneapi/setvars.sh\n./llama-bench -m Qwen3.8-27B-thinkingcap-abliterated-mtp-Q4_K_M.gguf -p 512 -n 128 -ngl 99\n\n# Vulkan (runs directly)\n./llama-bench -m Qwen3.8-27B-thinkingcap-abliterated-mtp-Q4_K_M.gguf -p 512 -n 128 -ngl 99\n```\n\n- **Model fully offloaded to GPU** (`-ngl 99` )\n- **5 warmup runs + 5 measured runs** (default)\n- **Single-threaded CPU** (`-t 1` ) to isolate GPU performance\n- **Cache types:** K=f16, V=f16 (default)\n- **Batch size:** 2048,**Ubatch:** 512 (default)\n\n## Results\n\n### Prompt Processing (pp512) — 512 token prompt ingestion\n\n| Backend | Throughput | StdDev | Relative | \n|---|---|---|---|\n| **Vulkan** | **271.35 t/s** | ±1.21 | **1.00× (baseline)** | \n| SYCL | 98.21 t/s | ±0.70 | 0.36× | \n\n**Vulkan is 2.76× faster** for prompt processing.\n\nThis is the **most impactful metric** for:\n\n- Long-context ingestion (RAG, document analysis)\n- Batch/parallel request processing\n- Time-to-first-token in server deployments\n\n### Token Generation (tg128) — 128 token continuation\n\n| Backend | Throughput | StdDev | Relative | \n|---|---|---|---|\n| SYCL | **6.06 t/s** | ±0.01 | **1.00× (baseline)** | \n| Vulkan | 5.89 t/s | ±0.06 | 0.97× | \n\n**SYCL is ~3% faster** for token generation — essentially a tie within noise.\n\nThis metric matters for:\n\n- Interactive chat latency\n- Streaming response speed\n- Single-user throughput\n\n## Performance Histogram\n\n*Chart generated from 5-run averages per backend*\n\n## Why the Difference?\n\n| Factor | Vulkan Advantage | SYCL Advantage | \n|---|---|---|\n| **Matrix cores** | `KHR_cooperative_matrix` (Mesa) | XMX/DPAS via oneDNN + ESIMD | \n| **Memory model** | Unified Memory (UMA) — zero-copy | USM device + host-pinned staging | \n| **Kernel fusion** | Limited (shader-based) | oneDNN graph fusion + flash attention | \n| **Driver overhead** | Lower (Mesa userspace) | Higher (Level Zero + JIT) | \n| **Compiler** | Clang (mature, optimized) | icx/icpx (aggressive vectorization) | \n| **Flash Attention** | Not yet in Vulkan backend | Enabled via oneDNN ( `GGML_SYCL_FA_ONEDNN=1` ) | \n\n### Prompt Processing (Memory-Bound)\n\nVulkan wins because:\n\n- **Lower driver overhead** for large batched matmuls\n- **Direct UMA access** — no staging copies for prompt ingestion\n- **Mesa's cooperative matrix** efficiently maps to Xe² cores\n\n### Token Generation (Compute-Bound, Small Matmuls)\n\nSYCL wins marginally because:\n\n- **oneDNN optimized kernels** for small GEMMs (1×128, 32×32)\n- **XMX/DPAS instructions** better utilized at low batch sizes\n- **Graph capture** reduces launch overhead for repeated decode steps\n\n## Practical Recommendations\n\n| Use Case | Recommended Backend | \n|---|---|\n| **RAG / Document QA** | **Vulkan** (2.76× faster prompt ingestion) | \n| **Batch inference server** | **Vulkan** (higher throughput) | \n| **Interactive chat (single user)** | **Either** (tg128 within noise) | \n| **Long-context (>32k)** | **Vulkan** (better memory scaling) | \n| **No proprietary drivers wanted** | **Vulkan** (Mesa only) | \n| **Maximum flash-attn optimization** | **SYCL** (oneDNN FA enabled) | \n\n## Raw Data\n\n```\n# llama.cpp version\nbuild: a956a852c (1386)\n\n# SYCL\n| qwen35 27B Q4_K - Medium | 15.65 GiB | 27.32 B | SYCL | 99 | pp512 | 98.21 ± 0.70 |\n| qwen35 27B Q4_K - Medium | 15.65 GiB | 27.32 B | SYCL | 99 | tg128 | 6.06 ± 0.01 |\n\n# Vulkan\n| qwen35 27B Q4_K - Medium | 15.65 GiB | 27.32 B | Vulkan | 99 | pp512 | 271.35 ± 1.21 |\n| qwen35 27B Q4_K - Medium | 15.65 GiB | 27.32 B | Vulkan | 99 | tg128 | 5.89 ± 0.06 |\n```\n\n## Reproducing\n\n```\n# Clone llama-bench (llama.cpp fork with bench tooling)\ngit clone https://github.com/ggml-org/llama.cpp llama-bench\ncd llama-bench\n\n# Build Vulkan\ncmake -B build-vulkan -DGGML_VULKAN=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ ...\ncmake --build build-vulkan -j$(nproc)\n\n# Build SYCL (requires Intel oneAPI)\nsource /opt/intel/oneapi/setvars.sh\ncmake -B build-sycl-linux -DGGML_SYCL=ON -DGGML_SYCL_DNN=ON ... -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx\ncmake --build build-sycl-linux -j$(nproc)\n\n# Run benchmarks\n./build-vulkan/bin/llama-bench -m <model> -p 512 -n 128 -ngl 99\nsource /opt/intel/oneapi/setvars.sh && ./build-sycl-linux/bin/llama-bench -m <model> -p 512 -n 128 -ngl 99\n```\n\n## Conclusion\n\n**For Intel Arc B390 on Linux today: Vulkan is the better default.**\n\n- **2.76× faster prompt processing** — huge for real-world workloads\n- **No proprietary stack** — Mesa works out of the box\n- **Token generation parity** — SYCL's 3% edge is negligible\n\nSYCL remains valuable for:\n\n- Flash Attention workloads (oneDNN integration)\n- Maximum compute utilization on Xe² matrix cores\n- Developers already invested in oneAPI ecosystem\n\nAs Mesa's `hasvk` driver matures and adds cooperative matrix fp16/bf16 accumulation, Vulkan's lead will likely grow.", "url": "https://wpnews.pro/news/llama-cpp-1-0-qwen3-8-27b-vulkan-vs-sycl-on-intel-arc-b390", "canonical_source": "https://grigio.org/llama-cpp-1-0-qwen3-8-27b-vulkan-vs-sycl-on-intel-arc-b390/", "published_at": "2026-11-14 23:14:05+00:00", "updated_at": "2026-11-14 23:22:49.827859+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools", "mlops"], "entities": ["llama.cpp", "Qwen3.8-27B", "Intel Arc B390", "Intel Core Ultra X7 358H", "Intel oneAPI", "Mesa", "Xiaomi Book Pro 14", "CachyOS"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/llama-cpp-1-0-qwen3-8-27b-vulkan-vs-sycl-on-intel-arc-b390", "markdown": "https://wpnews.pro/news/llama-cpp-1-0-qwen3-8-27b-vulkan-vs-sycl-on-intel-arc-b390.md", "text": "https://wpnews.pro/news/llama-cpp-1-0-qwen3-8-27b-vulkan-vs-sycl-on-intel-arc-b390.txt", "jsonld": "https://wpnews.pro/news/llama-cpp-1-0-qwen3-8-27b-vulkan-vs-sycl-on-intel-arc-b390.jsonld"}}