LLM Scaler is an GenAI solution for text generation, image generation, video generation etc. running on Intel® Arc™ Pro B60 and B70 GPUs. LLM Scalar leverages standard frameworks such as vLLM, ComfyUI, SGLang Diffusion, Xinference etc and ensures the best performance for State-of-Art GenAI models running on Arc Pro B60/B70 GPUs.
- 🔥[2026.08] We released
`intel/llm-scaler-vllm:0.21.0-b2`
to support Multi-token Prediction (MTP) and Lora Serving for Qwen3.6-27B, Qwen3.6-35B-A3B, gemma-4-31B-it and gemma-4-26B-A4B-it models, and support per-block quantization models Qwen3.6-27B-FP8 and Qwen3.6-35B-A3B-FP8. - [2026.07] We released
intel/llm-scaler-omni:0.1.0-b8
to support ComfyUI 0.27.0,more workflows and models. - [2026.07] We released
`intel/llm-scaler-vllm:0.21.0-b1`
to support gemma-4 (12B, 31B and 26B-A4B) and diffusiongemma (26B-A4B) models, and experimentally support XPU graph. - [2026.06] We released
`intel/llm-scaler-vllm:0.14.0-b8.3.2`
to fix Qwen3.5/3.6-27B accuracy issues. - [2026.06] We released
`intel/llm-scaler-vllm:0.14.0-b8.3.1`
to enable FP8 KV Cache and fix bugs for Qwen3/Qwen3.5 models. - [2026.05] We released
intel/llm-scaler-vllm:0.14.0-b8.3
to improve performance for Qwen3.5/3.6 series and Qwen3-Coder-Next, and enabled model streaming load to reduce peak memory. - [2026.05] We released
`intel/llm-scaler-vllm:1.4`
(or,`intel/llm-scaler-vllm:0.14.0-b8.2.1`
) with new platform image and support Intel® Arc™ Pro B70 GPU. - [2026.05] We released
`intel/llm-scaler-omni:0.1.0-b7`
for more model workflows and performance improvments. - [2026.03] We released
`intel/llm-scaler-vllm:0.14.0-b8.1`
to support Qwen3.5-27B, Qwen3.5-35B-A3B and Qwen3.5-122B-A10B (FP8/INT4 online quantization, GPTQ) - [2026.03] We released
`intel/llm-scaler-omni:0.1.0-b6`
for ComfyUI to support CacheDiT and torch.compile(), ComfyUI-GGUF, and more model workflows, and support FP8 for SGLang Diffusion. - [2026.03] We released
`intel/llm-scaler-vllm:0.14.0-b8`
for vLLM 0.14.0 and PyTorch 2.10 support, various new models support and performance improvement. - [2026.01] We released
`intel/llm-scaler-vllm:1.3`
(or,`intel/llm-scaler-vllm:0.11.1-b7`
) for vLLM 0.11.1 and PyTorch 2.9 support, various new models support and performance improvement. - [2026.01] We released
`intel/llm-scaler-omni:0.1.0-b5`
for Python 3.12 and PyTorch 2.9 support, various ComfyUI workflows and more SGLang Diffusion support. - [2025.12] We released
`intel/llm-scaler-vllm:1.2`
, same image as`intel/llm-scaler-vllm:0.10.2-b6`
. - [2025.12] We released
`intel/llm-scaler-omni:0.1.0-b4`
to support ComfyUI workflows for Z-Image-Turbo, Hunyuan-Video-1.5 T2V/I2V with multi-XPU, and experimentially support SGLang Diffusion. - [2025.11] We released
`intel/llm-scaler-vllm:0.10.2-b6`
to support Qwen3-VL (Dense/MoE), Qwen3-Omni, Qwen3-30B-A3B (MoE Int4), MinerU 2.5, ERNIE-4.5-vl etc. - [2025.11] We released
`intel/llm-scaler-vllm:0.10.2-b5`
to support gpt-oss models and released`intel/llm-scaler-omni:0.1.0-b3`
to support more ComfyUI workflows, and Windows installation. - [2025.10] We released
intel/llm-scaler-omni:0.1.0-b2
to support more models with ComfyUI workflows and Xinference. - [2025.09] We released
`intel/llm-scaler-vllm:0.10.0-b3`
to support more models (MinerU, MiniCPM-v-4.5 etc), and released`intel/llm-scaler-omni:0.1.0-b1`
to enable first omni GenAI models using ComfyUI and Xinference on Arc Pro B60 GPU. - [2025.08] We released
intel/llm-scaler-vllm:1.0
.
llm-scaler-vllm
supports running text generation models using vLLM, featuring:
support (P2P or USM)CCL andINT4 quantized online serving, plus pre-quantized FP8 model supportFP8 andEmbedding model supportReranker model supportMulti-Modal model supportOmni,** Tensor Paralleland Pipeline Parallel****Data Parallel**- Finding maximum Context Length
- Multi-Modal WebUI
- BPE-Qwen tokenizer
Please follow the instructions in the [Getting Started](/intel/llm-scaler/blob/main/vllm/README.md/#1-getting-started-and-usage) to use `llm-scaler-vllm`
.
| Model Name | FP16 | Dynamic Online FP8 | Dynamic Online Int4 | MXFP4 | Notes |
|---|---|---|---|---|---|
| openai/gpt-oss-20b | ✅ | ||||
| openai/gpt-oss-120b | ✅ | ||||
| deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B | ✅ | ✅ | ✅ | ||
| deepseek-ai/DeepSeek-R1-Distill-Qwen-7B | ✅ | ✅ | ✅ | ||
| deepseek-ai/DeepSeek-R1-Distill-Llama-8B | ✅ | ✅ | ✅ | ||
| deepseek-ai/DeepSeek-R1-Distill-Qwen-14B | ✅ | ✅ | ✅ | ||
| deepseek-ai/DeepSeek-R1-Distill-Qwen-32B | ✅ | ✅ | ✅ | ||
| deepseek-ai/DeepSeek-R1-Distill-Llama-70B | ✅ | ✅ | ✅ | ||
| deepseek-ai/DeepSeek-R1-0528-Qwen3-8B | ✅ | ✅ | ✅ | ||
| deepseek-ai/DeepSeek-V2-Lite | ✅ | ✅ | export VLLM_MLA_DISABLE=1 | ||
| deepseek-ai/deepseek-coder-33b-instruct | ✅ | ✅ | ✅ | ||
| Qwen/Qwen3-8B | ✅ | ✅ | ✅ | || | Qwen/Qwen3-14B | ✅ | ✅ | ✅ | || | Qwen/Qwen3-32B | ✅ | ✅ | ✅ | ||
| Qwen/Qwen3-30B-A3B | ✅ | ✅ | ✅ | ||
| Qwen/Qwen3-235B-A22B | ✅ | ||||
| Qwen/Qwen3-Coder-30B-A3B-Instruct | ✅ | ✅ | ✅ | ||
| Qwen/Qwen3-Coder-Next | ✅ | ✅ | |||
| Qwen/Qwen3.5/3.6-27B | ✅ | ✅ | ✅ | || | Qwen/Qwen3.5/3.6-35B-A3B | ✅ | ✅ | ✅ | || | Qwen/Qwen3.6-27B-FP8 | Pre-quantized offline FP8 model | ||||
| Qwen/Qwen3.6-35B-A3B-FP8 | Pre-quantized offline FP8 model | ||||
| Qwen/Qwen3.5-122B-A10B | ✅ | ✅ | |||
| Qwen/QwQ-32B | ✅ | ✅ | ✅ | ||
| mistralai/Ministral-8B-Instruct-2410 | ✅ | ✅ | ✅ | ||
| mistralai/Mixtral-8x7B-Instruct-v0.1 | ✅ | ✅ | ✅ | ||
| meta-llama/Llama-3.1-8B | ✅ | ✅ | ✅ | ||
| meta-llama/Llama-3.1-70B | ✅ | ✅ | ✅ | ||
| baichuan-inc/Baichuan2-7B-Chat | ✅ | ✅ | ✅ | with chat_template | | | baichuan-inc/Baichuan2-13B-Chat | ✅ | ✅ | ✅ | with chat_template | | | THUDM/CodeGeex4-All-9B | ✅ | ✅ | ✅ | with chat_template | |
| zai-org/GLM-4-9B-0414 | ✅ | use bfloat16 | |||
| zai-org/GLM-4-32B-0414 | ✅ | use bfloat16 | |||
| zai-org/GLM-4.5-Air | ✅ | ✅ | |||
| zai-org/GLM-4.7-Flash | ✅ | ✅ | |||
| ByteDance-Seed/Seed-OSS-36B-Instruct | ✅ | ✅ | ✅ | ||
| miromind-ai/MiroThinker-v1.5-30B | ✅ | ✅ | ✅ | ||
| tencent/Hunyuan-0.5B-Instruct | ✅ | ✅ | ✅ | follow the guide in |
hereReference CommandsReference Commandshere--quantization fp8
supports running image/voice/video generation etc., featuring Omni Studio
mode (using ComfyUI) and Omni Serving
mode (via SGLang Diffusion or Xinference).
Please follow the instructions in the Getting Started to use llm-scaler-omni
.
| Qwen-Image | Multi B60 Wan2.2-T2V-14B |
|---|---|
Omni Stuido
supports Image Generation/Edit, Video Generation, Audio Generation, 3D Generation etc.
| Model Category | Model | Type |
|---|---|---|
| Image Generation | ||
| Qwen-Image, Qwen-Image-Edit | Text-to-Image, Image Editing | |
| Image Generation | ||
| Stable Diffusion 3.5 | Text-to-Image, ControlNet | |
| Image Generation | ||
| Z-Image-Turbo | Text-to-Image | |
| Image Generation | ||
| Flux.1, Flux.1 Kontext dev | Text-to-Image, Multi-Image Reference, ControlNet | |
| Image Generation | ||
| FireRed-Image-Edit-1.1 | Image Editing | |
| Video Generation | ||
| Wan2.2 TI2V 5B, Wan2.2 T2V 14B, Wan2.2 I2V 14B | Text-to-Video, Image-to-Video | |
| Video Generation | ||
| Wan2.2 Animate 14B | Video Animation | |
| Video Generation | ||
| HunyuanVideo 1.5 8.3B | Text-to-Video, Image-to-Video | |
| Video Generation | ||
| LTX-2 | Text-to-Video, Image-to-Video | |
| 3D Generation | ||
| Hunyuan3D 2.1 | Text/Image-to-3D | |
| Audio Generation | ||
| VoxCPM1.5, IndexTTS 2 | Text-to-Speech, Voice Cloning | |
| Video Upscaling | ||
| SeedVR2 | Video Restoration and Upscaling |
Please check ComfyUI Support for more details.
Omni Serving
supports Image Generation, Audio Generation etc.
- Image Generation (
/v1/images/generations
): Stable Diffusion 3.5, Flux.1-dev - Text to Speech (
/v1/audio/speech
): Kokoro 82M - Speech to Text (
/v1/audio/transcriptions
): whisper-large-v3 Please check Xinference Support for more details.
- Please check out the Docker image releases for
llm-scaler-vllmandllm-scaler-omni
- Please report a bug or raise a feature request by opening a Github Issue