# Alibaba Ships Qwen3-8B as Ready-to-Run Local AI With Dual Reasoning Modes > Source: > Published: 2026-11-14 09:31:28+00:00 # Alibaba Ships Qwen3-8B as Ready-to-Run Local AI With Dual Reasoning Modes Alibaba's compact 8B reasoning model ships in quantized GGUF formats, offering hybrid thinking mode, 131K context, and local inference on modest hardware. - Qwen3-8B released as GGUF quantizations (Q4_K_M through Q8_0) for local inference on [Hugging Face](https://huggingface.co/Qwen/Qwen3-8B-GGUF?utm_source=alphasignal) . - Hybrid thinking: single checkpoint switches between reasoning and fast chat via /think and /no_think. - 8.2B dense params, 36 layers, GQA 32/8, 32K native context extendable to 131K with YaRN. - Outperforms QwQ and Qwen2.5-Instruct on math, code, and commonsense reasoning per the [technical report](https://arxiv.org/abs/2505.09388?utm_source=alphasignal) . - Runs out of the box on llama.cpp, Ollama, LM Studio, vLLM; Apache 2.0 license allows commercial use. - Avoid greedy decoding; use presence_penalty 1.5 on quantized builds to stop repetition. Alibaba’s Qwen team has published official quantized builds of Qwen3-8B on the [model page](https://huggingface.co/Qwen/Qwen3-8B-GGUF?utm_source=alphasignal). The repository provides `Q4_K_M` and `Q8_0` GGUF files for llama.cpp, recent Ollama releases, LM Studio, and compatible runtimes. Developers can run the same checkpoint in an extended reasoning mode or a faster direct-answer mode without converting the original weights. ## GGUF removes a conversion step GGUF packages model weights and runtime metadata in a format designed for local inference. Quantization stores weights at reduced precision, lowering storage and memory requirements while introducing a potential quality tradeoff. | Build | Approximate size | Practical tradeoff | |---|---|---| | `Q4_K_M` | 5.03 GB | Lower memory use and faster loading; suitable as a local default | | `Q8_0` | 8.71 GB | Higher weight precision with greater RAM and VRAM requirements | Total runtime memory exceeds the file size because inference also allocates a key-value cache, compute buffers, and prompt state. With a default 16-bit KV cache, a 131,072-token window for this architecture can consume roughly 18 GiB for the cache alone, before weights and other buffers. KV-cache quantization and partial CPU offload can reduce VRAM pressure, with corresponding speed or quality costs. ## Reasoning becomes a request-level switch Qwen3-8B supports thinking and non-thinking behavior within one checkpoint. Thinking mode emits intermediate reasoning inside `` blocks before the final response, while non-thinking mode answers directly with lower token use and latency. | Mode | Behavior | Control | |---|---|---| | Thinking | Generates an extended reasoning trace before the answer | `enable_thinking=true` or`/think` | | Non-thinking | Returns a direct answer | `enable_thinking=false` | ## This story is for Pro members You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.