{"slug": "run-a-4b-llm-model-locally-on-your-pc-this-has-been-tested-on-windows-10", "title": "Run a 4B LLM model locally on your PC, this has been tested on Windows 10", "summary": "A developer has published a step-by-step guide for running a 4B-parameter LLM locally on a Windows 10 PC using Mozilla's llamafile tool and a Qwen3 model from Hugging Face. The guide details downloading llamafile v0.10.5 and the qwen3-4b-thinking-2507.Q4_K_M.gguf file, then launching a local server on 127.0.0.1:8080.", "body_md": "| Run a 4B LLM model locally on your PC, this has been tested on Windows 10 | |\n| Step 1: | |\n| Lookup \"llamafile github\" or go to https://github.com/mozilla-ai/llamafile | |\n| Download the latest release. As of this writing, the latest version is llamafile v0.10.5 | |\n| The file would be llamafile-0.10.5 | |\n| Rename the file to llamafile-0.10.5.exe | |\n| Step 2: | |\n| Go to https://huggingface.co/models | |\n| Search for qween3 | |\n| Click on pramodlohra/Qween3_4B_thinking_finetune | |\n| Go to Files and versions | |\n| Download qwen3-4b-thinking-2507.Q4_K_M.gguf | |\n| Step 3: | |\n| Create a folder called qwen | |\n| Copy llamafile-0.10.5.exe and qwen3-4b-thinking-2507.Q4_K_M.gguf into this folder | |\n| Step 4: | |\n| In this folder, create a file called run_qwen.cmd | |\n| Write the following lines in it | |\n| @echo off | |\n| cd /d \"%~dp0\" | |\n| llamafile-0.10.5.exe --server --model \"qwen3-4b-thinking-2507.Q4_K_M.gguf\" | |\n| pause | |\n| Save the run_qwen.cmd file | |\n| Now you are ready to click on the run_qwen.command file and run the model locally | |\n| If everything is done correctly, something like this is what would show up in the command window | |\n| 0.00.000.673 I cmn common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg) | |\n| 0.00.000.921 W srv llama_server: ----------------- | |\n| 0.00.000.926 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set | |\n| 0.00.000.926 W srv llama_server: this can be a security risk (cross-origin attacks) | |\n| 0.00.000.927 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 | |\n| 0.00.000.927 W srv llama_server: ----------------- | |\n| 0.00.001.045 W srv llama_server: sandbox: disabled in GPU mode | |\n| 0.00.002.429 I srv load_model: loading model './qwen3-4b-thinking-2507.Q4_K_M.gguf' | |\n| 0.00.429.537 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden | |\n| 0.04.590.135 I srv load_model: initializing, n_slots = 4, n_ctx_slot = 14080, kv_unified = 'true' | |\n| 0.04.600.756 I srv llama_server: model loaded | |\n| 0.04.600.778 I srv llama_server: listening on http://127.0.0.1:8080 | |\n| You can start using the model on 127.0.0.1:8080, Enjoy! |", "url": "https://wpnews.pro/news/run-a-4b-llm-model-locally-on-your-pc-this-has-been-tested-on-windows-10", "canonical_source": "https://gist.github.com/barshasantak/7ea564a5e5ddaf7e955c85d73360eb94", "published_at": "2026-08-12 23:08:24+00:00", "updated_at": "2026-08-12 23:19:38.298232+00:00", "lang": "en", "topics": ["large-language-models", "developer-tools"], "entities": ["Mozilla", "Hugging Face", "llamafile", "Qwen3", "pramodlohra"], "alternates": {"html": "https://wpnews.pro/news/run-a-4b-llm-model-locally-on-your-pc-this-has-been-tested-on-windows-10", "markdown": "https://wpnews.pro/news/run-a-4b-llm-model-locally-on-your-pc-this-has-been-tested-on-windows-10.md", "text": "https://wpnews.pro/news/run-a-4b-llm-model-locally-on-your-pc-this-has-been-tested-on-windows-10.txt", "jsonld": "https://wpnews.pro/news/run-a-4b-llm-model-locally-on-your-pc-this-has-been-tested-on-windows-10.jsonld"}}