Hmm… wouldn’t it be easier to establish a working baseline with an existing GGUF before experimenting with merges? :
I read this as two direct questions with several possible bottlenecks. Hardware limits, long prompts, coding accuracy, fireapi and model merging do not necessarily share a root cause. I would answer the two questions first, then choose only the troubleshooting branch that matches the symptom.
1. For a Qwen GGUF below the roughly 2–3 GB target: start with an existing Instruct quantization. The official Qwen2.5-Coder-1.5B GGUF provides qwen2.5-coder-1.5b-instruct-q4_k_m.gguf, about 1.12 GB (the tested file was 1,117,320,768 bytes). For a speed-oriented comparison, the official 0.5B GGUF has Q4_K_M around 491 MB; if coding quality is the bottleneck, the official 3B GGUF has Q4_K_M around 2.1 GB. These are file sizes, not RAM-use or old-i5 speed measurements; none guarantees correct DX12 code. Check each license for the intended use.
2. For low-overhead llama.cpp plus fireapi: if the native inference/dispatch path works, keep it; make one layer responsible for rendering the chat template, and handle action validation and authorization in trusted Windows host code. An SSD-hosted GGUF is just the model file; it does not require an extra HTTP server. llama-server is an optional alternative when several applications need one shared endpoint, not a prerequisite for native C integration.
I would avoid making a huge system prompt responsible for API correctness, code correctness and action permission. Keep permanent instructions short; supply only the relevant SDK declaration, project source and build error for the current coding task. The compiler can check the source; a host-side gate can check the proposed operation. This reduces what a long prompt must carry, although its speed and correctness still need measurement.
The public Fire-Gem v2.02 sources include native inference, ChatML formatting and dispatch. Its models.json already lists the same 1.5B Q4_K_M filename under E:\models. Its 1.1 MB size label conflicts with the real ~1.12 GB file, so treat the registry as a list rather than proof of file presence, size or runtime compatibility.
If the actual file is there and loads correctly, there is nothing to download. Otherwise, with the current Hugging Face hf CLI installed, a single-file download is enough:
hf download Qwen/Qwen2.5-Coder-1.5B-Instruct-GGUF qwen2.5-coder-1.5b-instruct-q4_k_m.gguf --local-dir "E:\models"
Use the SSD directory your current build actually reads; the example above reproduces a public v2.02 setting. This fetches one GGUF, not an F16 checkpoint or a weight merge. Record the actual file, llama.cpp build, CPU backend and context.
For a first check, I would use just three small requests, one at a time. Dry-run or log the dispatcher without executing actions if that option exists; otherwise restrict testing to its harmless popup, never shell commands:
Reply OK. No Windows action should occur. READY, if that capability exists in your current build. Exactly one intended event should occur.<firebox>EXAMPLE</firebox> or [ACTION: RunModuleCheck] syntax means, explicitly without executing it. The response may legitimately quote the marker; the host should treat that quotation as text.
Keep the request, raw generated text, dispatcher decision and actual event count separate. A quoted marker is legitimate prose, not permission to execute. No arbitrary shell action or administrator access is needed.
A. Only the first request is slow
Compare cold GGUF , SSD reads, backend/context initialization
and a second warm request. Do not call this low decoding speed.
B. Increasing the system prompt makes the first output much later
Check the exact rendered prompt and token count, truncation, prefill,
repeated processing, template duplication and context/cache reuse.
C. Even a short response produces each token slowly
Compare 0.5B/1.5B Q4 at the same prompt, CPU backend and physical-core
thread settings. Measure generation separately from prompt processing.
D. Memory pressure or paging appears
Reduce actual context, simultaneous requests and runtime buffers;
check total process RAM, not just the GGUF file or advertised VRAM.
E. Generated code is plausible but wrong
Retrieve only the relevant SDK headers/docs and project code,
compile a tiny example, and inspect the actual MSVC diagnostics.
F. The model names an action but Fire-Gem does not run it
Inspect the chat template, raw output, parser/dispatcher and approval
separately. If quoted prose triggers an action, separate text from
executable proposals before adjusting the weights.
G. The goal really is merging or custom compression
First identify which kind of merge is intended. Shard joining,
quantization, LoRA export and checkpoint fusion are different jobs.
The branches can coexist. Cold , prefill cost, code-generation mistakes and action-parser behavior should not be collapsed into a single claim that the model needs merging.
I ran a small Colab probe of that 1.5B Q4_K_M on Linux / NVIDIA L4 using a prebuilt JamePeng llama-cpp-python 0.4.7 CUDA wheel (no source build). The model and wheel were hash-checked; the log reported 29/29 GPU-offloaded layers, and GPU memory rose by about 1,342 MiB. All 11 worker generations completed, which is not 11 correct coding tasks or a measurement of this model on your i5.
Eight outputs were a small action-format check: four tasks with a public-style prompt, paired with four using an extra reminder not to execute quoted markers. Four of eight met their predefined criteria in this one-seed test. The requested <firebox>READY</firebox> action was produced, and a normal reply did not contain an action. All four quote-only explanation cases also included literal action markers that the public v2.02 strstr detector would flag; the extra reminder did not eliminate them.
This was generated text plus a static comparison with public C dispatcher logic, not a Windows run. It did not open a popup, run a batch file or inspect the current fireapi. A small L4 timing comparison does not predict old-i5 performance.
The public C code may differ from the current executable. If a current Fire-Gem already has guarded action parsing, retrieval or reusable context, reuse those facilities instead of rebuilding them.
A. If CPU inference and long prompts are the problem For the first pass, I would check the existing Qwen 1.5B Q4 file, then choose a tiny no-action/action/quote check or a short-versus-long timing check, whichever matches the problem you see. Either can identify the useful next branch. If it is mainly a coding-accuracy problem, use a targeted Windows SDK reference and the compiler; if it is an action-channel problem, inspect the host parser; if the quantized baseline itself is the obstacle, then model comparison, quantization or a genuine merge becomes worth the additional work. None of these paths requires replacing Fire-Gem’s working design.