# A 27B Model That Reverse-Engineered a License Check (And How to Run It)

> Source: <https://blog.herlein.com/post/qwen38-27b-gx10-setup/>
> Published: 2026-08-23 08:00:02+00:00

XDA ran a piece that stopped me mid-scroll: [a 27B open-weights model reverse-engineered a commercial application’s license check](https://www.xda-developers.com/qwen-3-8-27b-reverse-engineering-job-frontier-model/) — recovered a deliberately obscured crypto key out of ARM64 assembly, caught and corrected its own mistake without being told, and produced a working bypass PoC. In about thirty minutes. On a desktop box. I have (a version of) that box. So I went and set it up.

The model is **Qwen3.8-27B**. Artificial Analysis has it as the top open-weights model in the 4B–40B class out of 135 models, at 52 on their intelligence index. The hardware in the article was a Lenovo ThinkStation PGX — NVIDIA Grace Blackwell, 128 GB unified memory, 273 GB/s. Mine is an [ASUS Ascent GX10](https://www.asus.com/us/networking-iot-servers/desktop-ai-supercomputer/ultra-small-ai-supercomputers/asus-ascent-gx10/), which is the same GB10 silicon in a different box.

I’ve been building toward this for a while. In May I wrote that [AI inference costs are the wake-up call for 2026 and 2027](https://blog.herlein.com/post/ai-inference-costs-reality-check/). In July I argued that [tokens should be NRE, not COGS](https://blog.herlein.com/post/tokens-are-nre-not-cogs/). Earlier this month I [benchmarked the GX10 against a ZBook Ultra G1a](https://blog.herlein.com/post/gx10-vs-zbook-g1a-coding-agent/) on a real agentic coding task and came away convinced the small box is the one you actually reach for.

This post is the missing piece: the actual setup. What follows is the recipe I ran, corrected against a real GX10 bring-up, including the parts that bit me.

## Before You Start: Two Things That Will Ruin Your Day

**Disk.** You need about 100 GB free. The base container image alone is 57 GB, weights are ~19 GB, caches on top of that.

```
df -h ~
```

**Free the unified memory first.** This is the one that got me. SGLang runs at `--mem-fraction-static 0.90`

, which claims ~90% of system RAM. Anything else already resident will collide — and on GB10 that collision doesn’t produce a nice error, it produces a **hard reboot** during CUDA-graph capture.

The usual culprit is `ollama`

, which pins its model in unified memory and sits there.

```
nvidia-smi                    # check the Processes list at the bottom
sudo systemctl stop ollama
sudo systemctl disable ollama # stop it grabbing memory on boot
nvidia-smi                    # want: "No running processes found"
free -h                       # want: ~100+ GB free
```

GB10 quirk:`nvidia-smi`

reports`Memory-Usage: Not Supported`

in the summary table. That’s normal for unified memory — it’s not broken. Read theprocess listat the bottom, or use`free -h`

.

Also note `free -h`

shows about **121 GB**, not 128 — firmware and system reserve the rest. At 0.90 that means SGLang takes ~109 GB with comfortable headroom. Don’t “fix” this. `0.95`

reboots the box.

## OS and Stack Check

The GX10 ships DGX OS, but a stock **Ubuntu 24.04 LTS (arm64)** install works fine and is fully supported — driver, CUDA, Docker, Container Toolkit all run natively. Mine’s Ubuntu. Confirm what you have:

```
cat /etc/os-release     # Ubuntu 24.04 (noble) or DGX OS — either is fine
docker version          # arm64 engine; 29.x is current
nvidia-ctk --version    # NVIDIA Container Toolkit present
nvidia-smi              # GB10 visible, driver loaded
```

A healthy box: `NVIDIA GB10`

, driver 580.x, CUDA 13.x, Docker 29.x `linux/arm64`

, Container Toolkit 1.19.x. If Docker or the toolkit are missing:

```
sudo apt update
sudo apt install -y docker.io
sudo systemctl enable --now docker
sudo usermod -aG docker "$USER"          # log out and back in after this
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
```

Optionally set a Hugging Face token for faster pulls:

```
echo 'export HF_TOKEN=hf_xxxxxxxxxxxxxxxxx' >> ~/.bashrc && source ~/.bashrc
```

## Clone and Configure

```
cd ~
git clone https://github.com/MiaAI-Lab/Qwen3.8-27B-SGLang-DGX-Spark.git
cd Qwen3.8-27B-SGLang-DGX-Spark
cp .env.sample .env
```

The defaults are sane for code work: NVFP4 weights, native 262K context, YaRN off, 16 concurrent requests.

**One hard rule: keep YARN=0 and CONTEXT_LENGTH=262144.** DFlash2 is not compatible with YaRN context extension on this build. A larger context leaking into the draft config throws

`AttributeError ... max_position_embeddings`

at boot and you’ll spend an hour thinking it’s a driver problem. Confirm before you launch:

```
grep -E '^YARN|^CONTEXT_LENGTH' .env
```

Also: the repo hardcodes `PORT=8888`

and `SERVED_MODEL_NAME=qwen3.8-27b-sglang`

. Overriding either one requires patching `start.sh`

. I didn’t bother — I use 8888 and read the served model id from `/v1/models`

. (The patch is a two-line change if you want it; I put it at the end.)

One caveat worth saying out loud: **this upstream repo is new and moving.** Sanity-check script names and `.env`

keys against the current README before you run anything. Don’t trust my transcription over their repo.

## Build and Launch

Build the DFlash2 image — a pure local overlay onto the pinned SGLang image, with every overlaid file checksum-verified:

``` php
./patch/build-dflash2-image.sh --minimal
# -> lmsysorg/sglang:qwen38-27b-dflash2-minoverlay
```

First run pulls ~57 GB. Go do something else. Take a nap. Or a walk. It takes a bit, even with a fast network.

Then launch:

```
DF_EXTRA="--sleep-on-idle" \
IMAGE=lmsysorg/sglang:qwen38-27b-dflash2-minoverlay \
./start-dflash.sh
```

`--sleep-on-idle`

matters more than it looks. Without it the SGLang scheduler busy-spins at ~97% CPU doing *nothing*, which on a small fanless-ish box you can hear and feel.

First boot downloads weights and compiles kernels: **10–20 minutes, mostly silent.** Weight load alone is about 5 minutes. Warm restarts are 2–3 minutes. Watch it in another pane:

```
tail -f .sglang.log
```

You’re up when you see `Uvicorn running on http://0.0.0.0:8888`

and `Application startup complete`

.

## Verify

Fair warning on the first thing that’ll confuse you: `curl -s`

swallows connection errors, so an empty reply piped to `json.tool`

gives you `Expecting value: line 1 column 1`

. That almost always means **not listening yet**, not crashed. Check `docker ps`

and the log before you panic.

```
PORT=8888

# Is it up? Also grabs the real served model id.
MODEL=$(curl -s http://127.0.0.1:$PORT/v1/models \
  | python3 -c 'import sys,json;print(json.load(sys.stdin)["data"][0]["id"])')
echo "Serving model id: $MODEL"

# A code-shaped call with thinking OFF — the fast lane:
curl -s http://127.0.0.1:$PORT/v1/chat/completions \
  -H 'Content-Type: application/json' -d "{
    \"model\": \"$MODEL\",
    \"messages\": [{\"role\":\"user\",\"content\":\"Write an idiomatic Go function that returns the nth Fibonacci number iteratively.\"}],
    \"max_tokens\": 300,
    \"temperature\": 0.2,
    \"chat_template_kwargs\": {\"enable_thinking\": false},
    \"stream_options\": {\"include_usage\": true}
  }" | python3 -m json.tool
```

Logs are `tail -f .sglang.log`

or `docker logs -f qwen3.8-27b-sglang`

. Stop with `./stop.sh`

.

## Pointing Your Agents At It

The server exposes three protocols on `http://<gx10-ip>:8888`

:

**OpenAI Chat Completions**—`/v1/chat/completions`

, works out of the box**OpenAI Responses**—`/v1/responses`

, needs a compat patch for strict SDK clients**Anthropic-style**—`/v1/messages`

For **OpenAI-compatible agents** — aider, opencode, Cline, Continue, and the ones I’ve been [testing lately](https://blog.herlein.com/post/skills-mapper-sm/):

```
export OPENAI_BASE_URL="http://<gx10-ip>:8888/v1"
export OPENAI_API_KEY="sk-local"     # any non-empty string; the server ignores it
```

aider, concretely:

```
aider --openai-api-base http://<gx10-ip>:8888/v1 \
      --openai-api-key sk-local \
      --model openai/qwen3.8-27b-sglang
```

**Claude Code** speaks the Anthropic protocol and `/v1/messages`

is right there:

```
export ANTHROPIC_BASE_URL="http://<gx10-ip>:8888"
export ANTHROPIC_API_KEY="sk-local"
```

Treat that one as best-effort. This build 400s on some exotic Responses/message item types — MCP tool-call echoes, `web_search_call`

echoes, `item_reference`

. Plain function tools and chat work fully. For predictable agentic coding *today*, the OpenAI-compatible path is the tested one.

By the way — the XDA reverse-engineering run used the `pi`

harness, which is worth knowing. The harness matters as much as the endpoint. I’ve made [that argument at length](https://blog.herlein.com/post/dot-agents/).

## The Explicit `docker run`

If you’d rather own every flag than trust a launcher — and honestly, for something you’re going to run every day, you should read this at least once:

```
PORT=8888
IMAGE=lmsysorg/sglang:qwen38-27b-dflash2-minoverlay
CACHE=$HOME/Qwen3.8-27B-SGLang-DGX-Spark/.cache

docker run -d --name qwen38-code \
  --network host --ipc host --privileged --gpus all --shm-size 32g \
  -e HF_HOME=/root/.cache/huggingface \
  -e TRITON_CACHE_DIR=/root/.triton \
  -v ${CACHE}/huggingface:/root/.cache/huggingface \
  -v ${CACHE}/triton:/root/.triton \
  ${IMAGE} python3 -m sglang.launch_server \
    --model-path RadixArk/Qwen3.8-27B-NVFP4 \
    --served-model-name qwen38-27b --trust-remote-code \
    --tp 1 \
    --attention-backend flashinfer --chunked-prefill-size 8192 \
    --disable-prefill-cuda-graph --kv-cache-dtype fp8_e4m3 \
    --mamba-ssm-dtype bfloat16 --mamba-full-memory-ratio 4.21 \
    --mamba-radix-cache-strategy extra_buffer \
    --max-mamba-cache-size 64 --max-running-requests 16 \
    --context-length 262144 \
    --speculative-algorithm DFLASH \
    --speculative-draft-model-path z-lab/Qwen3.8-27B-DFlash2 \
    --speculative-draft-model-revision 50307d4c4cde6860d4eee73e2547cd786fe8e8a4 \
    --speculative-num-draft-tokens 8 \
    --reasoning-parser qwen3 --tool-call-parser qwen3_coder \
    --sampling-defaults model --enable-metrics --enable-cache-report \
    --stream-interval 1 --sleep-on-idle \
    --mem-fraction-static 0.90 --host 0.0.0.0 --port ${PORT}
```

The flags that actually matter for code: `--speculative-num-draft-tokens 8`

(block size), `--tool-call-parser qwen3_coder`

(agent function calls — you need this), `--stream-interval 1`

(per-token SSE), `--mem-fraction-static 0.90`

(the safe headroom), `--kv-cache-dtype fp8_e4m3`

(keeps KV memory sane at 262K context).

Setting `--served-model-name qwen38-27b`

here makes the model id `qwen38-27b`

regardless of the repo default — which is the easy way around the port/name patch if you’re going explicit anyway.

## Conclusion

Here’s what I keep circling back to.

A 27B model, quantized to 4 bits, running locally, did a static reverse-engineering job on ARM64 assembly — including *catching its own error and fixing it* — in thirty minutes. Unattended. Off the network.

That is not the story the “local models are toys” crowd has been telling. It’s also not the story the “you need a trillion parameters” crowd has been telling. Both of them are about to be wrong in the same direction, and I’ve said before that [today’s models are the worst you’ll ever use](https://blog.herlein.com/post/agent-naysayers/).

I’m not claiming this replaces a frontier model for everything. It doesn’t, and I’ll write that post honestly when I’ve run it against enough real work. What I *am* claiming is that the line between “send this to the cloud” and “run this locally” moved, again, and it moved further than I expected it to this quarter. Every time it moves, the NRE-versus-COGS math gets better for owning the hardware.

Setup is an afternoon, most of it waiting on a 57 GB pull. Go run it and find out where your own line is.

**What’s next for me:** running this endpoint under the coding agents I’ve been evaluating, and seeing whether a local 27B holds up in a real agentic loop the way it did on a single well-scoped analysis task. Those are very different asks. That’s the post I actually want to write.

Credit where it’s due: the XDA piece by way of Artificial Analysis’s benchmarks is what sent me down this road, and the [MiaAI-Lab repo](https://github.com/MiaAI-Lab/Qwen3.8-27B-SGLang-DGX-Spark) is doing the unglamorous packaging work that makes this a one-afternoon job instead of a one-week job.

If you get this running — or if you hit something my troubleshooting table doesn’t cover — drop me a note on LinkedIn: [linkedin.com/in/gherlein](https://www.linkedin.com/in/gherlein/). I’ll add it.