Large language models are increasingly delivered as network-dependent services, yet their underlying runtimes (llama.cpp, vLLM, and derivatives) are trivially portable. This work addresses the gap between portable and deployable: constructing a fully offline, zero-install, GPU-accelerated LLM environment that boots from a removable USB drive on an Apple M4 Mac mini with 16 GB of unified memory. We analyze the llamafile distribution model (Cosmopolitan Libc fat binaries, mmap'd weight , runtime GPU compilation), the GGUF quantization ecosystem, and the memory/performance physics of Apple Silicon unified memory, and derive a set of falsifiable hypotheses (H1–H5) that an empirical benchmark phase will confirm or refute. We further identify and correct seven defects in a prior community guide — most critically a Windows-only binary selection, an overly conservative context window, and a missing first-run compiler dependency that silently disables GPU acceleration. The resulting plan is a complete, reproducible methodology: environment verification, controlled benchmarking via the llama.cpp metrics endpoint, removable-storage characterization, packaging, and security review.
Keywords: llamafile, llama.cpp, GGUF, offline inference, Apple Silicon, Metal, portable computing, quantization
State-of-the-art language models are conventionally consumed through hosted APIs. Local execution, when pursued, typically presupposes a developer environment: a package manager, a virtual environment, GPU driver stacks, and multi-gigabyte model registries. Each of these assumptions fails in the context of air-gapped, portable, or one-off computing — a clinician on an isolated ward network, a security reviewer in a vault, a field engineer on an aircraft, or a user who simply does not wish to install anything on a machine they do not own.
This paper develops the theory and an execution plan for the minimal viable offline LLM appliance: a single USB pendrive that, when inserted and double-clicked, serves a locally hosted, GPU-accelerated chat interface with no network connectivity and no system modifications.
Two technologies make this tractable:
- llamafile (Mozilla AI) — a single-file distribution of llama.cpp built on Cosmopolitan Libc, executable natively across macOS, Windows, and Linux without installation.
- GGUF — llama.cpp's self-describing model container, in which weights are stored at aggressively compressed precision (quantization), bringing a multi-billion-parameter model within reach of consumer memory.
The specific target platform is an Apple M4 Mac mini with 16 GB unified memory running macOS 14.1 or later. Apple Silicon is uniquely suited to this problem class: the GPU shares a single physical memory pool with the CPU, enabling Metal offload without the VRAM/RAM split that complicates discrete-GPU systems.
- A corrected, macOS-validated revision of a community offline-LLM guide, cataloguing seven concrete defects (Corrections table, §Research Findings).
- A mechanistic theory of the llamafile runtime (§3): fat-binary structure, mmap weight , and one-time runtime compilation of GPU backends.
- A memory and performance model for 16 GB unified memory (§3.4–3.5), including the KV-cache arithmetic that motivates context-size selection.
- A falsifiable experimental methodology (§4): five hypotheses, an instrumentation strategy via the
/metricsendpoint, and a decision policy for model selection.
This work targets macOS ARM64 (Metal) only; Windows CUDA behavior is referenced solely to bound scope. It does not train or fine-tune models. It evaluates two model tiers (Qwen3-4B and Qwen3-8B, Q4_K_M) and does not claim optimality over the broader GGUF catalogue. Security analysis is limited to threat identification, not formal verification.
The llama.cpp substrate. Most open-weight local inference traces to llama.cpp (Georgi Gerganov et al.), a C/C++ inference engine that popularized CPU-first execution and the GGUF format. Its server mode exposes an OpenAI-compatible HTTP API plus a bundled web chat UI and a JSON metrics endpoint — the instrumentation used throughout this plan.
Portable prior art. Community projects such as Portable-Offline-LLM (geevarghesekthomas84-sys) demonstrate the concept — a USB drive containing llama.cpp binaries, a models/ directory, and launch scripts. This prior art is predominantly Windows-oriented (.bat launchers, .exe binaries) and — critically — omits GPU verification methodology; on a misconfigured host it runs silently on CPU. The present work addresses both omissions.
The llamafile distribution model. llamafile (Mozilla AI) packages llama.cpp as a Cosmopolitan "actually portable executable" (APE). A single release artifact runs on six OS families; optionally, model weights may be appended to the same file, producing a self-contained "llamafile" that is executed directly. Our plan uses the non-fused form (runtime + separate GGUF), keeping the model swappable on a small drive.
Model family selection. Qwen3 (Alibaba) provides strong reasoning-per-parameter efficiency, hybrid "thinking" variants, and 262K-token native context — properties that fit a memory-constrained portable target. Its quantized GGUFs are produced by unsloth, the most widely used quantization house on Hugging Face; the 8B Q4_K_M is among the most-downloaded GGUF artifacts platform-wide, which we treat as weak but non-trivial evidence of practical utility.
Related commercial alternatives (Ollama offline bundles, LM Studio portable mode) require installation and are excluded by the zero-install requirement.
Cosmopolitan Libc compiles a single ELF/MZ hybrid — a polyglot file that is simultaneously a valid shell script, a Windows PE, and a Unix ELF. The llamafile release binary exploits this in three nested layers:
- Script wrapper. An MZ-prefixed shell script; Windows parses the PE header, Unix executes the script body. One asset name serves all platforms.
- Appended, page-aligned ZIP. The true
llama-serverexecutable, the GPU modulesources , and (in fused llamafiles) model weights live in a ZIP appended to the script. The ZIP is aligned to the page size by a purpose-built ~500-linezipaligntool: Metal requires page-aligned memory for buffer handoff to the GPU. - APE (8 KB). On Unix-likes, a small stub maps the binary portions into memory before transferring control.
At startup the runtime mmap() s its own file, mapping weights directly into the process address space — no extraction, no copy, no installation. Corollary for removable storage: the operating system serves the GGUF from the drive's page cache on first access; subsequent reads hit RAM. Drive throughput therefore bounds cold-start load time but not generation throughput (§3.6).
GPU backends cannot be statically linked into a Cosmopolitan binary: vendor runtimes are platform-specific and enormous, and the polyglot build targets six OSes. llamafile instead ships GPU code as source and compiles it on first run:
- On macOS ARM64, the Metal backend (
metal.c) extractsggml-metal.mandggml-metal.metalfrom its embedded ZIP and invokes the system C compiler (supplied by Xcode Command Line Tools) to produce a self-containedggml-metal.dylib, targeting thenative GPU microarchitecture of the host. The dylib is loaded viacosmo_dlopen()and registered with GGML as a first-class backend. - The compiled module is cached at
$TMPDIR/.llamafileor$HOME/.llamafile— compiled once, reused thereafter. - Failure mode: if the compiler is absent, the build fails and llamafilesilently falls back to CPU — an order-of-magnitude performance loss indistinguishable from success in logs. This is the dominant failure mode on a fresh Mac, and the central design driver of Phase 1 (mandatory
xcode-select --install) and of the launcher's--gpu appleflag, which converts the silent fallback into a fatal startup error (fatal error: support for --gpu apple ... wasn't available). - ABI correctness across this host/dylib boundary is not incidental: on ARM64 the module is compiled with
-ffixed-x28(reserving the TLS register so host and dylib share ABI state), and exported functions use__ms_abi__so a single module contract spans Windows and Unix calling conventions.
Release-artifact corollary. Only the full release binary (llamafile-0.10.5, 334.5 MB) embeds the compiler toolchain; the -thin variant (40.4 MB) does not and cannot bring up Metal without a prebuilt dylib. Asset selection is therefore a hard functional requirement, not a size trade-off.
GGUF is a single-file container holding weights and the metadata required to serve them — architecture, tokenizer, and the Jinja chat template — which is why no template configuration is required at serve time.
Quantization stores weights at reduced precision. GGUF quants are block-based: weights are grouped (typically 128 per block) with a shared per-block scale, and the "K" family (K-quants) additionally partitions each block by importance using a k-means-style split, retaining a small subset of sensitive weights at higher precision. Effective size relative to FP16:
| Quant | Size vs FP16 | Regime |
|---|---|---|
| Q8_0 | ~50 % | near-lossless |
| Q6_K | ~62 % | high quality |
| Q5_K_M | ~69 % | strong quality |
| Q4_K_M | ~75 % | quality/memory optimum |
| Q3_K_M | ~81 % | degraded |
| Q2_K | ~87 % | avoid |
For the 8B tier this yields 5.03 GB at Q4_K_M versus ~10 GB at FP16 — the difference between "fits a 16 GB machine" and "does not". Q4_K_M is the established Pareto optimum of perplexity per byte; the plan adopts it for both model tiers (8B: 5.03 GB; 4B-Thinking: 2.6 GB, the smallest Qwen3 variant judged to retain acceptable reasoning).
Steady-state resident memory is approximately:
R ≈ W + K + O
where W is the quantized weights, K the KV cache, and O the runtime overhead (~1–2 GB). The KV cache is the only term that scales with the context window:
K(tokens) = 2 · n_layers · n_kv_heads · head_dim · 2 bytes (f16) · tokens
Qwen3 uses grouped-query attention with only 8 KV heads; the cache therefore scales with KV heads, not query heads — a factor-8 reduction versus MHA. For Qwen3-8B (36 layers, head_dim 128):
K ≈ 2 · 36 · 8 · 128 · 2 B · tokens ≈ 144 KB / token
→ 1.18 GB @ 8,192 ctx → 2.36 GB @ 16,384 ctx
Projected steady-state budgets on the 16 GB target:
| Model | W (RAM) | K @ 8,192 | Peak R |
|---|---|---|---|
| 4B-Thinking Q4_K_M | ~2.8 GB | ~0.6 GB | ~5 GB |
| 8B Q4_K_M | ~5.3 GB | ~1.2 GB | ~8 GB |
| 8B Q4_K_M @ 16,384 ctx | ~5.3 GB | ~2.4 GB | ~9.5 GB |
All tiers retain ≥ 6 GB headroom — no swap, no OOM. The original guide's --ctx-size 2048 conservatism is unnecessary on this hardware and wastes reasoning headroom; the plan adopts 8,192 default, 16,384 maximum, beyond which the quality/memory trade-off flattens for 4B–8B models.
Metal offload on Apple Silicon exploits a structural advantage: weights live in the same physical memory the GPU reads, at full memory bandwidth, with no PCIe transfer and no VRAM copy. The page-aligned mmap buffers of §3.1 are handed to Metal as-is. This is why a 5 GB model is GPU-accelerated at all on a 16 GB machine, and why Apple Silicon parts typically exceed equivalently priced discrete-GPU laptops in tokens/second for these model sizes. Empirical targets (hypotheses to be measured, not claims): ~40–60 t/s for the 4B tier, ~25–40 t/s for the 8B tier, Q4_K_M, on M4.
Loud-failure principle. CPU fallback (§3.2) is indistinguishable in the happy-path log; single-digit tokens/second is the symptom. Any benchmark figure in that range invalidates the GPU hypothesis and reruns Phase 1 verification.
- exFAT is the correct filesystem: no 4 GB file ceiling (FAT32's limit), negligible per-file overhead, no journaling (reduced flash wear), and native read/write on macOS, Windows, and Linux. APFS on external media is macOS-only and journaled to no portable benefit.
- Cold start is page-cache fill. The GGUF is read from the drive once and served from RAM thereafter. Throughput expectations: ~5 GB at USB 3.2 Gen 2 (~300–1000 MB/s) ≈ 5–17 s; at USB 2.0 (~40 MB/s) ≈ 2 min. The plan therefore treats drive speed as a cold-start parameter, not a throughput parameter.
- Gatekeeper. Any file delivered via HTTP carries the
com.apple.quarantineextended attribute; a quarantined executable refuses to launch. Attribute removal (xattr -d) or right-click → Open is mandatory setup, applied in Phase 1.
The appliance is by construction a self-extracting, runtime-compiling executable — startup executes embedded code by design. The corresponding threat model:
- Untrusted-host risk. Running the stick on a machine one does not trust transfers trust to whatever the stick carries. Mitigation: treat the stick as a trusted appliance on trusted machines only.
- Unauthenticated server. The bundled server binds to
127.0.0.1by default. Binding0.0.0.0would expose an unauthenticated LLM API to the LAN — prohibited in this plan. - Untrusted model input. The GGUF parser is part of the attack surface. Mitigation: source models from well-known quantizers (unsloth) and the runtime from the maintainer's GitHub releases; pin checksums at packaging time.
- At-rest exposure. Models are unencrypted on the drive; appropriate for public-weight models, incompatible with private weights.
The plan is structured as an experiment: five falsifiable hypotheses, each mapped to a test and a falsification criterion.
| # | Hypothesis | Test | Falsified by |
|---|---|---|---|
| H1 | Metal builds and loads on the target Mac | Phase 1, --gpu apple |
CPU-only logs; fatal error |
| H2 | 8B Q4_K_M fits 16 GB with ≥ 6 GB headroom | Phase 2 metrics at 8,192 / 16,384 ctx | Peak RAM ≥ 10 GB |
| H3 | GPU tokens/second in the §3.5 range | Phase 2 /metrics |
Single-digit t/s (silent CPU fallback) |
| H4 | Cold-start from the stick is acceptable | Phase 3 timing | Load > 60 s on USB 3.2 |
| H5 | Double-click flow needs zero terminal interaction | Phase 4 acceptance | Gatekeeper/CLT failure on cold boot |
Instrumentation. llamafile's server exposes /metrics (llama.cpp JSON) — machine-readable tokens/second, prompt processing, and memory figures. Controlled variables per run: model, quantization, context size, prompt, sampling temperature. Three context levels (2,048 / 8,192 / 16,384) × two model tiers, recorded into a benchmark matrix; the decision policy (Table 5, §6) selects the shipped configuration from measured data, keeping model selection empirical rather than a priori.
Build a fully offline, portable AI assistant that runs from a USB pendrive on an Apple M4 Mac mini (16 GB RAM) — no internet, no accounts, no installation. The stick carries a single self-contained binary (llamafile), a GGUF model, and a double-clickable launcher.
This document is the research + execution plan. Nothing here has been implemented yet; Phase 4 (packaging) is the implementation phase and is explicitly deferred.
| Property | Value |
|---|---|
| Machine | Apple M4 Mac mini |
| RAM | 16 GB unified memory |
| OS | macOS 14.1+ (Sonoma) required; Apple Silicon (ARM64) |
| GPU | Built-in Apple Silicon GPU via Metal (offload on by default) |
| Storage | USB drive, exFAT formatted |
| Runtime | llamafile (single-file binary, no install) |
| Model | Qwen3 GGUF (4B or 8B tier, see decision matrix) |
OfflineAI/ ← root of the USB stick
│
├── llamafile-0.10.5 ← full macOS binary (334.5 MB, NOT -thin)
├── models/
│ └── Qwen3-8B-Q4_K_M.gguf ← 5.03 GB (or 4B tier, 2.6 GB)
│ └── Qwen3-4B-Thinking-2507-Q4_K_M.gguf ← optional portable tier
│
└── Start.command ← double-click to launch (chmod +x)
When Start.command runs:
cds to its own directory (works from any mounted drive path)- Launches
llamafile --serverwith the model + context size - Opens
http://localhost:8080— the built-in llama.cpp web UI - First run compiles a Metal GPU module (one-time, via Xcode CLT)
Everything below was verified via web research before writing this plan.
- v0.10.5 is the current release (published 2026-08-03), tracking llama.cpp b10103.
- Release assets for macOS (from the 0.10.5 release page):
| Asset | Size | Use it? |
|---|---|---|
llamafile-0.10.5 |
334.5 MB | ✅ Yes — full binary, embeds the GPU compiler toolchain needed for Metal |
llamafile-0.10.5-thin |
40.4 MB | ❌ No — lacks the embedded compiler; GPU setup will fail/skip |
llamafile-0.10.5.zip |
255.7 MB | Optional — zip wrapper of the same binary |
llamafile-0.10.5.exe |
— | ❌ Windows only |
-
No
-x86_64asset exists in 0.10.5 — the singlellamafile-0.10.5binary is the macOS one. (Verify universal-arch behavior in Phase 1.) -
Metal is supported only on macOS ARM64 (Apple Silicon). Offload isenabled by default — no
-nglflag needed (that flag is for NVIDIA/AMD). -
First run compiles
ggml-metal.dylibfrom embedded sources using theXcode Command Line Tools ; cached in$TMPDIR/.llamafileor$HOME/.llamafile. -
llamafile silently falls back to CPU if the Metal module fails to build/load. Mitigation: run with
--gpu apple, which turns failure into a loud startup error. -
The known Windows CUDA breakage (issue #961:
no pre-built GPU library foundon 0.10.x) isWindows-only — not applicable on macOS.
| Model | Quant | File size | RAM @ load (~2K ctx) | Notes |
|---|---|---|---|---|
| Qwen3-4B-Thinking-2507 | Q4_K_M | 2.6 GB | ~3 GB | Portable tier — runs on any machine, any stick |
| Qwen3-8B | Q4_K_M | 5.03 GB | ~5.5–6 GB | Recommended for M4/16 GB — much better quality, ~10 GB headroom |
| Qwen3-8B-Thinking-2507 | Q4_K_M | ~5.2 GB | ~6 GB | Reasoning variant of 8B; verify exact size in Phase 2 |
| 7B class (Llama 3.1 etc.) | Q4_K_M | ~4.4 GB | ~5–5.5 GB | Alternative if Qwen3 unavailable |
Quantization scale (vs FP16): Q8_0 ~50 %, Q6_K ~62 %, Q5_K_M ~69 %, Q4_K_M ~75 % (the sweet spot), Q3_K_M ~81 %, Q2_K ~87 % (avoid).
Source repos: unsloth/Qwen3-4B-Thinking-2507-GGUF, unsloth/Qwen3-8B-GGUF on Hugging Face (the 8B non-thinking model is the most-downloaded GGUF on the platform; the doc's original 4B-Thinking pick is confirmed correct for the portable tier).
The user's original doc was Windows-oriented and had several issues for this project:
| # | Original doc / earlier plan | Problem | Correction |
|---|---|---|---|
| 1 | llamafile-0.10.5.exe |
Windows binary | Use the plain llamafile-0.10.5 macOS binary |
| 2 | Start.bat with.\ prefix |
Windows syntax | Start.command shell script with$(dirname "$0") |
| 3 | --ctx-size 2048 default |
Too conservative for 16 GB | Default 8192 , max16384 (Qwen3 GQA keeps KV cache small) |
| 4 | (unstated) first-run GPU compile | Fresh Macs lack Xcode CLT → silent CPU fallback | Install CLT ( xcode-select --install ) and verify Metal with--gpu apple |
| 5 | (unstated) Gatekeeper | Downloaded binary is quarantined → won't launch | xattr -d com.apple.quarantine once |
| 6 | Metal dylib path ( ~/Library/... ) |
Wrong path in an earlier plan draft | Actual: $TMPDIR/.llamafile or$HOME/.llamafile |
| 7 | Single 4B model | Undersized for the M4/16 GB target | Add 8B Q4_K_M tier (5.03 GB) as primary recommendation |
Status: PLANNED — nothing implemented. Phase 4 is the implementation phase and will not be started until this plan is approved.
Goal: prove the machine can run llamafile with Metal before down 5 GB of models.
- Confirm OS + hardware:
sw_vers # needs Darwin 23.1.0+ (macOS 14.1+)
uname -m # needs arm64
system_profiler SPHardwareDataType | grep -E "Chip|Memory"
- Install Xcode Command Line Tools if missing (required for the one-time Metal module compile):
xcode-select -p # prints path if already installed
xcode-select --install # only if missing
- Download the binary (full asset, not
-thin):
curl -L -o llamafile-0.10.5 \
https://github.com/mozilla-ai/llamafile/releases/download/0.10.5/llamafile-0.10.5
chmod +x llamafile-0.10.5
- Strip Gatekeeper quarantine:
xattr -d com.apple.quarantine llamafile-0.10.5
- Verify Metal builds and loads (loud failure mode): Expect a "building/ Metal module" log line. If it silently reports CPU-only, stop and debug before Phase 2.
./llamafile-0.10.5 --gpu apple --version
Exit criteria: --gpu apple succeeds; startup logs show the Metal module build/load.
Goal: choose the model tier with measured numbers, not vibes.
- Download candidates:
curl -L -o models/Qwen3-8B-Q4_K_M.gguf \
https://huggingface.co/unsloth/Qwen3-8B-GGUF/resolve/main/Qwen3-8B-Q4_K_M.gguf
curl -L -o models/Qwen3-4B-Thinking-2507-Q4_K_M.gguf \
https://huggingface.co/unsloth/Qwen3-4B-Thinking-2507-GGUF/resolve/main/Qwen3-4B-Thinking-2507-Q4_K_M.gguf
- Benchmark each model × each context size (2048 / 8192 / 16384):
./llamafile-0.10.5 --server --model models/<model>.gguf \
--ctx-size <N> --port 8080 --gpu apple --metrics
- Record results into a table (template below):
| Model | ctx | Load time | t/s | Peak RAM | Quality note |
|---|---|---|---|---|---|
| 4B-Thinking Q4_K_M | 2048 | — | — | — | — |
| 4B-Thinking Q4_K_M | 8192 | — | — | — | — |
| 8B Q4_K_M | 2048 | — | — | — | — |
| 8B Q4_K_M | 8192 | — | — | — | — |
| 8B Q4_K_M | 16384 | — | — | — | — |
Exit criteria: filled benchmark table; model + ctx decision made (expect: 8B Q4_K_M @ 8192 default, 4B as portable fallback).
Goal: confirm the stick is fast enough that load time isn't painful.
- Format the drive exFAT (Disk Utility) — required for >4 GB files, and keeps the stick readable on Windows/Linux too. (GUID partition map.)
- Copy the full folder layout from above.
- Measure end-to-end: insert drive → double-click
Start.command→ server reachable. - Record the time-to-first-token for both models. Compare USB 3.2 Gen 2 vs Thunderbolt if both are available.
- Note: model file loads into RAM at startup (mmap'd), so stick speed mostly affects cold-start, not generation speed.
Exit criteria: full cold-start works from the stick; load time recorded and acceptable (< ~30 s).
Goal: turn the verified setup into a polished, reproducible launcher.
Start.command— double-clickable launcher:
#!/bin/sh
cd "$(dirname "$0")"
MODEL="models/Qwen3-8B-Q4_K_M.gguf"
CTX=8192
./llamafile-0.10.5 --server --model "$MODEL" --ctx-size $CTX --port 8080 --gpu apple
- Editable
MODEL/CTXvariables at top of file chmod +x Start.command
- Editable
- First-run helper (small shell script) that:
- Checks
xcode-select -p, prompts to install if missing - Strips
com.apple.quarantinefrom the binary if present - Runs
--gpu appleverification and prints the measured t/s from/metrics
- Checks
- Multi-model picker (only if both tiers are kept): menu prompt to choose 4B (portable) or 8B (quality).
- Optional repo integration : add
scripts/portable-ai/to the supercli repo — a generator that scaffolds the folder layout + launcher so the stick is reproducible from source. (Awaiting user decision.) - Write a short
README.txtfor the stick (cross-platform: who opens a.commandon Windows won't see instructions).
Exit criteria: clean stick → format → copy → double-click → chat, with zero terminal interaction required on the happy path.
- Security notes (document in the stick README + this plan):
- llamafile is a self-extracting executable that compiles code at runtime — only run it on machines you trust.
- The server binds to
127.0.0.1by default — donot add--host 0.0.0.0(would expose the model to the LAN). - Don't run the stick's binaries on shared/unknown machines.
- Models are untrusted third-party files; the Qwen3 GGUFs come from the unsloth repo (well-known quantizer).
- Cold-boot test : eject, re-insert, run again — confirm the Metal dylib cache makes the second boot faster.
- Portability test (optional): run the stick on a second Apple Silicon Mac to validate the "any machine" claim for the 4B tier.
- Verify macOS Gatekeeper doesn't block
Start.commandon a fresh machine (right-click → Open flow documented as fallback).
Exit criteria: two consecutive cold boots pass; security notes written; portability claim tested or explicitly out of scope.
- Model tier on the stick : single 8B (recommended for this machine) vs both 4B+8B with a picker vs single 4B (max portability)?
- Repo integration : add a
scripts/portable-ai/generator to supercli, or keep this as a standalone guide + stick? - Drive : USB 3.2 Gen 2 / Thunderbolt available, or plain USB-A? (Affects cold-start expectations only.)
| If benchmark shows… | Then… |
|---|---|
| 8B @ 8192 ctx < 10 GB RAM and ≥ 15 t/s | Ship 8B-only stick |
| 8B too slow / too hot | Ship 4B-Thinking-only stick |
| Both fit comfortably | Ship both + picker (repo integration option 2 becomes attractive) |
- llamafile releases: https://github.com/mozilla-ai/llamafile/releases (v0.10.5, 2026-08-03)
- GPU support docs: https://docs.mozilla.ai/llamafile/reference/support (Metal default on macOS ARM64,
--gpuflags, dylib paths) - Metal runtime details:
llamafile/metal.c(one-time dylib compile via Xcode CLT) - Models: https://huggingface.co/unsloth/Qwen3-4B-Thinking-2507-GGUF ,https://huggingface.co/unsloth/Qwen3-8B-GGUF
- Reference portable-stick project:
geevarghesekthomas84-sys/Portable-Offline-LLM(USB 3.0+, exFAT,models/layout) - Non-issue on macOS: Windows CUDA breakage
mozilla-ai/llamafileissue #961
Status: PLAN ONLY. Phase 4 (implementation) will not begin until this plan is approved and Phases 1–3 produce the benchmark/decision data above.