{"slug": "fluency-is-not-authority-building-local-ai-characters-for-signal-drift", "title": "Fluency Is Not Authority: Building Local AI Characters for Signal Drift", "summary": "Signal Drift, a CRT-noir hacking thriller, ships with locally run AI characters powered by Qwen2.5-1.5B-Instruct, a 1.54-billion-parameter model fine-tuned with a custom LoRA and quantized to Q4_K_M, to let players type free-text dialogue while the authored game controls plot reality. The game's developer trained the model on 3,057 supervised fine-tuning rows and 1,953 preference pairs to keep characters terse and in-character, avoiding invented plot outcomes, and runs inference locally via llama.cpp on Windows, macOS, and Linux.", "body_md": "One of the easiest ways to break a narrative game is to let a convincing sentence become true.\n\nThe scene I kept returning to while building **Signal Drift** was simple. The player breaks into a fictional host, finds a log that contradicts the client's briefing, and confronts that client in IRC. A conventional dialogue tree can handle the accusations I anticipated, but not every interpretation a player might type. A language model can respond to almost anything—but an unconstrained model may invent a confession, a new conspirator, or an outcome that never occurred.\n\nI did not need a model to write the plot. I needed it to let a character inhabit a plot the game already knew was true.\n\nCORE RULEThe model performs the character. The authored game decides reality.\n\n## Why free-text dialogue belonged in a terminal game\n\nSignal Drift is a CRT-noir hacking thriller built around fictional terminals, logs, mail, IRC rooms, evidence, and readable Linux-style commands. Players scan hosts, recover credentials, move through SSH sessions, reconstruct events, and decide whether to expose, sell, alter, or erase what they find.\n\nTyping is already the game's primary verb. Asking a character “why does this timestamp disagree with the audit log?” feels natural in a way that selecting a dialogue wheel beside a terminal would not.\n\nThe goal was not infinite dialogue. It was to cover the human space between authored beats: challenging a lie, asking what a discovered file implies, provoking a rival, revisiting a betrayal, or talking to someone after the contract is over.\n\nInference runs locally, without an external AI service.\n\nThe same feature ships on Windows, macOS, and Linux.\n\nJavaScript remains authoritative for state and safety.\n\nCharacters receive designed memory, not endless history.\n\nLocal inference eliminated an API dependency and recurring inference bill, but it did not make the feature free. It moved the cost into download size, memory, startup time, cross-platform packaging, hardware variance, and support.\n\n## The model I actually shipped\n\nSignal Ghost uses [Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct), an Apache-2.0 model with roughly 1.54 billion parameters. I trained a Signal Ghost v19 dialogue LoRA, fused it into the base model, converted the result to GGUF, and quantized it to Q4_K_M.\n\nThe shipping file is 986,048,512 bytes—about 940 MiB. It runs through [llama.cpp](https://github.com/ggml-org/llama.cpp), pinned in the current build to b9835.\n\nThe smaller smoke-test model would announce that it was “just a computer program,” fall into generic security prose, or keep talking after the dramatic moment had passed. Compact was not the same as usable.\n\nThe 1.5B Q4_K_M build was the smallest version I tested that could remain terse, stay inside a character, and respect the game's boundaries on a CPU-safe path.\n\n### Training corpus\n\n**3,057** deduplicated supervised fine-tuning rows**1,953** chosen-versus-rejected preference pairs**40** authored multi-turn conversations / 120 prefixes**1,944** unique assistant outputs\n\nThe training sequence was an 800-iteration MLX-LM LoRA pass, a 160-iteration repair pass using failures observed in the quantized model, a 480-iteration reference-free CPO-style preference pass, and a final 80-iteration repair polish.\n\nThe corpus teaches voice and behavior, not plot state. Stable control tags identify character, conversational mode, rapport tier, intent, safety boundary, and output contract. The LoRA owns voice; runtime retrieval supplies facts.\n\n## Memory is a data structure, not a transcript\n\n“The characters remember you” can easily become an overclaim. Signal Ghost does not remember everything, and I do not send an entire conversation history back through the model.\n\nEach recurring character has an explicit persisted record: a rolling summary, recent full turns, extracted standing facts, total and compacted turn counts, and a last-interaction timestamp.\n\n**16** latest full turns\n\n**900** summary characters\n\n**18** standing facts\n\n**3 + 3** facts + turns retrieved\n\nFact extraction looks for things the game can use dramatically: artifacts discussed, trust or vulnerability pressure, explicitly named characters, “remember this” cues, the player's chosen name, relationship statements, and mood cues such as fear, anger, loneliness, or exhaustion.\n\nRetrieval is deliberately small and understandable. It is lexical and relevance-ranked rather than vector search. An ordinary reply receives the three most relevant facts and three most relevant turns. An explicit recall request can receive five facts and four turns, plus an older summary fragment. If the exact fact is absent, the prompt tells the character to admit the gap rather than fabricate memory.\n\nRelevance beat chronology in testing. A 1.5B local model performed better when given the three exchanges that answered the current question than when given eight arbitrary recent turns and a rolling wall of text.\n\nPublic IRC adds another wrinkle: a player can mention one character while someone else is speaking. When appropriate, Signal Ghost stores an “overheard via” memory for the mentioned character. That gives cross-room conversation some continuity without pretending every character has omniscient access to every private message.\n\nRemembered dialogue colors performance. It does not rewrite evidence.\n\n## Treating a local model like an engine subsystem\n\nThe shipping runtime prefers a persistent `llama-server`\n\nprocess. The game begins prewarming it shortly after the application loads, keeps the model resident, and sends completion requests over a random loopback-only `127.0.0.1`\n\nport. The web UI is disabled, and the server has one inference slot.\n\n```\npreferred server\n  → CPU-safe server\n  → one-shot llama-cli\n  → authored dialogue\n```\n\nThe startup timeout is 90 seconds and a generation request is capped at 45 seconds. A failed server enters a short backoff rather than being relaunched continuously. The game reaps active model processes during shutdown so Steam does not remain stuck on “Stop.”\n\nContext is intentionally small: 1,024 tokens on Linux CPU, 1,280 on other desktop builds, and 1,536 on the Linux and Steam Deck Vulkan path. Replies usually receive 56–72 generated tokens, with platform-dependent caps of 72–96. A character in IRC needs a sharp, state-aware reaction—not a research-paper context window.\n\nmacOS currently uses the CPU-safe Accelerate/BLAS route because forcing Metal crashed on the development host. Linux and Steam Deck prefer a bundled Vulkan runner and fall back to CPU. Windows currently uses the CPU-safe path.\n\n### One controlled benchmark\n\nPerformance varies enough that one developer machine should not be treated as every player's result. This is a lower-end datapoint: a 2018 Intel Core i7-8750H MacBook Pro with 16 GB RAM, eight CPU threads, a 1,280-token context, and no GPU layers.\n\n| Measurement | Result |\n|---|---|\n| Cold server readiness | ~59 seconds |\n| Resident memory after load | ~1.0 GiB |\n| 93-token prompt / 17 output tokens | 4.74 seconds |\n| Prompt evaluation | 28.0 tok/s |\n| Generation | 14.0 tok/s |\n| Repeated identical prefix | 0.95 / 0.91 sec |\n\nThe repeated-prefix figures benefit from llama.cpp's prompt cache and should not be read as typical gameplay latency. Real prompts are longer and change with mission state, memory, and player input.\n\nOne benchmark output said, “I am here to help if that is what you are looking for.” It was safe and structurally valid, but bland. Rule-based validation can reject a leaked prompt or invented fact more reliably than it can recognize every boring sentence.\n\n## What broke while building it\n\n### The wrong prompt format looked like bad behavior\n\nQwen2.5-Instruct expects native ChatML role markers. A generic format caused refusals. ChatML plus the correct stop token fixed a problem no amount of character prose would have solved.\n\n### Small models copy scaffolding\n\nThe bridge strips runner banners, echoed ChatML, control tokens, speaker prefixes, code fences, and timing text. It rejects character-bible parroting, hidden metrics, handle leaks, transcript echoes, and real-world intrusion detail.\n\n### Characters collapse into one helpful assistant\n\nThe data includes rejected cross-character examples. Sampling differs by character and mode: evidence questions are tighter; personal downtime can vary more.\n\n### Relationship dialogue drifted back into work\n\nDowntime, on-job, public-channel, mission, and direct-message registers are separate. Validators flag work language where an emotional answer belongs.\n\n### Packaging became narrative quality\n\nA character is not convincing if the response never arrives. Process lifetime, platform libraries, context budgets, and fallbacks all shape whether a conversation feels occupied or broken.\n\n## Testing behavior without snapshotting sentences\n\nExact-string snapshots are a poor fit for sampled dialogue. I test contracts instead: no speaker tags or Markdown, no prompt leakage, no hidden score or payout, no real-world abuse detail, no cross-character contamination, no mission language in personal downtime, and no private context leaking into the wrong character.\n\n- A betrayal or memory cue can influence a later prompt.\n- A prompt-extraction attempt receives an authored, in-world boundary.\n- A missing or failed runner cannot corrupt a save or block a mission.\n- A grounded reaction cannot invent unsupported entities, numbers, or outcomes.\n\n**2,073 / 2,073** static dialogue-contract fixtures passing\n\nThat total covers 1,657 golden rows, 145 cast-coverage rows, 95 memory rows, 120 multi-turn prefixes, and 56 repair cases. These validate authored examples and contract fixtures; they are not a claim that every stochastic output is perfect.\n\nA separate live-model probe runs the quantized GGUF with multiple seeds. The distinction matters. A model can pass a curated set and still become generic on the third sample, as the benchmark demonstrated.\n\n## What shipping changed\n\nSignal Drift launched on June 29. On July 3, I shipped the first full Signal Ghost release: the persistent runner, more open conversation, stronger voices, relationship context, public IRC cross-talk, and mission-aware fallout. On July 8, I followed with a wider onboarding and narrative update—guided starter intrusion, more voice and music, leaked recordings, personal mail, stronger continuity, and a finale that spends more of the player's trail.\n\nThe second update is an important reminder: generated dialogue cannot replace authored structure. A character can answer beautifully, but if the player does not understand the terminal, cannot locate the evidence, or cannot see how one contract connects to the next, the system has nothing meaningful to react to.\n\nThe model fills negative space. It is not the structure holding the building up.\n\n## Five lessons I would carry into another game\n\n- 01\n**Give the model a narrow dramatic job.**“Perform this character's reaction” is safer and more useful than “run the story.”\n\n- 02\n**Keep canonical state outside the model.** Fluency is not authority.\n\n- 03\n**Design memory explicitly.** Retention, retrieval, forgetting, and honest gaps are game-design decisions.\n\n- 04\n**Treat latency and failure as interaction design.** A warm runner and an invisible fallback matter as much as the prompt.\n\n- 05\n**Test invariants, not favorite outputs.** Invented truth, leaked context, wrong voice, and blocked play are the dangerous failures.\n\nThe model makes sentences. The game decides which sentences are possible, which are remembered, and which are allowed to matter.\n\nSignal Ghost uses a bundled local language model for fictional character dialogue. Inference and chat generation occur on the player's device and do not require an external AI service. Steam features such as cloud saves are separate. Some pre-generated artwork was created with generative AI and edited for use in the game.", "url": "https://wpnews.pro/news/fluency-is-not-authority-building-local-ai-characters-for-signal-drift", "canonical_source": "https://engineering.signaldrift.net", "published_at": "2026-07-31 04:54:39+00:00", "updated_at": "2026-07-31 05:22:27.116410+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-products", "ai-tools"], "entities": ["Signal Drift", "Qwen2.5-1.5B-Instruct", "llama.cpp", "Signal Ghost"], "alternates": {"html": "https://wpnews.pro/news/fluency-is-not-authority-building-local-ai-characters-for-signal-drift", "markdown": "https://wpnews.pro/news/fluency-is-not-authority-building-local-ai-characters-for-signal-drift.md", "text": "https://wpnews.pro/news/fluency-is-not-authority-building-local-ai-characters-for-signal-drift.txt", "jsonld": "https://wpnews.pro/news/fluency-is-not-authority-building-local-ai-characters-for-signal-drift.jsonld"}}