cd /news/large-language-models/companion-lab-an-open-local-lab-an-u… · home › topics › large-language-models › article
[ARTICLE · art-172536] src=discuss.huggingface.co ↗ pub= topic=large-language-models verified=true sentiment=· neutral

Companion Lab: an open local lab, an unscripted companion, and a whole novella

A hands-on test of the open-source Companion Lab found that moving a prompt instruction between the start and later in a conversation changes more than placement: early instructions get joined to the leading system message, while late ones become a separate system message after earlier turns, altering message boundaries and potentially the model's rendering. The tester ran Qwen3-4B-Instruct-2507 Q4_K_M GGUF on a Colab T4 via a prebuilt JamePeng CUDA wheel and the Lab's OpenAI-compatible adapter, and reported that one follow-up probe produced only 144 output tokens and hit finish_reason=length, which the Lab correctly marked incomplete. The tester cautioned that the samples are far too small to rank placements or infer anything about Helena's private memory or long-term behavior, and recommended checking assembled messages, role handling by the selected model/backend, and whether generation completed before running a larger evaluation.

recommendation

Before ranking prompt placements, inspect the assembled message roles and boundaries and confirm finish_reason is not length; a 144-token cap produced a truncated answer that the Lab flagged as incomplete.

more in playbook: Evals and testing for LLM apps
read3 min views2 publishedNov 19, 2026

For now, I tried running it: Of the three things you suggested comparing, I would start with prompt placement. I tried the public Companion Lab with a real GGUF model served through an OpenAI-compatible endpoint. That gave me a small experiment I could actually reproduce with the included turns context plugin, two saved prompt versions, and no knowledge of the private context system behind Helena.

My first useful observation was about what the Lab sent, rather than which placement produced the better answer. Moving an instruction to the beginning also joined it to the leading system message; moving it late made it a separate system message after earlier turns. Both are legitimate arrangements, but the message boundaries and potentially the model’s rendering change too. The saved requests make this visible.

For “what would make the lab more useful,” my first small request would be a portable A/B comparison packet: one easy download containing the two saved requests, two replies, prompt-version IDs and available model/adapter/completion metadata. This comes from having to assemble those pieces myself. The Lab already records the comparison requests and replies under data/compares/, and it has a separate Save full transcript function for a normal conversation. I am not suggesting it lacks logging. I would simply find it useful to take one complete comparison away, inspect it later or share a synthetic reproduction without joining the files by hand.

My default first experiment would be:

turns plugin, and two saved prompt versions differing in the placement of one instruction. Keep the actual instruction text, final question, short conversation history and generation settings fixed as far as possible.request-a.json and request-b.json: message count, order, roles, grouping and content. Also note the adapter’s additions and what the backend can actually accept.finish_reason, output limits and errors when available. A correct but cut-off explanation is not a completed answer. What I saw in brief: I used Qwen3-4B-Instruct-2507 Q4_K_M GGUF on a Colab T4 through a prebuilt JamePeng CUDA wheel and the Lab’s OpenAI-compatible adapter. Both placement arrangements reached the model, and the native logs confirmed GPU inference; one attempted answer later ran into my own output limit. Three short, invented continuity situations were judged as expected by the completed responses. I also tested a third manually rearranged message sequence directly through the native worker. That third condition was not a hidden third lane in the public Side by side UI. The samples are far too small to rank placements or infer anything about Helena’s private memory, novella criticism or long-term behavior.

One useful negative result was mine, not the Lab’s: my follow-up probe gave a response only 144 output tokens, so one explanation reached finish_reason=length. The Lab correctly marked it incomplete. A targeted native retry with a larger budget completed it. That reminded me to inspect completion status before attributing a short answer to prompt wording or placement.

I would therefore put three inexpensive checks ahead of a larger evaluation: the assembled messages, how the selected model/backend handles their roles, and whether generation completed. The Hugging Face chat-template guide is useful context: a list of role/content objects is not necessarily the exact token sequence a model receives. And the Companion Lab README already makes the public/private boundary explicit; the published 50-turn interaction is an unscripted example, not a controlled comparison. Keeping a missed detail in that record, rather than only the polished moments, is useful context for readers.

Below are the details and optional branches I would follow depending on whether the real question is placement, a memory selector, a different model, or useful criticism of a story. None of them requires turning the Lab into a full benchmark platform.

  1. Prompt placement: what actually moves So my two practical answers remain prompt placement first, using the public turns plugin and the saved request diff, and a portable complete A/B packet as the improvement I’d most appreciate. The GGUF run gave me an actual reason for both choices. Memory selection, cross-model behavior and story criticism are interesting follow-ups if those become the specific questions someone wants to explore, not extra requirements for this public release. Thanks for sharing a workbench where those questions can be inspected rather than only guessed at.
── more in #large-language-models 4 stories · sorted by recency
── more on @companion lab 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/companion-lab-an-ope…] indexed:0 read:3min 2026-11-19 · —