Originally published at olund.dev.
Part of my tooling generates long-form written content in a configured
voice: an editorial identity that describes the audience, the register, the
vocabulary, what the writer sounds like. The identity is prose in a config.
And prose configs for tone have a nasty property: they are write-only. You
describe the voice you want ("unhurried, curious, explains from first
principles, never hypey"), the generator consumes it, and you learn whether
your description worked only after paying for a full generation run and
reading the result. If the register is off, you edit adjectives and pay
again.
The feedback loop is the problem. Tuning a persona through full generations
is like adjusting a recipe by catering a wedding each time. What I wanted
was a taste: given this identity, say two lines in this voice, right now,
for approximately nothing. The fix is a preview agent, and the design goal was to make it as close to
free as an LLM call gets - because a preview you hesitate to run is a
preview that does not get run. Everything about it is subtraction:
The prompt side is one rule repeated three ways: match the identity's tone,
register, and vocabulary precisely; each line must be a speakable line of prose
on its own; do not perform meta-commentary. Sample lines that describe
the voice instead of being the voice are the failure mode, and the
instructions attack it directly.
In the settings UI, next to the identity editor, there is a sample button.
Type an optional topic, click, and a moment later: a handful of lines in the configured voice. Edit the identity, sample again. The tuning loop
drops from "generate a full piece, read it, wince" to seconds per
iteration.
Two UI decisions carry more weight than they look like they should:
The preview invalidates nothing. It is a plain fire-and-return call
with no cache updates, because it changes no state. Wiring a preview into
the app's data layer as if it were a mutation is a category error that
makes every preview cost a refetch.
The empty state teaches. With no identity configured, the button does
not disable silently; the endpoint refuses with "identity is unset" and the
UI says so. A preview feature whose precondition is invisible reads as
broken.
There is also a seam decision underneath: the route awaits the worker
synchronously, bounded at two minutes, instead of returning a job id the
client polls. Previews are interactive - the human is sitting there. The
moment a preview needs a progress bar, it has failed at being a preview,
so the API shape encodes the latency budget: if this cannot answer while
the user watches, it should error, not stream status updates.
The general shape: when a system consumes a human-authored description and produces something expensive, insert the cheapest possible sampler between the two. The description-to-output gap is where confidence
Samplers earn their place when they are:
I now reach for this shape whenever a prose config drives generation:
sample the persona before the piece, the summary style before the batch,
the reviewer's severity before the review run. Each sampler is an
afternoon of work, because subtraction is fast to build - the entire agent
definition fits on one screen, and the worker runtime it rides on already
existed.
The quiet lesson underneath is about model routing. The instinct is to
send every task to the strongest model available. But a preview's job is
to be representative and immediate, not maximal - and a small local
model with a tight schema and inline context is more representative of
"what will the configured voice sound like" than a frontier model
improvising with more freedom. Match the worker to the narrowness of the
task, and some tasks turn out to be nearly free.