# Give your LLM more DSLs > Source: > Published: 2026-10-05 20:24:58+00:00 # Give your LLM more DSLs! + Ask once, compile the answer, run many times It’s been thoroughly shown at this point that having LLMs author prompts for themselves (free-form that is, not DSPy-style) is disastrous, and that [LLM-authored skills actually make things worse](https://arxiv.org/html/2602.12670v4), but [writing code is what these next-token-predictors excel at](https://arxiv.org/abs/2402.01030)! There’s a frequently-used trick for getting LLMs to do out-of-band tasks, which is: come up with a thoughtful DSL, capture your current state as a DSL, get the LLM to manipulate the DSL, then translate that back to your domain. It turns out this works pretty well for *in*-domain tasks too. A great example of this is the continuous improvement loop in [`captain-hook`](https://yasyf.com/writing/less-prompts-more-guardrails/): getting the LLM to scan transcripts for corrections the user made, then tracking and codifying those using a DSL over Claude Code hooks gives you a nice anti-slop flywheel. your hooks live in nested json, but the model only ever edits a few lines of dsl. ``` { "hooks": { "PreToolUse": [ { "matcher": "Bash", "hooks": [ { "type": "command", "if": "Bash(rm *)", "command": "echo 'BLOCKED: rm is banned in this repo. use trash instead.' >&2; exit 2" }, { "type": "command", "if": "Bash(pnpm install*)", "command": "echo 'BLOCKED: pnpm install at the root. cd into the workspace first.' >&2; exit 2" } ] } ] } } block_command( ["rm"], reason="rm is banned in this repo", hint="use trash instead", ) block_command( ["pnpm", "install"], reason="pnpm install at the root", hint="cd into the workspace first", ) ``` - added Another great example came up recently while building Aneta’s context compiler. The idea behind the context compiler is pretty simple: 1. Take a bunch of data, prompts, and examples and represent them in a unified IR. 2. Run cache-aware optimization passes over that IR (every pass weighs the tradeoffs of busting the cache vs decreasing tokens, depending on what that particular call is optimizing for). 3. Render that IR to the provider-specific format for the LLM being called. The trick is capturing a closure at IR-construction time that knows how to render a given piece of context in both verbose and compact ways, and then allowing the optimization passes to toggle one or the other (in addition to many standard compaction techniques). The compiler can be optimizing for cost, quality, speed, or some linear combination of those. the same context compiled for `COST`, `QUALITY`, and `SPEED`, where each pass weighs the tokens it saves against the cached tokens it gives up. context nodes, in the order they are senttokens open a node to see both of its renderings. - cache ends here - verbose, 900 tokens ``` php query_table(arm="b") -> {"arm": "b", "n": 12, "index": 0.44, "sd": 0.03, "p": 0.01, "rows": [12 rows of raw counts]} ``` compact, 240 tokenssent ``` {"arm": "b", "n": 12, "index": 0.44, "sd": 0.03, "p": 0.01} ``` - read from cache - sent fresh - cut by a pass passes for `COST` - runscompact the new tool resultsaves 660 tokens, loses none from cache - skipscompact the examples and the tablesaves 3,520 tokens, loses 7,884 from cache - skipsswap figures 1 and 2 for their 24-token captions, since this turn needs only figure 3saves 2,898 tokens, loses 3,584 from cache `COST` only runs a pass that leaves the cache alone, since a cached token bills at a fraction of a fresh one. | tokens in this request | | | |---|---|---| | objective | total | read from cache | |---|---|---| | no passes | 9,464 | 8,564 | | `COST` | 8,804 | 8,564 | | `QUALITY` | 6,566 | 4,980 | | `SPEED` | 2,386 | 680 | COST sends 8,804 tokens, 8,564 of them read from cache. One of the more interesting optimization passes we do is around images. Images take up HUGE numbers of tokens, yet are often necessary to include in scientific queries as they are figures that are referenced, and OCRing them just doesn’t give sufficient fidelity. However blindly including them results in much larger contexts, and the quality degradation that comes with those. When operating in `SPEED` or `QUALITY` modes, the compiler has a pass that allows it to selectively exclude images from the context when compiling (along with a tool for the LLM to call if it then decides it needs that image). Except, how do you decide when to include the image? what re-sending every figure on every turn costs, against sending only the one each turn needs. one figure, cut into 28 px patches 1000 x 1000 px -> 36 x 36 patches -> 1,296 tokens over a thread image tokens sent over 9 turns - re-send every figure - 23,328 tokens - send only the needed one - 11,664 tokens re-sending every figure costs 2x. You can start with simple heuristics like how long ago the image was last mentioned, but that breaks down pretty fast in deep research. Another idea is to have a small sidecar LLM (Haiku-class or smaller) make the decision at every turn; this works surprisingly well when in `QUALITY` mode, but is prohibitively slow for `SPEED`. Inspecting the decisions that sidecar is making yields an interesting insight however: given a static summary of the conversation the decision is a pure function of the message about to be sent. should the chart be included with each message? the right call first, then what each strategy did. - right call - in when answering the message needs the chart - heuristic - in when the message or one of the 2 before it says "chart" - sidecar - a small llm decides on every turn - wrong call - still deciding each sidecar call holds its message for about 4,100 ms, shown here 5x faster.the heuristic answers in under 1 ms and gets 3 of 10 wrong. the sidecar gets all 10 right, but each message waits about 4,100 ms for it. So the question we want to quickly answer is: given a somewhat-recent summary of the conversation, and the message about to be sent, should we include this image? Most of the features the sidecar LLM gathers to make that decision end up being either those easily derived from classic NLP toolboxes, or small binary decisions that an on-device model could make. So how do we handle image inclusion in `SPEED` mode? You can probably guess where I’m going with this… a DSL! It turns out you can just ask the same sidecar model to write a small function, and provide it with a few helpers. Here are the ones we provide: ``` php async def nli_scores(a: str, b: str) -> tuple[float, float, float]: ... async def text_similarity(a: str, b: str) -> float: ... async def text_entities(a: str) -> list[Entity]: ... async def text_tokens(a: str) -> list[Token]: ... async def call_llm(prompt: str) -> str: ... async def call_llm_typed[M](prompt: str, model: type[M]) -> M: ... ``` the sidecar writes the scorer once per image, and the sandbox sends back anything it can't run. compilerhow much does a new message need the cell division bar chart? write score_message to answer from 0 to 100. ``` php async def score_message(message: str) -> int: text = message.lower() ref = "cell division bar chart mitotic index data" sim, nli = await gather( text_similarity(message, ref), nli_scores(message, ref), ) score = int((sim * 0.5 + nli[0] * 0.5) * 100) def bonus(raw: int) -> int: if "chart" in text or "graph" in text or "data" in text: return min(100, raw + 15) return raw return bonus(score) if "chart" in text or "graph" in text or "data" in text: score = min(100, score + 15) return score ``` 1. sandboxrejects draft 1 because nested def bonus() is not allowedrejected 2. sidecarinlines the bonus, 11 lines instead of 15 3. sandboxsample message "which phase dominates in the division chart?" scores 71 of 100, threshold 65include - helper call - removed from draft 1 - added in draft 2 With just these six small helpers, we end up with relatively effective scoring functions (this is a real one!) that are cheap enough to run on every new message. ``` php async def score_message(message: str) -> int: text = message.lower() ref = "cell division bar chart mitotic index data" sim, nli = await gather( text_similarity(message, ref), nli_scores(message, ref), ) score = int((sim * 0.5 + nli[0] * 0.5) * 100) if "chart" in text or "graph" in text or "data" in text: score = min(100, score + 15) return score ``` the `score_message` above, run on five messages. a message that scores 65 or more gets the chart. `score_message` on the selected message `async def score_message(message: str) -> int: text = message.lower() ref = "cell division bar chart mitotic index data" sim, nli = await gather( text_similarity(message, ref),# sim = 0.625 nli_scores(message, ref),# nli[0] = 0.5 ) score = int((sim * 0.5 + nli[0] * 0.5) * 100)# score = 56 if "chart" in text or "graph" in text or "data" in text:# True score = min(100, score + 15)# score = 71 return score# 71include = await score_message(message) >= 65include` - values for the selected message similarity 0.625 and entailment 0.5 average out to 56, and "chart" adds 15, so 71 clears 65. It’s then just a matter of asynchronously generating one of these every time an image is sent in a conversation (`history x image -> message -> int`), keeping a [Monty](https://github.com/pydantic/monty) interpreter ready, et voilà! Using this approach has let us be much more free with the number of images we introduce in a long research thread. Give your LLM more DSLs! the same ten messages, with the generated scorer added. - right call - in when answering the message needs the chart - heuristic - in when the message or one of the 2 before it says "chart" - sidecar - a small llm decides on every turn - scorer - a function the sidecar wrote decides on every turn - wrong call - still deciding each sidecar call holds its message for about 4,100 ms, shown here 5x faster.the scorer misses only turn 6, at about 20 ms a message against the sidecar's 4,100 ms.