# How We Got an LLM to Draw Charts Without Ever Touching a Pixel > Source: > Published: 2026-08-16 12:41:39+00:00 Let's get something out of the way first. Having data is good. Having a database full of reviews, commits, and org activity sitting there quietly, untouched, unread, never once glanced at by a human being with a coffee and an opinion? That's not "having data." That's a very expensive data graveyard. At [LiveReview](https://github.com/HexmosTech/LiveReview), we build what we call a **Blast-Radius Aware AI Code Review for Business-Critical Systems**. Which is a fancy way of saying: we review your code, we figure out how bad it would be if a change goes wrong, and we don't shut up about it until someone fixes it. Along the way we accumulate a review data: who reviewed, how much, how fast, how often, which repos are on fire. And for a while, that pile just sat there. Engineering leaders would ask "is adoption increasing?" and get back a vibe, not an answer. So we built [Livi](https://livereview.hexmos.com/#/chat), a chat bot that answers real questions about that data with real charts, not paragraphs of hedging. This post technically about how Livi draws those charts. Specifically: why we never let the LLM touch a pixel, how the same chart definition ends up as both a live interactive graph in your browser and a flat PNG in a Slack thread, and why teaching a language model to pick the *right* chart shape is a surprisingly deep rabbit hole. The tempting, wrong idea is: "let's have the LLM generate an image." Please don't. Image-generating models are a different beast entirely, and even if you got one to draw a bar chart, you'd have no way to verify the numbers on it are real. You'd be trusting a model that hallucinates plausible-sounding review counts to also render them faithfully into pixels. That's not a chart, that's chart-shaped fan fiction. The actually good idea, and the one every serious LLM-charting integration eventually converges on, is: the LLM writes ** Vega-Lite**, a JSON grammar for describing charts declaratively. You don't say "draw a blue bar going up." You say: ``` { "mark": "bar", "encoding": { "x": { "field": "month", "type": "temporal" }, "y": { "field": "review_count", "type": "quantitative" } } } ``` That's it. That's the entire chart. No pixels, no drawing, just a description of *what the data means and how it should be mapped to a picture*. [Vega-Lite](https://vega.github.io/vega-lite/) does the actual drawing. The LLM's job shrinks down to something it's genuinely good at: filling in a well-defined schema. Models are much better at "pick `mark: bar` or `mark: line` " than "hallucinate 600 pixels of a correct y-axis." And critically: the LLM never sees the actual numbers before they're rendered. It writes the SQL, we run it, and the real result set gets stitched into `data.values` by our own Go code. The model can be as creative as it wants about *presentation*. It gets zero creative license over *the numbers*. Here's the pipeline, roughly: A few things worth dwelling on here, because each one exists because something went wrong first. **Why two SQL-writing steps instead of one?** Because the model needs to know how many rows the answer will have *before* it can decide whether a chart makes sense or whether it should hand you a CSV instead. Nobody wants a bar chart with 4,000 bars. So step one is basically "how big is this going to be," and step two is "okay, now actually get me the data and tell me how to draw it." **Why is there a SQL guard at all?** Because an LLM writing raw SQL against a multi-tenant database is one confidently-worded prompt away from `org_id = 1 OR 1 = 1` . We run every generated query through a guard that rejects anything that isn't read-only, checks every table against a denylist, and specifically looks for the shape of a tenant-isolation bypass (constant-vs-constant comparisons, bare `OR TRUE` , all the classics). It's not glamorous work, but it's the difference between "cool AI feature" and "why is Org A looking at Org B's review data" showing up in an incident channel. (meme placeholder: "Well Yes, But Actually No" / bike fall guy. Top text: "the query has an`org_id` filter." Bottom text (mid-fall): "`WHERE org_id = 1 OR 1=1` ") **Why does the LLM only ever see a narrowed slice of the schema, not the whole database?** Because our actual schema has north of fifty tables, and dumping all of them into every prompt is both expensive and a great way to get the model confused about which `created_at` belongs to which table. More on this in a second, it deserves its own section. Here's a problem that doesn't show up until your schema stops being a toy demo. We're at 58 tables and counting: reviews, pull requests, AI comments, review feedback, billing, licensing, job queues, the works. If we pasted the full schema into every single prompt, we'd be burning thousands of tokens per question just describing `license_seat_assignments` to a model that was asked "how many reviews happened last month" and does not, will never, care. So instead of "here is the entire database, good luck," we use [dbctx](https://github.com/shrsv/dbctx), a Go library built specifically for this problem: given a natural-language question and a live Postgres connection, hand back only the tables that actually matter, formatted as compact, LLM-friendly text instead of a raw `information_schema` dump. **dbctx** is a Go library and CLI tool that compiles a PostgreSQL database into a portable, queryable context index (`.dtx` file). It extracts schema, relationships, field semantics, representative values, JSONB structure, and builds a full-text search index — all from deterministic introspection, statistics, and heuristics, with no generative LLM and no external services required for the core index. It also supports an optional local semantic embedding signal and an optional, user-controlled terminology dictionary — both additive, both off-by-default-cost, described below. Use it to give text-to-SQL systems, AI agents, and database-aware applications a compact, relevant slice of your database schema at query time, instead of dumping the entire `information_schema` into every prompt. **Key features:** It does this with a genuinely layered retrieval pipeline, not just a keyword grep. Lexical and fuzzy matching against table and column names, full-text search, matching against actual sampled values in the data, an optional semantic embedding pass, and a curated terminology layer we feed it ourselves (so "LOC" resolves to `billable_loc` , and "MR" and "PR" both resolve to the same underlying pull-request concept, because our users say both depending on which Git host they came from). Whatever scores above zero gets pulled in, plus anything reachable through a foreign key from something that scored, because a join target with zero lexical overlap with the question is still often exactly what the query needs. The payoff is not subtle. On our real schema, a typical question narrows 58 tables down to somewhere around 25 to 30, which is roughly half the context gone before the model has written a single character of SQL. That's not a rounding-error optimization, that's the difference between a prompt the model can actually reason clearly about and one where the important tables are buried in a wall of billing and license-seat noise. So now we have a Vega-Lite spec. Great. If the user is on the web dashboard, we're basically done, more on that in a second. But what about Slack? What about Discord? Those platforms don't run a JavaScript charting library inside a chat message. A Slack message is not a browser tab. You cannot politely ask Slack to interpret a Vega-Lite spec and render SVG for you. So for anywhere that isn't our own frontend, we need an actual image file. This is where [vl-convert](https://github.com/vega/vl-convert) comes in: a Rust binary (with Python and Node bindings, but we shell out to the CLI) that takes a Vega-Lite spec and rasterizes it straight to PNG, no headless browser required. That last part matters more than it sounds. The old-and-busted way to render a chart server-side is to spin up a headless Chrome instance, load a page with a charting library, screenshot it, and pray your Docker image doesn't balloon to two gigabytes. `vl-convert` skips all of that. It's a single binary, it takes JSON in, it gives PNG bytes out, and it's fast enough that nobody notices it happening. (meme placeholder: Kombucha Girl, disgusted-then-intrigued two-panel. Disgusted panel: "spin up headless Chrome to screenshot a chart." Intrigued panel: "one Rust binary, JSON in, PNG out.") Here's the part I actually think is neat. We generate **one** Vega-Lite spec per chart. What happens to it next depends entirely on where it's going. (meme placeholder: Trade Offer / Minecraft villager trading. Give: "one Vega-Lite spec." Take: "a live interactive chart in the browser, or a flat PNG in a Slack thread, depending on where it lands.") On the web, the frontend just hands the raw spec to [react-vega](https://github.com/vega/react-vega) and lets the browser do the work. You get hover tooltips, you get resizing, you get an actually interactive chart, and our backend does zero image rendering for that path. It just ships JSON. For Slack and Discord, the exact same spec gets routed through `vl-convert` instead, turned into a flat PNG, and attached as a file to the message. The bot doesn't know or care that it's the "same" chart taking a different road. From the pipeline's point of view, a Vega-Lite spec is just a Vega-Lite spec. Where it ends up decides whether it becomes living, breathing SVG or a JPEG-adjacent screenshot sitting quietly in a chat thread forever. This split is also why we can add new chart destinations cheaply. Want an emailed weekly digest with embedded charts? Same spec, same `vl-convert` path, new delivery mechanism. The hard problem (getting a *correct*, *sensible* chart spec out of an LLM) is solved exactly once. The whole point of [Livi](https://livereview.hexmos.com/#/chat) was never "add a chatbot," it was "close the loop between the data we're already collecting and the person who actually needs to act on it." A CTO asking "are engineers actually using this thing" should get an answer that looks like a calendar heatmap of usage rhythm, not a spreadsheet and a shrug. The recipe, if you're building something similar, is genuinely not complicated: `vl-convert` , in our case) and let LiveReview reviews your code, tells you how bad a change could go, and won't shut up until it's fixed — and Livi turns all that accumulated review data into charts a human can actually act on. If you liked the post, drop a ⭐ on the [repo](https://github.com/HexmosTech/LiveReview) and try [LiveReview](https://hexmos.com/livereview/) now.