{"slug": "how-we-got-an-llm-to-draw-charts-without-ever-touching-a-pixel", "title": "How We Got an LLM to Draw Charts Without Ever Touching a Pixel", "summary": "LiveReview, a developer tool for AI-powered code review, has built Livi, a chat bot that answers questions about review data with charts. The bot uses an LLM to generate Vega-Lite chart specifications rather than images, ensuring data accuracy, and includes a SQL guard to prevent tenant-isolation bypasses.", "body_md": "Let's get something out of the way first.\n\nHaving data is good. Having a database full of reviews, commits, and org activity sitting there quietly, untouched, unread, never once glanced at by a human being with a coffee and an opinion? That's not \"having data.\" That's a very expensive data graveyard.\n\nAt [LiveReview](https://github.com/HexmosTech/LiveReview), we build what we call a **Blast-Radius Aware AI Code Review for Business-Critical Systems**.\n\nWhich is a fancy way of saying: we review your code, we figure out how bad it would be if a change goes wrong, and we don't shut up about it until someone fixes it.\n\nAlong the way we accumulate a review data: who reviewed, how much, how fast, how often, which repos are on fire.\n\nAnd for a while, that pile just sat there.\n\nEngineering leaders would ask \"is adoption increasing?\" and get back a vibe, not an answer.\n\nSo we built [Livi](https://livereview.hexmos.com/#/chat), a chat bot that answers real questions about that data with real charts, not paragraphs of hedging.\n\nThis post technically about how Livi draws those charts.\n\nSpecifically: why we never let the LLM touch a pixel, how the same chart definition ends up as both a live interactive graph in your browser and a flat PNG in a Slack thread, and why teaching a language model to pick the *right* chart shape is a surprisingly deep rabbit hole.\n\nThe tempting, wrong idea is: \"let's have the LLM generate an image.\" Please don't.\n\nImage-generating models are a different beast entirely, and even if you got one to draw a bar chart, you'd have no way to verify the numbers on it are real.\n\nYou'd be trusting a model that hallucinates plausible-sounding review counts to also render them faithfully into pixels.\n\nThat's not a chart, that's chart-shaped fan fiction.\n\nThe actually good idea, and the one every serious LLM-charting integration eventually converges on, is: the LLM writes ** Vega-Lite**, a JSON grammar for describing charts declaratively.\n\nYou don't say \"draw a blue bar going up.\" You say:\n\n```\n{\n  \"mark\": \"bar\",\n  \"encoding\": {\n    \"x\": { \"field\": \"month\", \"type\": \"temporal\" },\n    \"y\": { \"field\": \"review_count\", \"type\": \"quantitative\" }\n  }\n}\n```\n\nThat's it.\n\nThat's the entire chart.\n\nNo pixels, no drawing, just a description of *what the data means and how it should be mapped to a picture*.\n\n[Vega-Lite](https://vega.github.io/vega-lite/) does the actual drawing.\n\nThe LLM's job shrinks down to something it's genuinely good at: filling in a well-defined schema.\n\nModels are much better at \"pick `mark: bar`\n\nor `mark: line`\n\n\" than \"hallucinate 600 pixels of a correct y-axis.\"\n\nAnd critically: the LLM never sees the actual numbers before they're rendered.\n\nIt writes the SQL, we run it, and the real result set gets stitched into `data.values`\n\nby our own Go code.\n\nThe model can be as creative as it wants about *presentation*.\n\nIt gets zero creative license over *the numbers*.\n\nHere's the pipeline, roughly:\n\nA few things worth dwelling on here, because each one exists because something went wrong first.\n\n**Why two SQL-writing steps instead of one?** Because the model needs to know how many rows the answer will have *before* it can decide whether a chart makes sense or whether it should hand you a CSV instead.\n\nNobody wants a bar chart with 4,000 bars.\n\nSo step one is basically \"how big is this going to be,\" and step two is \"okay, now actually get me the data and tell me how to draw it.\"\n\n**Why is there a SQL guard at all?** Because an LLM writing raw SQL against a multi-tenant database is one confidently-worded prompt away from `org_id = 1 OR 1 = 1`\n\n.\n\nWe run every generated query through a guard that rejects anything that isn't read-only, checks every table against a denylist, and specifically looks for the shape of a tenant-isolation bypass (constant-vs-constant comparisons, bare `OR TRUE`\n\n, all the classics).\n\nIt's not glamorous work, but it's the difference between \"cool AI feature\" and \"why is Org A looking at Org B's review data\" showing up in an incident channel.\n\n(meme placeholder: \"Well Yes, But Actually No\" / bike fall guy. Top text: \"the query has an`org_id`\n\nfilter.\" Bottom text (mid-fall): \"`WHERE org_id = 1 OR 1=1`\n\n\")\n\n**Why does the LLM only ever see a narrowed slice of the schema, not the whole database?**\n\nBecause our actual schema has north of fifty tables, and dumping all of them into every prompt is both expensive and a great way to get the model confused about which `created_at`\n\nbelongs to which table. More on this in a second, it deserves its own section.\n\nHere's a problem that doesn't show up until your schema stops being a toy demo.\n\nWe're at 58 tables and counting: reviews, pull requests, AI comments, review feedback, billing, licensing, job queues, the works.\n\nIf we pasted the full schema into every single prompt, we'd be burning thousands of tokens per question just describing `license_seat_assignments`\n\nto a model that was asked \"how many reviews happened last month\" and does not, will never, care.\n\nSo instead of \"here is the entire database, good luck,\" we use [dbctx](https://github.com/shrsv/dbctx), a Go library built specifically for this problem: given a natural-language question and a live Postgres connection, hand back only the tables that actually matter, formatted as compact, LLM-friendly text instead of a raw `information_schema`\n\ndump.\n\n**dbctx** is a Go library and CLI tool that compiles a PostgreSQL database into a portable, queryable context index (`.dtx`\n\nfile). It extracts schema, relationships, field semantics, representative values, JSONB structure, and builds a full-text search index — all from deterministic introspection, statistics, and heuristics, with no generative LLM and no external services required for the core index. It also supports an optional local semantic embedding signal and an optional, user-controlled terminology dictionary — both additive, both off-by-default-cost, described below.\n\nUse it to give text-to-SQL systems, AI agents, and database-aware applications a compact, relevant slice of your database schema at query time, instead of dumping the entire `information_schema`\n\ninto every prompt.\n\n**Key features:**\n\nIt does this with a genuinely layered retrieval pipeline, not just a keyword grep.\n\nLexical and fuzzy matching against table and column names, full-text search, matching against actual sampled values in the data, an optional semantic embedding pass, and a curated terminology layer we feed it ourselves (so \"LOC\" resolves to `billable_loc`\n\n, and \"MR\" and \"PR\" both resolve to the same underlying pull-request concept, because our users say both depending on which Git host they came from).\n\nWhatever scores above zero gets pulled in, plus anything reachable through a foreign key from something that scored, because a join target with zero lexical overlap with the question is still often exactly what the query needs.\n\nThe payoff is not subtle.\n\nOn our real schema, a typical question narrows 58 tables down to somewhere around 25 to 30, which is roughly half the context gone before the model has written a single character of SQL.\n\nThat's not a rounding-error optimization, that's the difference between a prompt the model can actually reason clearly about and one where the important tables are buried in a wall of billing and license-seat noise.\n\nSo now we have a Vega-Lite spec. Great.\n\nIf the user is on the web dashboard, we're basically done, more on that in a second.\n\nBut what about Slack? What about Discord? Those platforms don't run a JavaScript charting library inside a chat message.\n\nA Slack message is not a browser tab.\n\nYou cannot politely ask Slack to interpret a Vega-Lite spec and render SVG for you.\n\nSo for anywhere that isn't our own frontend, we need an actual image file.\n\nThis is where [vl-convert](https://github.com/vega/vl-convert) comes in: a Rust binary (with Python and Node bindings, but we shell out to the CLI) that takes a Vega-Lite spec and rasterizes it straight to PNG, no headless browser required.\n\nThat last part matters more than it sounds.\n\nThe old-and-busted way to render a chart server-side is to spin up a headless Chrome instance, load a page with a charting library, screenshot it, and pray your Docker image doesn't balloon to two gigabytes.\n\n`vl-convert`\n\nskips all of that.\n\nIt's a single binary, it takes JSON in, it gives PNG bytes out, and it's fast enough that nobody notices it happening.\n\n(meme placeholder: Kombucha Girl, disgusted-then-intrigued two-panel. Disgusted panel: \"spin up headless Chrome to screenshot a chart.\" Intrigued panel: \"one Rust binary, JSON in, PNG out.\")\n\nHere's the part I actually think is neat.\n\nWe generate **one** Vega-Lite spec per chart.\n\nWhat happens to it next depends entirely on where it's going.\n\n(meme placeholder: Trade Offer / Minecraft villager trading. Give: \"one Vega-Lite spec.\" Take: \"a live interactive chart in the browser, or a flat PNG in a Slack thread, depending on where it lands.\")\n\nOn the web, the frontend just hands the raw spec to [react-vega](https://github.com/vega/react-vega) and lets the browser do the work.\n\nYou get hover tooltips, you get resizing, you get an actually interactive chart, and our backend does zero image rendering for that path.\n\nIt just ships JSON.\n\nFor Slack and Discord, the exact same spec gets routed through `vl-convert`\n\ninstead, turned into a flat PNG, and attached as a file to the message.\n\nThe bot doesn't know or care that it's the \"same\" chart taking a different road.\n\nFrom the pipeline's point of view, a Vega-Lite spec is just a Vega-Lite spec.\n\nWhere it ends up decides whether it becomes living, breathing SVG or a JPEG-adjacent screenshot sitting quietly in a chat thread forever.\n\nThis split is also why we can add new chart destinations cheaply.\n\nWant an emailed weekly digest with embedded charts? Same spec, same `vl-convert`\n\npath, new delivery mechanism.\n\nThe hard problem (getting a *correct*, *sensible* chart spec out of an LLM) is solved exactly once.\n\nThe whole point of [Livi](https://livereview.hexmos.com/#/chat) was never \"add a chatbot,\" it was \"close the loop between the data we're already collecting and the person who actually needs to act on it.\"\n\nA CTO asking \"are engineers actually using this thing\" should get an answer that looks like a calendar heatmap of usage rhythm, not a spreadsheet and a shrug.\n\nThe recipe, if you're building something similar, is genuinely not complicated:\n\n`vl-convert`\n\n, in our case) and let LiveReview reviews your code, tells you how bad a change could go, and won't shut up until it's fixed — and Livi turns all that accumulated review data into charts a human can actually act on.\n\nIf you liked the post, drop a ⭐ on the [repo](https://github.com/HexmosTech/LiveReview) and try [LiveReview](https://hexmos.com/livereview/) now.", "url": "https://wpnews.pro/news/how-we-got-an-llm-to-draw-charts-without-ever-touching-a-pixel", "canonical_source": "https://dev.to/lovestaco/how-we-got-an-llm-to-draw-charts-without-ever-touching-a-pixel-1i21", "published_at": "2026-08-16 12:41:39+00:00", "updated_at": "2026-08-16 13:12:28.450658+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-products", "ai-tools", "ai-infrastructure"], "entities": ["LiveReview", "Livi", "Vega-Lite", "HexmosTech"], "alternates": {"html": "https://wpnews.pro/news/how-we-got-an-llm-to-draw-charts-without-ever-touching-a-pixel", "markdown": "https://wpnews.pro/news/how-we-got-an-llm-to-draw-charts-without-ever-touching-a-pixel.md", "text": "https://wpnews.pro/news/how-we-got-an-llm-to-draw-charts-without-ever-touching-a-pixel.txt", "jsonld": "https://wpnews.pro/news/how-we-got-an-llm-to-draw-charts-without-ever-touching-a-pixel.jsonld"}}