Every AI-engineer portfolio says "I build with LLMs." I wanted mine to prove it in the first ten seconds.
So the chat box on my homepage has a button: Run a real LLM on your GPU. Click it and a 0.5B-parameter model downloads into your browser, boots on your graphics card through WebGPU, and answers questions about my work. No server. No API key. Nothing you type leaves your machine.
👉 Try it: rahul-agarwal.in (desktop Chrome or Edge works best)
Here's how it works, and the four things I got wrong before it worked well.
const adapter = await navigator.gpu.requestAdapter();
const f16 = adapter.features.has('shader-f16');
const model = f16
? 'Qwen2.5-0.5B-Instruct-q4f16_1-MLC'
: 'Qwen2.5-0.5B-Instruct-q4f32_1-MLC'; // fallback for GPUs without fp16
const webllm = await import('https://cdn.jsdelivr.net/npm/@mlc-ai/web-llm@0.2.86/+esm');
const engine = await webllm.CreateMLCEngine(model, {
initProgressCallback: r => updateProgressBar(r.progress, r.text),
});
That's it. WebLLM exposes an OpenAI-style API, so streaming looks familiar:
const chunks = await engine.chat.completions.create({
messages, stream: true, temperature: 0.2, max_tokens: 160,
});
for await (const c of chunks) {
const token = c.choices[0]?.delta?.content || '';
output.textContent += token;
window.neural.fire(1); // light up the hero network
}
Two details matter: pin the CDN version (an unpinned import can change under you), and check for shader-f16 instead of assuming it. Plenty of laptop GPUs don't have it.
A half-billion-parameter model knows nothing about me and will happily invent things. So it never answers from memory. Every question goes through a retrieval step first:
// Simplified from the real code, which also weights longer keywords higher
function retrieve(question) {
const q = question.toLowerCase();
return NOTES
.map(note => ({ note, score: note.keys.filter(k => q.includes(k)).length }))
.filter(x => x.score > 0)
.sort((a, b) => b.score - a.score)
.slice(0, 2)
.map(x => x.note);
}
const SYSTEM = `Answer in 1 to 3 short sentences using ONLY the notes provided.
If the notes don't contain the answer, say you don't know.
Never invent companies, numbers, dates or links.`;
The notes are a dozen short paragraphs I wrote (projects, stack, experience). Keyword scoring sounds crude next to embeddings, but with a dozen documents it's instant, deterministic and easy to debug. Under every answer the UI shows what it used:
Q: Where does he work now?
A: Rahul is the sole engineer at Aryan News Agency, where he builds and develops websites and apps.
↳ retrieved 1 note (intro) · 22 tokens in 2.8s · generated on your GPU
And the test I cared about most:
Q: What is his favourite food?
A: Rahul is not specifically known for his favorite food.
No hallucination. Low temperature plus a strict "only the notes" prompt does most of the work.
The model loaded fine, but the first question sat there "thinking." WebGPU compiles its shaders lazily, on the first real inference. The fix is a throwaway warm-up generation while the progress bar is still visible:
await engine.chat.completions.create({
messages: [{ role: 'user', content: 'Hi' }], max_tokens: 4,
});
Now the slow part happens during , where users expect waiting, not after they've asked a question.
My headline "decodes" from random glyphs into the real text. It looked great, and it gave the page a Cumulative Layout Shift of 1.41 (anything above 0.25 is "poor"). Random glyphs have different widths, so the line re-wrapped about 60 times a second and shoved everything below it around.
The fix: wrap each word in an inline-block, lock it to its real width, scramble inside it, then unwrap.
for (const w of words) {
w.span.style.display = 'inline-block';
w.span.style.width = w.span.getBoundingClientRect().width + 'px';
}
// ...animate...then restore plain text nodes
words.forEach(w => w.span.replaceWith(document.createTextNode(w.text)));
Same effect, layout shift 0.00. One more catch: measure after document.fonts.ready, or you lock in the fallback font's widths.
I added a strict Content-Security-Policy (hash-based, no unsafe-inline). Locally: zero violations. In production: errors. My domain was proxied through Cloudflare in front of Vercel, and Cloudflare was injecting its own inline scripts (analytics and bot detection) and rewriting my email address into an obfuscated link. That put an <a> inside my "copy email" <button>.
Switching the DNS records to "DNS only" removed the extra hop and the rewriting. The strict CSP went back on, and the page got faster. If you put a CDN in front of a platform that's already a CDN, check what it's adding to your pages.
Plenty of visitors (many phones, older browsers) can't run the model. The chat works without it: by default it streams pre-written answers token by token, clearly labelled as "scripted mode". The real model is opt-in, behind an explicit button that shows the download size. Nobody gets a 270 MB surprise on mobile data.
| Framework | none: plain HTML, CSS and JavaScript | | HTML + CSS + JS over the wire | ~37 KB gzipped | | Lighthouse (desktop) | 100 / 100 / 100 / 100 | | Lighthouse (mobile) | 97 / 100 / 100 / 100 | | Mozilla Observatory | A+ | | Model download (optional) | ~270 MB once, then cached |
No React, no build step. Every heavy thing (the model, the runtime) loads only when someone asks for it, and every animation s when it scrolls off screen.
Have you shipped anything that runs a model fully client-side? I'd love to hear what you used and what broke. Drop it in the comments 👇
I'm Rahul, an AI engineer in Bengaluru who builds tools that ship. More of my work is at rahul-agarwal.in, and the code is on GitHub.