I put a real LLM in my portfolio. It runs on your GPU, not my server. Engineer Rahul Agarwal built a portfolio site that runs a 0.5B-parameter Qwen2.5-Instruct model entirely in the visitor's browser via WebGPU and MLC's WebLLM, with no server or API key involved. The chat answers questions about his work through a keyword-scoring retrieval step over a dozen hand-written notes, constrained by a strict system prompt to avoid hallucination, and includes a warm-up generation to hide WebGPU's lazy shader compilation. He also fixed a Cumulative Layout Shift of 1.41 caused by a text-scramble headline animation by locking each word to its real width inside inline-block spans. Every AI-engineer portfolio says "I build with LLMs." I wanted mine to prove it in the first ten seconds. So the chat box on my homepage has a button: Run a real LLM on your GPU . Click it and a 0.5B-parameter model downloads into your browser, boots on your graphics card through WebGPU, and answers questions about my work. No server. No API key. Nothing you type leaves your machine. ๐Ÿ‘‰ Try it: rahul-agarwal.in https://www.rahul-agarwal.in desktop Chrome or Edge works best Here's how it works, and the four things I got wrong before it worked well. js const adapter = await navigator.gpu.requestAdapter ; const f16 = adapter.features.has 'shader-f16' ; const model = f16 ? 'Qwen2.5-0.5B-Instruct-q4f16 1-MLC' : 'Qwen2.5-0.5B-Instruct-q4f32 1-MLC'; // fallback for GPUs without fp16 const webllm = await import 'https://cdn.jsdelivr.net/npm/@mlc-ai/web-llm@0.2.86/+esm' ; const engine = await webllm.CreateMLCEngine model, { initProgressCallback: r = updateProgressBar r.progress, r.text , } ; That's it. WebLLM exposes an OpenAI-style API, so streaming looks familiar: js const chunks = await engine.chat.completions.create { messages, stream: true, temperature: 0.2, max tokens: 160, } ; for await const c of chunks { const token = c.choices 0 ?.delta?.content || ''; output.textContent += token; window.neural.fire 1 ; // light up the hero network } Two details matter: pin the CDN version an unpinned import can change under you , and check for shader-f16 instead of assuming it. Plenty of laptop GPUs don't have it. A half-billion-parameter model knows nothing about me and will happily invent things. So it never answers from memory. Every question goes through a retrieval step first: // Simplified from the real code, which also weights longer keywords higher function retrieve question { const q = question.toLowerCase ; return NOTES .map note = { note, score: note.keys.filter k = q.includes k .length } .filter x = x.score 0 .sort a, b = b.score - a.score .slice 0, 2 .map x = x.note ; } const SYSTEM = Answer in 1 to 3 short sentences using ONLY the notes provided. If the notes don't contain the answer, say you don't know. Never invent companies, numbers, dates or links. ; The notes are a dozen short paragraphs I wrote projects, stack, experience . Keyword scoring sounds crude next to embeddings, but with a dozen documents it's instant, deterministic and easy to debug. Under every answer the UI shows what it used: Q: Where does he work now? A: Rahul is the sole engineer at Aryan News Agency, where he builds and develops websites and apps. โ†ณ retrieved 1 note intro ยท 22 tokens in 2.8s ยท generated on your GPU And the test I cared about most: Q: What is his favourite food? A: Rahul is not specifically known for his favorite food. No hallucination. Low temperature plus a strict "only the notes" prompt does most of the work. The model loaded fine, but the first question sat there "thinking." WebGPU compiles its shaders lazily, on the first real inference. The fix is a throwaway warm-up generation while the progress bar is still visible: await engine.chat.completions.create { messages: { role: 'user', content: 'Hi' } , max tokens: 4, } ; Now the slow part happens during loading, where users expect waiting, not after they've asked a question. My headline "decodes" from random glyphs into the real text. It looked great, and it gave the page a Cumulative Layout Shift of 1.41 anything above 0.25 is "poor" . Random glyphs have different widths, so the line re-wrapped about 60 times a second and shoved everything below it around. The fix: wrap each word in an inline-block, lock it to its real width, scramble inside it, then unwrap. js for const w of words { w.span.style.display = 'inline-block'; w.span.style.width = w.span.getBoundingClientRect .width + 'px'; } // ...animate...then restore plain text nodes words.forEach w = w.span.replaceWith document.createTextNode w.text ; Same effect, layout shift 0.00 . One more catch: measure after document.fonts.ready , or you lock in the fallback font's widths. I added a strict Content-Security-Policy hash-based, no unsafe-inline . Locally: zero violations. In production: errors. My domain was proxied through Cloudflare in front of Vercel, and Cloudflare was injecting its own inline scripts analytics and bot detection and rewriting my email address into an obfuscated link. That put an