# I Trained a GPT on 15 Years of My Life — Then Interrogated It > Source: > Published: 2026-07-17 00:00:00+00:00 # I Trained a GPT on 15 Years of My Life — Then Interrogated It Everyone explains LLMs with someone else's diagrams. I wanted to hold one — small enough to audit, trained on data I understand better than anyone: my own. So over a couple of days I ran the entire modern LLM pipeline — pretraining, supervised fine-tuning, chat interface, interpretability probes — on a laptop, using my Google Takeout as the corpus. I built a 10.77M-parameter GPT from scratch on 4.45M characters of my own Google data — pretraining, SFT, and interpretability probes, for $0, in ~4 hours on a MacBook Air. - Model belief tracks the **log** of training-data mentions (r = 0.95) **Context beats volume**: my second-most-mentioned weekday is one the model won't recommend** The last training run wins**: fine-tuning re-weighted every entity toward recent data- My own counting bug (“ac **ross**” ≠ Ross) shows why you should audit every mention dashboard The result is a tiny ghost of me that schedules coffee at “14:30am,” believes Wednesday is the best day of the week, and taught me more about AI “knowledge” than a year of reading papers. ## The corpus: me Google Takeout will hand you everything the company knows about you. Mine was 133 GB, most of it photos and video. The text — the part a language model can eat — was tiny by comparison: **67,792 search queries** across 15 years (Search, Maps, YouTube, Flights, Gmail search) — 1.35M characters after parsing**92,322 emails** in one mbox, from which I extracted my ~10k sent messages — 3.1M characters of pure me-voice Combined pretraining corpus: **4.45M characters**. For scale, that's four times the Shakespeare dataset every tutorial uses, and a rounding error of a rounding error of what a frontier model trains on. One gotcha worth passing on: Takeout's activity exports default to HTML, and Google separates “Searched for” from your query with a *non-breaking space* that silently breaks naive parsers. Data cleaning starts before you have data. ## Pretraining: 5.07 → 1.60 I used Karpathy's nanoGPT: a character-level GPT — 6 layers, 6 heads, 384 dimensions, 10.77M parameters — trained on one task: *predict the next character*. No labels, no instructions. On an M-series MacBook Air, 8,000 training steps took about two hours. Loss is “average surprise per character.” 5.07 is random keyboard-mashing; 1.60 means the model usually knows what I'll type next. It plateaued around step 5,000 — a 10M-param model had extracted everything it could from 4.5M characters. What did it learn? My *shape*. Prompt it with nothing and it writes complete emails — greeting, scheduling proposal with my exact date format (“how about Friday (1/29) at 6p CT?”), “Let me know!”, “Cheers, Bernard.” It also exposed my past: I apparently once mail-merged a catch-up template to hundreds of people, because the model recites “I've launched a couple of startups (eSports coaching, real estate listings, bakery CRM)” verbatim, constantly. **Your most-repeated text becomes the model's strongest belief.** This is why real labs deduplicate training data with prejudice. ## SFT: teaching it to shut up A pretrained model is autocomplete, not a chatbot. Ask it “best name?” and it doesn't answer — it drifts into email boilerplate and starts writing to someone named Marcos. A question creates no obligation to answer; that's a social contract, and contracts aren't in the corpus. The fix is supervised fine-tuning, and the mechanics are almost anticlimactic: *same training loop, new data, lower learning rate*. The data is the clever part. Threading my mbox by `In-Reply-To` headers yielded **8,520 real (incoming message → my actual reply) pairs** — organic chat data most hobbyists have to fake: ``` <|them|> So many of these prompt tracking companies are raising money... <|bernard|> Ugh, I know. There must be a ton of money sloshing around... <|end|> ``` Six thousand gentle steps later, the same 10.77M parameters hold turns, match my register, and — critically — stop talking. The chat model answers “want to grab lunch this week?” with “Sounds good!” instead of a three-paragraph email. Post-training didn't add knowledge; **it changed which knowledge wins the race to the next token.** you> hey any interest in trying that new thai place saturday? bernard> Alright, I'm down. See you at 14:30am if you are in SF :) Perfectly me-shaped agreement, complete with a time that doesn't exist. That's what 10M parameters buys: form without comprehension. It also memorized real phone numbers from my mail — harmless on my laptop, but the reason you should never publish weights trained on private data. ## Cracking it open A model this small can be audited: I wrote probes that read the model's *exact* internal probabilities — no sampling, no noise; one forward pass returns its true credence for any continuation. First discovery: the model invented character classes from scratch. Its learned vector for `!` is nearly identical to its vector for the 😊 emoji — nobody told it they do the same job in my emails; it noticed. Then the real question: **what does the model believe about the people and brands in my life, and why?** I probed its confidence for names and tools, then counted actual mentions in the training data to see what drives belief. Three findings and one confession. ### Finding 1 — Belief scales with the log of exposure Model confidence completing “Content optimization: ___”, against training-data mentions. Every ~10× more exposure buys a roughly constant jump in belief. (Yes, I co-founded Clearscope — which is exactly why my email mentions it 1,977 times, and why my model worships it. Bias in, bias out, measurably.) ### Finding 2 — Context beats volume at the margins The cleanest demonstration came from weekdays. Friday is my second-most-mentioned day (995 mentions — deadlines, “by Friday”, “last Friday”). But I rarely *propose* Friday. And the model knows the difference: The Friday problem: second-most-mentioned day, fifth in the model's scheduling confidence — below Saturday, which has 42% of Friday's mentions. Being talked about is not the same as being recommended, and next-token models learn the difference. Even better: asked “best day?” — a phrasing that appears nowhere in my data — the model still answers Wednesday-first. Associations generalize to questions nobody ever asked. That's the mechanism behind AI answering questions about your brand that no page on the internet answers directly. ### Finding 3 — The last training run wins Fine-tuning re-weighted everything toward the reply-pair data. Entities frequent in my recent replies strengthened; everyone else decayed — one name fell from 10% to 1% confidence across 6,000 steps of simply never appearing. Model belief is not a trophy; it's a lease that expires at the next retrain. The Jennifer anomaly: fourth in mentions (41), first in belief (32%) — her mentions are 78% SFT-side (recent), sit inside reply blocks (the exact region the probe reads), and “Jennifer” is secretly four different people the model pooled into one string. Models track character sequences, not humans. ### Finding 4 — My own bug, or: why mention counts lie Confession that became the best lesson of the project. My first counts said “Ross” was the most-mentioned name in my corpus at 319. The dramatic chart I built on that — famous-but-never-greeted Ross — was wrong. A case-insensitive substring grep had counted every “ac**ross**”, “g** ross**” and “c** ross**ed” as a Ross-mention. His real count: 10. One word-boundary fix rewrote a finding I'd already internalized. If a two-person audit of a 4.5M-character corpus can be fooled this easily, be appropriately skeptical of any dashboard selling you “entity mention rates” over the open web — the counting pipeline is where the lies get in, and nobody shows you their grep. ## The playbook: what actually moves an entity's weight in an AI model The ELI5 is that a language model is a parrot that learned the world from flashcards of “what text follows what,” and an entity's mention rate only matters through what it does to those flashcards: **Volume works in powers of ten.** Belief tracks log(mentions), r ≈ 0.9. Ten more mentions is noise; ten*times*more mentions is a new belief.**The answer slot beats the article.**“Best X: you” constructions are flashcards; passing mentions aren't. Friday: famous, never proposed, not believed in.** Recency compounds.**The last training run redistributes all the probability. AI visibility is a lease, not a trophy.** Own a unique name.**Generic names pool with strangers and dilute; distinctive names concentrate every mention into one belief.** Associations generalize.**Build one strong context and the model extends it to questions nobody ever asked. You're not optimizing keywords; you're installing a prior. One-line version:AI models don't know who's famous — they know who reliably appears in the answer slot, recently, at 10× scale, under one unmistakable name. ### The receipt - Training data: 4.45M chars (mine) · Model: 10.77M params · Hardware: MacBook Air - Pretrain + SFT wall-clock: ~4 hours · Cost: $0 - Distance to ChatGPT: RLHF + ~6 orders of magnitude Every step is reproducible with nanoGPT, a Google Takeout export, and a weekend. The model is useless as an assistant and priceless as an X-ray: every mystified claim about what AI “knows” became, at this scale, a number I could measure and a grep I could check. I recommend building your own ghost. Just don't publish the weights — it knows your phone number. Newsletter ## Get the next post by email. One email when I publish something new. No spam, no fixed schedule, unsubscribe anytime.