I Trained a GPT on 15 Years of My Life — Then Interrogated It

A developer trained a 10.77M-parameter GPT on 4.45M characters of his own Google data from 15 years, achieving a loss of 1.60 after pretraining and then fine-tuning it on his email replies to create a chatbot. The project revealed that the model's beliefs track the log of training-data mentions and that the last training run re-weights entities toward recent data, demonstrating how AI knowledge is shaped by data volume and recency.

I Trained a GPT on 15 Years of My Life — Then Interrogated It Everyone explains LLMs with someone else's diagrams. I wanted to hold one — small enough to audit, trained on data I understand better than anyone: my own. So over a couple of days I ran the entire modern LLM pipeline — pretraining, supervised fine-tuning, chat interface, interpretability probes — on a laptop, using my Google Takeout as the corpus. I built a 10.77M-parameter GPT from scratch on 4.45M characters of my own Google data — pretraining, SFT, and interpretability probes, for $0, in ~4 hours on a MacBook Air. - Model belief tracks the log of training-data mentions r = 0.95 Context beats volume : my second-most-mentioned weekday is one the model won't recommend The last training run wins : fine-tuning re-weighted every entity toward recent data- My own counting bug “ac ross ” ≠ Ross shows why you should audit every mention dashboard The result is a tiny ghost of me that schedules coffee at “14:30am,” believes Wednesday is the best day of the week, and taught me more about AI “knowledge” than a year of reading papers. The corpus: me Google Takeout will hand you everything the company knows about you. Mine was 133 GB, most of it photos and video. The text — the part a language model can eat — was tiny by comparison: 67,792 search queries across 15 years Search, Maps, YouTube, Flights, Gmail search — 1.35M characters after parsing 92,322 emails in one mbox, from which I extracted my ~10k sent messages — 3.1M characters of pure me-voice Combined pretraining corpus: 4.45M characters . For scale, that's four times the Shakespeare dataset every tutorial uses, and a rounding error of a rounding error of what a frontier model trains on. One gotcha worth passing on: Takeout's activity exports default to HTML, and Google separates “Searched for” from your query with a non-breaking space that silently breaks naive parsers. Data cleaning starts before you have data. Pretraining: 5.07 → 1.60 I used Karpathy's nanoGPT: a character-level GPT — 6 layers, 6 heads, 384 dimensions, 10.77M parameters — trained on one task: predict the next character . No labels, no instructions. On an M-series MacBook Air, 8,000 training steps took about two hours. Loss is “average surprise per character.” 5.07 is random keyboard-mashing; 1.60 means the model usually knows what I'll type next. It plateaued around step 5,000 — a 10M-param model had extracted everything it could from 4.5M characters. What did it learn? My shape . Prompt it with nothing and it writes complete emails — greeting, scheduling proposal with my exact date format “how about Friday 1/29 at 6p CT?” , “Let me know ”, “Cheers, Bernard.” It also exposed my past: I apparently once mail-merged a catch-up template to hundreds of people, because the model recites “I've launched a couple of startups eSports coaching, real estate listings, bakery CRM ” verbatim, constantly. Your most-repeated text becomes the model's strongest belief. This is why real labs deduplicate training data with prejudice. SFT: teaching it to shut up A pretrained model is autocomplete, not a chatbot. Ask it “best name?” and it doesn't answer — it drifts into email boilerplate and starts writing to someone named Marcos. A question creates no obligation to answer; that's a social contract, and contracts aren't in the corpus. The fix is supervised fine-tuning, and the mechanics are almost anticlimactic: same training loop, new data, lower learning rate . The data is the clever part. Threading my mbox by In-Reply-To headers yielded 8,520 real incoming message → my actual reply pairs — organic chat data most hobbyists have to fake: <|them| So many of these prompt tracking companies are raising money... <|bernard| Ugh, I know. There must be a ton of money sloshing around... <|end| Six thousand gentle steps later, the same 10.77M parameters hold turns, match my register, and — critically — stop talking. The chat model answers “want to grab lunch this week?” with “Sounds good ” instead of a three-paragraph email. Post-training didn't add knowledge; it changed which knowledge wins the race to the next token. you hey any interest in trying that new thai place saturday? bernard Alright, I'm down. See you at 14:30am if you are in SF : Perfectly me-shaped agreement, complete with a time that doesn't exist. That's what 10M parameters buys: form without comprehension. It also memorized real phone numbers from my mail — harmless on my laptop, but the reason you should never publish weights trained on private data. Cracking it open A model this small can be audited: I wrote probes that read the model's exact internal probabilities — no sampling, no noise; one forward pass returns its true credence for any continuation. First discovery: the model invented character classes from scratch. Its learned vector for is nearly identical to its vector for the 😊 emoji — nobody told it they do the same job in my emails; it noticed. Then the real question: what does the model believe about the people and brands in my life, and why? I probed its confidence for names and tools, then counted actual mentions in the training data to see what drives belief. Three findings and one confession. Finding 1 — Belief scales with the log of exposure Model confidence completing “Content optimization: ”, against training-data mentions. Every ~10× more exposure buys a roughly constant jump in belief. Yes, I co-founded Clearscope — which is exactly why my email mentions it 1,977 times, and why my model worships it. Bias in, bias out, measurably. Finding 2 — Context beats volume at the margins The cleanest demonstration came from weekdays. Friday is my second-most-mentioned day 995 mentions — deadlines, “by Friday”, “last Friday” . But I rarely propose Friday. And the model knows the difference: The Friday problem: second-most-mentioned day, fifth in the model's scheduling confidence — below Saturday, which has 42% of Friday's mentions. Being talked about is not the same as being recommended, and next-token models learn the difference. Even better: asked “best day?” — a phrasing that appears nowhere in my data — the model still answers Wednesday-first. Associations generalize to questions nobody ever asked. That's the mechanism behind AI answering questions about your brand that no page on the internet answers directly. Finding 3 — The last training run wins Fine-tuning re-weighted everything toward the reply-pair data. Entities frequent in my recent replies strengthened; everyone else decayed — one name fell from 10% to 1% confidence across 6,000 steps of simply never appearing. Model belief is not a trophy; it's a lease that expires at the next retrain. The Jennifer anomaly: fourth in mentions 41 , first in belief 32% — her mentions are 78% SFT-side recent , sit inside reply blocks the exact region the probe reads , and “Jennifer” is secretly four different people the model pooled into one string. Models track character sequences, not humans. Finding 4 — My own bug, or: why mention counts lie Confession that became the best lesson of the project. My first counts said “Ross” was the most-mentioned name in my corpus at 319. The dramatic chart I built on that — famous-but-never-greeted Ross — was wrong. A case-insensitive substring grep had counted every “ac ross ”, “g ross ” and “c ross ed” as a Ross-mention. His real count: 10. One word-boundary fix rewrote a finding I'd already internalized. If a two-person audit of a 4.5M-character corpus can be fooled this easily, be appropriately skeptical of any dashboard selling you “entity mention rates” over the open web — the counting pipeline is where the lies get in, and nobody shows you their grep. The playbook: what actually moves an entity's weight in an AI model The ELI5 is that a language model is a parrot that learned the world from flashcards of “what text follows what,” and an entity's mention rate only matters through what it does to those flashcards: Volume works in powers of ten. Belief tracks log mentions , r ≈ 0.9. Ten more mentions is noise; ten times more mentions is a new belief. The answer slot beats the article. “Best X: you” constructions are flashcards; passing mentions aren't. Friday: famous, never proposed, not believed in. Recency compounds. The last training run redistributes all the probability. AI visibility is a lease, not a trophy. Own a unique name. Generic names pool with strangers and dilute; distinctive names concentrate every mention into one belief. Associations generalize. Build one strong context and the model extends it to questions nobody ever asked. You're not optimizing keywords; you're installing a prior. One-line version:AI models don't know who's famous — they know who reliably appears in the answer slot, recently, at 10× scale, under one unmistakable name. The receipt - Training data: 4.45M chars mine · Model: 10.77M params · Hardware: MacBook Air - Pretrain + SFT wall-clock: ~4 hours · Cost: $0 - Distance to ChatGPT: RLHF + ~6 orders of magnitude Every step is reproducible with nanoGPT, a Google Takeout export, and a weekend. The model is useless as an assistant and priceless as an X-ray: every mystified claim about what AI “knows” became, at this scale, a number I could measure and a grep I could check. I recommend building your own ghost. Just don't publish the weights — it knows your phone number. Newsletter Get the next post by email. One email when I publish something new. No spam, no fixed schedule, unsubscribe anytime.