{"slug": "toy-demo-fine-tuning-a-3-2m-param-char-transformer-makes-it-forget-shakespeare", "title": "Toy demo: fine-tuning a 3.2M-param char transformer makes it forget Shakespeare (held-out numbers)", "summary": "A 3.2M-parameter character-level transformer pre-trained on Tiny Shakespeare to a validation loss of 1.50 forgot its original training after 1,000 fine-tuning steps on Mrs Beeton's Book of Household Management (1861), with the prompt \"ROMEO:\" returning \"377. INGREDIENTS.–4 oz. of salt\" on text held out from both corpora. The author also froze the fine-tuned model, hashed its weights with SHA-256, prompted it two ways and re-hashed, finding different outputs but an identical hash. The experiment, presented as a toy demonstration of catastrophic forgetting, was used in a 5-minute explainer on training versus fine-tuning versus prompting.", "body_md": "I pre-trained a 3.2M-parameter character-level transformer on Tiny Shakespeare (validation loss 1.50), then fine-tuned it for 1,000 steps on Mrs Beeton’s Book of Household Management (1861).\n\nResults on text held out from both:\n\nAfter fine-tuning, the prompt “ROMEO:” yields “377. INGREDIENTS.–4 oz. of salt”. A small, concrete example of forgetting.\n\nI also froze the fine-tuned model, hashed its weights (SHA-256), prompted it two ways and hashed again: different outputs, identical hash.\n\nI used both in a 5-minute explainer on training vs fine-tuning vs prompting for a general audience: [https://www.youtube.com/watch?v=hwl4Ve0NBW8](https://www.youtube.com/watch?v=hwl4Ve0NBW8)\n\nIt is a toy model and its output is nonsense with the right shape. Happy to answer questions about the setup. What would you use to demonstrate forgetting at this scale?", "url": "https://wpnews.pro/news/toy-demo-fine-tuning-a-3-2m-param-char-transformer-makes-it-forget-shakespeare", "canonical_source": "https://discuss.huggingface.co/t/toy-demo-fine-tuning-a-3-2m-param-char-transformer-makes-it-forget-shakespeare-held-out-numbers/191706#post_1", "published_at": "2026-11-10 18:02:07+00:00", "updated_at": "2026-11-10 18:19:54.954281+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-research", "neural-networks"], "entities": ["Tiny Shakespeare", "Mrs Beeton's Book of Household Management"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/toy-demo-fine-tuning-a-3-2m-param-char-transformer-makes-it-forget-shakespeare", "markdown": "https://wpnews.pro/news/toy-demo-fine-tuning-a-3-2m-param-char-transformer-makes-it-forget-shakespeare.md", "text": "https://wpnews.pro/news/toy-demo-fine-tuning-a-3-2m-param-char-transformer-makes-it-forget-shakespeare.txt", "jsonld": "https://wpnews.pro/news/toy-demo-fine-tuning-a-3-2m-param-char-transformer-makes-it-forget-shakespeare.jsonld"}}