{"slug": "i-built-a-tiny-japanese-gpt-from-scratch-in-pytorch-and-tried-a-toy-rsi-loop-s-t", "title": "I Built a Tiny Japanese GPT from Scratch in PyTorch and Tried a Toy 'RSI' Loop: Here's What It Does and Doesn't Show", "summary": "A second-year online student at ZEN University built a roughly 0.9M-parameter character-level Japanese GPT from scratch in basic PyTorch, then ran a toy best-of-N self-training loop modeled on STaR/ReST. The model was pretrained on about 1,678 characters of Japanese literature and fine-tuned on 27 dialogue pairs, with loss falling from 6.11 to 0.07 and 5.58 to 0.04 respectively. The developer cautions that the tiny educational model's self-grading scores should not be read as evidence of genuine self-improvement.", "body_md": "I'm a second-year student at ZEN University studying intelligent informatics (online), and I build AI agents as a personal project. I have no work experience in machine learning.\n\nI use LLMs through APIs and agents every day, but I wanted to see what happens inside one with my own hands. So I wrote a small Japanese GPT using only basic PyTorch operations: no pretrained weights, no high-level model libraries. Then I tried a toy version of \"RSI\" (recursive self-improvement): the model generates several answers, grades them itself, and retrains on the best ones.\n\nUp front: this is a **tiny educational model trained on a few KB of text**. Don't expect it to be useful, and please don't read the RSI numbers as \"the model got smarter.\" I'll explain why below. This is not AGI or anything close to it.\n\nA character-level, decoder-only Transformer with about 0.9M parameters (the pretraining version has 898,425, which I also checked by hand from the layer sizes). It has three stages:\n\n`mini_gpt.py`: character tokenizer + Transformer, pretrained on short excerpts of modern Japanese literature to continue text.`chat_gpt.py`: the same architecture, fine-tuned on `ユーザー: ...\\nAI: ...◁` dialogues so it behaves like a chatbot.`rsi_gpt.py`: a best-of-N self-training loop.\nThere is also a local web UI (`web_ui.py`, standard-library `http.server` only) that shows the top-5 probabilities for the first generated character and lets you edit the dialogue data and retrain in the browser.\n\nHyperparameters in `mini_gpt.py`:\n\n```\nBLOCK_SIZE = 64       # max context length\nN_EMBD = 128          # embedding dimension\nN_HEAD = 4            # attention heads\nN_LAYER = 4           # Transformer blocks\n```\n\nCausal self-attention masks out future characters with a lower-triangular mask before the softmax:\n\n```\natt = (q @ k.transpose(-2, -1)) * (1.0 / math.sqrt(head_dim))\natt = att.masked_fill(self.mask[:, :, :T, :T] == 0, float(\"-inf\"))\natt = F.softmax(att, dim=-1)\n```\n\nEach block is Pre-LayerNorm with two residual connections:\n\n``` python\ndef forward(self, x):\n    x = x + self.attn(self.ln1(x))  # residual + attention\n    x = x + self.mlp(self.ln2(x))   # residual + feed-forward\n    return x\n```\n\nThe tokenizer just numbers the characters that appear in the training text:\n\n```\nchars = sorted(list(set(text)))\nstoi = {ch: i for i, ch in enumerate(chars)}\n```\n\nI didn't change the architecture. Two things changed:\n\n`\"ユーザー: {question}\\nAI: \"`.`◁`, and generation stops when it is sampled.\n\n```\nif token_id == stop_token_id:\n    break  # stop when the end marker appears\n```\n\nWithout the stop character the model keeps writing the next `ユーザー: ...` turn by itself. I call that role leakage below.\n\nThis is a small-scale best-of-N rejection sampling loop, in the spirit of STaR / ReST:\n\nThe score is the sum of four parts (`evaluate_candidate`):\n\n| Part | Points | Meaning | \n|---|---|---|\n| Completion | 25 | ended naturally with `◁` | \n| Format | 25 | no \"ユーザー\"/\"AI:\" leakage, no newline, no 4+ repeated characters, at least 12 characters | \n| Relevance | up to 30 | contains keywords I hand-picked per prompt (15 points each) | \n| Self-confidence | up to 20 | the model's own average probability of the characters it sampled, times 20 | \n\n```\nmatched_kw = [kw for kw in prompt_info[\"keywords\"] if kw in reply]\nkw_score = min(30.0, len(matched_kw) * 15.0)\n...\nconf_score = round(candidate[\"avg_confidence\"] * 20.0, 1)\n```\n\nThe exploration temperature is lowered each generation: `max(0.45, 0.85 - (gen - 1) * 0.12)`, i.e. 0.85, 0.73, 0.61.\n\nSources: `PAPER.md` and `rsi_history.json`. I recomputed the RSI table from `rsi_history.json`.\n\n**Pretraining** (1,678 characters of Japanese literature excerpts, 377 distinct characters, 800 steps): loss went from 6.1098 at step 1 to 0.0683 at step 800. Prompted with \"吾輩は\", it produces the opening of *I Am a Cat*.\n\n**Dialogue fine-tuning** (27 Q&A pairs, 1,575 characters, 1,000 steps): loss went from 5.5833 at step 1 to 0.0387 at step 1000.\n\n**RSI** (7 prompts, N=6, threshold 75, 350 steps per generation):\n\n| Generation | Temperature | Avg best score | Avg worst score | Prompts with worst < 75 | Loss after retraining | \n|---|---|---|---|---|---|\n| Gen 1 | 0.85 | 97.56 | 80.77 | 4/7 | 0.0635 | \n| Gen 2 | 0.73 | 97.84 | 97.80 | 0/7 | 0.0490 | \n| Gen 3 | 0.61 | 97.84 | 97.80 | 0/7 | 0.0639 | \n\nGen 1 failures looked like this:\n\nThe average worst-case score went from 80.77 to 97.80, and prompts with a failing candidate went from 4/7 to 0/7. But please don't read this as the model getting smarter. Here is why.\n\n`chat_data.txt`). Gen 2 and Gen 3 are being asked questions whose answers they saw in training. Higher scores are expected and say nothing about unseen questions.\nSo what actually happened is: a tiny model picked its best-scoring answers to 7 questions and re-learned them.\n\n```\npip install -r requirements.txt   # torch>=2.0.0\n\npython mini_gpt.py                # pretrain and generate\npython chat_gpt.py                # train on dialogues (saves chat_model.pt)\npython chat_gpt.py --chat         # chat with the saved model\npython web_ui.py                  # local web UI (http://localhost:8765)\npython rsi_gpt.py                 # run 3 RSI generations\n```\n\nNote that `rsi_gpt.py` rewrites `chat_data.txt`, so back it up first. If you only want to poke at it, the demo Space below is easier.", "url": "https://wpnews.pro/news/i-built-a-tiny-japanese-gpt-from-scratch-in-pytorch-and-tried-a-toy-rsi-loop-s-t", "canonical_source": "https://dev.to/koheip/i-built-a-tiny-japanese-gpt-from-scratch-in-pytorch-and-tried-a-toy-rsi-loop-heres-what-it-does-415j", "published_at": "2026-11-20 12:34:26+00:00", "updated_at": "2026-11-20 12:42:32.727963+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research", "generative-ai"], "entities": ["ZEN University", "PyTorch", "STaR", "ReST"], "also_reported_by": [], "takeaway": "A ~0.9M-parameter character-level Transformer can be built from raw PyTorch ops and fine-tuned into a chatbot by adding a stop token and role-formatted dialogue, but a best-of-N self-grading loop on a few KB of text measures format compliance, not real capability gains.", "recommendation": {"advice": "When running self-training loops, treat self-assigned scores as format and confidence signals rather than evidence of improvement, and validate gains against held-out data.", "playbook": "training-data", "playbook_url": "https://wpnews.pro/playbooks/training-data"}, "alternates": {"html": "https://wpnews.pro/news/i-built-a-tiny-japanese-gpt-from-scratch-in-pytorch-and-tried-a-toy-rsi-loop-s-t", "markdown": "https://wpnews.pro/news/i-built-a-tiny-japanese-gpt-from-scratch-in-pytorch-and-tried-a-toy-rsi-loop-s-t.md", "text": "https://wpnews.pro/news/i-built-a-tiny-japanese-gpt-from-scratch-in-pytorch-and-tried-a-toy-rsi-loop-s-t.txt", "jsonld": "https://wpnews.pro/news/i-built-a-tiny-japanese-gpt-from-scratch-in-pytorch-and-tried-a-toy-rsi-loop-s-t.jsonld"}}