I Built a Tiny Japanese GPT from Scratch in PyTorch and Tried a Toy 'RSI' Loop: Here's What It Does and Doesn't Show A second-year online student at ZEN University built a roughly 0.9M-parameter character-level Japanese GPT from scratch in basic PyTorch, then ran a toy best-of-N self-training loop modeled on STaR/ReST. The model was pretrained on about 1,678 characters of Japanese literature and fine-tuned on 27 dialogue pairs, with loss falling from 6.11 to 0.07 and 5.58 to 0.04 respectively. The developer cautions that the tiny educational model's self-grading scores should not be read as evidence of genuine self-improvement. I'm a second-year student at ZEN University studying intelligent informatics online , and I build AI agents as a personal project. I have no work experience in machine learning. I use LLMs through APIs and agents every day, but I wanted to see what happens inside one with my own hands. So I wrote a small Japanese GPT using only basic PyTorch operations: no pretrained weights, no high-level model libraries. Then I tried a toy version of "RSI" recursive self-improvement : the model generates several answers, grades them itself, and retrains on the best ones. Up front: this is a tiny educational model trained on a few KB of text . Don't expect it to be useful, and please don't read the RSI numbers as "the model got smarter." I'll explain why below. This is not AGI or anything close to it. A character-level, decoder-only Transformer with about 0.9M parameters the pretraining version has 898,425, which I also checked by hand from the layer sizes . It has three stages: mini gpt.py : character tokenizer + Transformer, pretrained on short excerpts of modern Japanese literature to continue text. chat gpt.py : the same architecture, fine-tuned on ユーザー: ...\nAI: ...◁ dialogues so it behaves like a chatbot. rsi gpt.py : a best-of-N self-training loop. There is also a local web UI web ui.py , standard-library http.server only that shows the top-5 probabilities for the first generated character and lets you edit the dialogue data and retrain in the browser. Hyperparameters in mini gpt.py : BLOCK SIZE = 64 max context length N EMBD = 128 embedding dimension N HEAD = 4 attention heads N LAYER = 4 Transformer blocks Causal self-attention masks out future characters with a lower-triangular mask before the softmax: att = q @ k.transpose -2, -1 1.0 / math.sqrt head dim att = att.masked fill self.mask :, :, :T, :T == 0, float "-inf" att = F.softmax att, dim=-1 Each block is Pre-LayerNorm with two residual connections: python def forward self, x : x = x + self.attn self.ln1 x residual + attention x = x + self.mlp self.ln2 x residual + feed-forward return x The tokenizer just numbers the characters that appear in the training text: chars = sorted list set text stoi = {ch: i for i, ch in enumerate chars } I didn't change the architecture. Two things changed: "ユーザー: {question}\nAI: " . ◁ , and generation stops when it is sampled. if token id == stop token id: break stop when the end marker appears Without the stop character the model keeps writing the next ユーザー: ... turn by itself. I call that role leakage below. This is a small-scale best-of-N rejection sampling loop, in the spirit of STaR / ReST: The score is the sum of four parts evaluate candidate : | Part | Points | Meaning | |---|---|---| | Completion | 25 | ended naturally with ◁ | | Format | 25 | no "ユーザー"/"AI:" leakage, no newline, no 4+ repeated characters, at least 12 characters | | Relevance | up to 30 | contains keywords I hand-picked per prompt 15 points each | | Self-confidence | up to 20 | the model's own average probability of the characters it sampled, times 20 | matched kw = kw for kw in prompt info "keywords" if kw in reply kw score = min 30.0, len matched kw 15.0 ... conf score = round candidate "avg confidence" 20.0, 1 The exploration temperature is lowered each generation: max 0.45, 0.85 - gen - 1 0.12 , i.e. 0.85, 0.73, 0.61. Sources: PAPER.md and rsi history.json . I recomputed the RSI table from rsi history.json . Pretraining 1,678 characters of Japanese literature excerpts, 377 distinct characters, 800 steps : loss went from 6.1098 at step 1 to 0.0683 at step 800. Prompted with "吾輩は", it produces the opening of I Am a Cat . Dialogue fine-tuning 27 Q&A pairs, 1,575 characters, 1,000 steps : loss went from 5.5833 at step 1 to 0.0387 at step 1000. RSI 7 prompts, N=6, threshold 75, 350 steps per generation : | Generation | Temperature | Avg best score | Avg worst score | Prompts with worst < 75 | Loss after retraining | |---|---|---|---|---|---| | Gen 1 | 0.85 | 97.56 | 80.77 | 4/7 | 0.0635 | | Gen 2 | 0.73 | 97.84 | 97.80 | 0/7 | 0.0490 | | Gen 3 | 0.61 | 97.84 | 97.80 | 0/7 | 0.0639 | Gen 1 failures looked like this: The average worst-case score went from 80.77 to 97.80, and prompts with a failing candidate went from 4/7 to 0/7. But please don't read this as the model getting smarter. Here is why. chat data.txt . Gen 2 and Gen 3 are being asked questions whose answers they saw in training. Higher scores are expected and say nothing about unseen questions. So what actually happened is: a tiny model picked its best-scoring answers to 7 questions and re-learned them. pip install -r requirements.txt torch =2.0.0 python mini gpt.py pretrain and generate python chat gpt.py train on dialogues saves chat model.pt python chat gpt.py --chat chat with the saved model python web ui.py local web UI http://localhost:8765 python rsi gpt.py run 3 RSI generations Note that rsi gpt.py rewrites chat data.txt , so back it up first. If you only want to poke at it, the demo Space below is easier.