cd /news/artificial-intelligence/i-built-a-tiny-japanese-gpt-from-scr… · home › topics › artificial-intelligence › article
[ARTICLE · art-173220] src=dev.to ↗ pub= topic=artificial-intelligence verified=true sentiment=· neutral

I Built a Tiny Japanese GPT from Scratch in PyTorch and Tried a Toy 'RSI' Loop: Here's What It Does and Doesn't Show

A second-year online student at ZEN University built a roughly 0.9M-parameter character-level Japanese GPT from scratch in basic PyTorch, then ran a toy best-of-N self-training loop modeled on STaR/ReST. The model was pretrained on about 1,678 characters of Japanese literature and fine-tuned on 27 dialogue pairs, with loss falling from 6.11 to 0.07 and 5.58 to 0.04 respectively. The developer cautions that the tiny educational model's self-grading scores should not be read as evidence of genuine self-improvement.

recommendation

When running self-training loops, treat self-assigned scores as format and confidence signals rather than evidence of improvement, and validate gains against held-out data.

more in playbook: Fine-tuning and data
by read4 min views6 publishedNov 20, 2026

I'm a second-year student at ZEN University studying intelligent informatics (online), and I build AI agents as a personal project. I have no work experience in machine learning.

I use LLMs through APIs and agents every day, but I wanted to see what happens inside one with my own hands. So I wrote a small Japanese GPT using only basic PyTorch operations: no pretrained weights, no high-level model libraries. Then I tried a toy version of "RSI" (recursive self-improvement): the model generates several answers, grades them itself, and retrains on the best ones.

Up front: this is a tiny educational model trained on a few KB of text. Don't expect it to be useful, and please don't read the RSI numbers as "the model got smarter." I'll explain why below. This is not AGI or anything close to it.

A character-level, decoder-only Transformer with about 0.9M parameters (the pretraining version has 898,425, which I also checked by hand from the layer sizes). It has three stages:

mini_gpt.py: character tokenizer + Transformer, pretrained on short excerpts of modern Japanese literature to continue text.chat_gpt.py: the same architecture, fine-tuned on ユーザー: ...\nAI: ...◁ dialogues so it behaves like a chatbot.rsi_gpt.py: a best-of-N self-training loop. There is also a local web UI (web_ui.py, standard-library http.server only) that shows the top-5 probabilities for the first generated character and lets you edit the dialogue data and retrain in the browser.

Hyperparameters in mini_gpt.py:

BLOCK_SIZE = 64       # max context length
N_EMBD = 128          # embedding dimension
N_HEAD = 4            # attention heads
N_LAYER = 4           # Transformer blocks

Causal self-attention masks out future characters with a lower-triangular mask before the softmax:

att = (q @ k.transpose(-2, -1)) * (1.0 / math.sqrt(head_dim))
att = att.masked_fill(self.mask[:, :, :T, :T] == 0, float("-inf"))
att = F.softmax(att, dim=-1)

Each block is Pre-LayerNorm with two residual connections:

def forward(self, x):
    x = x + self.attn(self.ln1(x))  # residual + attention
    x = x + self.mlp(self.ln2(x))   # residual + feed-forward
    return x

The tokenizer just numbers the characters that appear in the training text:

chars = sorted(list(set(text)))
stoi = {ch: i for i, ch in enumerate(chars)}

I didn't change the architecture. Two things changed:

"ユーザー: {question}\nAI: ".◁, and generation stops when it is sampled.

if token_id == stop_token_id:
    break  # stop when the end marker appears

Without the stop character the model keeps writing the next ユーザー: ... turn by itself. I call that role leakage below.

This is a small-scale best-of-N rejection sampling loop, in the spirit of STaR / ReST:

The score is the sum of four parts (evaluate_candidate):

Part Points Meaning
Completion 25 ended naturally with ◁
Format 25 no "ユーザー"/"AI:" leakage, no newline, no 4+ repeated characters, at least 12 characters
Relevance up to 30 contains keywords I hand-picked per prompt (15 points each)
Self-confidence up to 20 the model's own average probability of the characters it sampled, times 20
matched_kw = [kw for kw in prompt_info["keywords"] if kw in reply]
kw_score = min(30.0, len(matched_kw) * 15.0)
...
conf_score = round(candidate["avg_confidence"] * 20.0, 1)

The exploration temperature is lowered each generation: max(0.45, 0.85 - (gen - 1) * 0.12), i.e. 0.85, 0.73, 0.61.

Sources: PAPER.md and rsi_history.json. I recomputed the RSI table from rsi_history.json.

Pretraining (1,678 characters of Japanese literature excerpts, 377 distinct characters, 800 steps): loss went from 6.1098 at step 1 to 0.0683 at step 800. Prompted with "吾輩は", it produces the opening of I Am a Cat.

Dialogue fine-tuning (27 Q&A pairs, 1,575 characters, 1,000 steps): loss went from 5.5833 at step 1 to 0.0387 at step 1000.

RSI (7 prompts, N=6, threshold 75, 350 steps per generation):

Generation Temperature Avg best score Avg worst score Prompts with worst < 75 Loss after retraining
Gen 1 0.85 97.56 80.77 4/7 0.0635
Gen 2 0.73 97.84 97.80 0/7 0.0490
Gen 3 0.61 97.84 97.80 0/7 0.0639

Gen 1 failures looked like this:

The average worst-case score went from 80.77 to 97.80, and prompts with a failing candidate went from 4/7 to 0/7. But please don't read this as the model getting smarter. Here is why.

chat_data.txt). Gen 2 and Gen 3 are being asked questions whose answers they saw in training. Higher scores are expected and say nothing about unseen questions. So what actually happened is: a tiny model picked its best-scoring answers to 7 questions and re-learned them.

pip install -r requirements.txt   # torch>=2.0.0

python mini_gpt.py                # pretrain and generate
python chat_gpt.py                # train on dialogues (saves chat_model.pt)
python chat_gpt.py --chat         # chat with the saved model
python web_ui.py                  # local web UI (http://localhost:8765)
python rsi_gpt.py                 # run 3 RSI generations

Note that rsi_gpt.py rewrites chat_data.txt, so back it up first. If you only want to poke at it, the demo Space below is easier.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @zen university 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/i-built-a-tiny-japan…] indexed:0 read:4min 2026-11-20 · —