cd /news/machine-learning/i-trained-a-small-language-model-in-… · home › topics › machine-learning › article
[ARTICLE · art-170860] src=dev.to ↗ pub= topic=machine-learning verified=true sentiment=↑ positive

I trained a small language model in TypeScript. Here is the path that worked.

A developer used Affon 0.4.2, a TypeScript ML framework, to define, train, evaluate, and checkpoint a small six-layer decoder language model entirely from a TypeScript application. Running on macOS arm64 with Metal, a debug configuration of 32 training batches and eight validation batches moved validation loss from 10.832 to 9.833 and saved a checkpoint, though the generated sample was incoherent. The developer framed the run as proof that the training path executes, not that the model is useful, and highlighted Affon's explicit 'Program' abstraction for inspecting operations and tensor shapes before execution.

by read1 min views12 publishedNov 16, 2026

TypeScript ML examples often stop at calling an existing model. I wanted to check the other path: define a decoder, train it on text, evaluate it, and save its state, all from a TypeScript application.

I used Affon's decoder app. It defines embeddings, causal attention, and six decoder layers. The bundled WikiText-2 text becomes 256-token windows. The training workflow evaluates batches, checks validation loss, and writes a checkpoint.

With Affon 0.4.2 installed on macOS arm64, this ran on Metal from a source checkout:

git clone https://github.com/neucine/affon.git
cd affon
RUNTIME_PACKAGE_PATH="$PWD/packages" \
  AFFON_TRAIN_CONFIG=apps/decoder-lm/configs/train-decoder-lm-wikitext-debug.config.json \
  affon run apps/decoder-lm/train.ts

The debug configuration stops after 32 training batches and evaluates eight validation batches. In my run, validation loss moved from 10.832 to 9.833 and the app saved clean-debug-run-epoch-1. The short generated sample was incoherent. That is expected from this check; these numbers show that the training path executes, not that the model is useful or competitive.

Affon calls the computation a Program. The decoder Program names its input shape and parameters, including the token embedding table and the layers that use it. A training Program is derived from the model, a cross-entropy loss, and an optimizer. The app compiles those Programs for a Session and keeps model state explicit.

That structure gives me something to inspect before execution: which operations and tensor shapes the model asks for, and where parameters belong in the nested decoder. It also lets the same model definition serve training and inference. The model source and workflow show the full path.

The complete walkthrough covers installation, output files, and the longer training configuration. If you work on TypeScript ML, I would value feedback on the Program boundary: what would you need to inspect before trusting a larger training run?

── more in #machine-learning 4 stories · sorted by recency
── more on @affon 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/i-trained-a-small-la…] indexed:0 read:1min 2026-11-16 · —