# I trained a small language model in TypeScript. Here is the path that worked.

> Source: <https://dev.to/chaoyangnz/i-trained-a-small-language-model-in-typescript-here-is-the-path-that-worked-2j5g>
> Published: 2026-11-16 19:46:37+00:00

TypeScript ML examples often stop at calling an existing model. I wanted to check the other path: define a decoder, train it on text, evaluate it, and save its state, all from a TypeScript application.

I used [Affon's decoder app](https://github.com/neucine/affon/tree/main/apps/decoder-lm). It defines embeddings, causal attention, and six decoder layers. The bundled WikiText-2 text becomes 256-token windows. The training workflow evaluates batches, checks validation loss, and writes a checkpoint.

With Affon 0.4.2 installed on macOS arm64, this ran on Metal from a source checkout:

```
git clone https://github.com/neucine/affon.git
cd affon
RUNTIME_PACKAGE_PATH="$PWD/packages" \
  AFFON_TRAIN_CONFIG=apps/decoder-lm/configs/train-decoder-lm-wikitext-debug.config.json \
  affon run apps/decoder-lm/train.ts
```

The debug configuration stops after 32 training batches and evaluates eight validation batches. In my run, validation loss moved from **10.832 to 9.833** and the app saved `clean-debug-run-epoch-1`. The short generated sample was incoherent. That is expected from this check; these numbers show that the training path executes, not that the model is useful or competitive.

Affon calls the computation a **Program**. The decoder Program names its input shape and parameters, including the token embedding table and the layers that use it. A training Program is derived from the model, a cross-entropy loss, and an optimizer. The app compiles those Programs for a Session and keeps model state explicit.

That structure gives me something to inspect before execution: which operations and tensor shapes the model asks for, and where parameters belong in the nested decoder. It also lets the same model definition serve training and inference. The [model source](https://github.com/neucine/affon/blob/main/apps/decoder-lm/src/model.ts) and [workflow](https://github.com/neucine/affon/blob/main/apps/decoder-lm/src/workflow.ts) show the full path.

The [complete walkthrough](https://affon.ai/docs/getting-started/train-language-model/) covers installation, output files, and the longer training configuration. If you work on TypeScript ML, I would value feedback on the Program boundary: what would you need to inspect before trusting a larger training run?
