TypeScript ML examples often stop at calling an existing model. I wanted to check the other path: define a decoder, train it on text, evaluate it, and save its state, all from a TypeScript application.
I used Affon's decoder app. It defines embeddings, causal attention, and six decoder layers. The bundled WikiText-2 text becomes 256-token windows. The training workflow evaluates batches, checks validation loss, and writes a checkpoint.
With Affon 0.4.2 installed on macOS arm64, this ran on Metal from a source checkout:
git clone https://github.com/neucine/affon.git
cd affon
RUNTIME_PACKAGE_PATH="$PWD/packages" \
AFFON_TRAIN_CONFIG=apps/decoder-lm/configs/train-decoder-lm-wikitext-debug.config.json \
affon run apps/decoder-lm/train.ts
The debug configuration stops after 32 training batches and evaluates eight validation batches. In my run, validation loss moved from 10.832 to 9.833 and the app saved clean-debug-run-epoch-1. The short generated sample was incoherent. That is expected from this check; these numbers show that the training path executes, not that the model is useful or competitive.
Affon calls the computation a Program. The decoder Program names its input shape and parameters, including the token embedding table and the layers that use it. A training Program is derived from the model, a cross-entropy loss, and an optimizer. The app compiles those Programs for a Session and keeps model state explicit.
That structure gives me something to inspect before execution: which operations and tensor shapes the model asks for, and where parameters belong in the nested decoder. It also lets the same model definition serve training and inference. The model source and workflow show the full path.
The complete walkthrough covers installation, output files, and the longer training configuration. If you work on TypeScript ML, I would value feedback on the Program boundary: what would you need to inspect before trusting a larger training run?