{"slug": "i-trained-a-small-language-model-in-typescript-here-is-the-path-that-worked", "title": "I trained a small language model in TypeScript. Here is the path that worked.", "summary": "A developer used Affon 0.4.2, a TypeScript ML framework, to define, train, evaluate, and checkpoint a small six-layer decoder language model entirely from a TypeScript application. Running on macOS arm64 with Metal, a debug configuration of 32 training batches and eight validation batches moved validation loss from 10.832 to 9.833 and saved a checkpoint, though the generated sample was incoherent. The developer framed the run as proof that the training path executes, not that the model is useful, and highlighted Affon's explicit 'Program' abstraction for inspecting operations and tensor shapes before execution.", "body_md": "TypeScript ML examples often stop at calling an existing model. I wanted to check the other path: define a decoder, train it on text, evaluate it, and save its state, all from a TypeScript application.\n\nI used [Affon's decoder app](https://github.com/neucine/affon/tree/main/apps/decoder-lm). It defines embeddings, causal attention, and six decoder layers. The bundled WikiText-2 text becomes 256-token windows. The training workflow evaluates batches, checks validation loss, and writes a checkpoint.\n\nWith Affon 0.4.2 installed on macOS arm64, this ran on Metal from a source checkout:\n\n```\ngit clone https://github.com/neucine/affon.git\ncd affon\nRUNTIME_PACKAGE_PATH=\"$PWD/packages\" \\\n  AFFON_TRAIN_CONFIG=apps/decoder-lm/configs/train-decoder-lm-wikitext-debug.config.json \\\n  affon run apps/decoder-lm/train.ts\n```\n\nThe debug configuration stops after 32 training batches and evaluates eight validation batches. In my run, validation loss moved from **10.832 to 9.833** and the app saved `clean-debug-run-epoch-1`. The short generated sample was incoherent. That is expected from this check; these numbers show that the training path executes, not that the model is useful or competitive.\n\nAffon calls the computation a **Program**. The decoder Program names its input shape and parameters, including the token embedding table and the layers that use it. A training Program is derived from the model, a cross-entropy loss, and an optimizer. The app compiles those Programs for a Session and keeps model state explicit.\n\nThat structure gives me something to inspect before execution: which operations and tensor shapes the model asks for, and where parameters belong in the nested decoder. It also lets the same model definition serve training and inference. The [model source](https://github.com/neucine/affon/blob/main/apps/decoder-lm/src/model.ts) and [workflow](https://github.com/neucine/affon/blob/main/apps/decoder-lm/src/workflow.ts) show the full path.\n\nThe [complete walkthrough](https://affon.ai/docs/getting-started/train-language-model/) covers installation, output files, and the longer training configuration. If you work on TypeScript ML, I would value feedback on the Program boundary: what would you need to inspect before trusting a larger training run?", "url": "https://wpnews.pro/news/i-trained-a-small-language-model-in-typescript-here-is-the-path-that-worked", "canonical_source": "https://dev.to/chaoyangnz/i-trained-a-small-language-model-in-typescript-here-is-the-path-that-worked-2j5g", "published_at": "2026-11-16 19:46:37+00:00", "updated_at": "2026-11-16 19:51:22.115821+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-tools", "developer-tools"], "entities": ["Affon", "TypeScript", "WikiText-2", "Metal", "macOS", "neucine"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/i-trained-a-small-language-model-in-typescript-here-is-the-path-that-worked", "markdown": "https://wpnews.pro/news/i-trained-a-small-language-model-in-typescript-here-is-the-path-that-worked.md", "text": "https://wpnews.pro/news/i-trained-a-small-language-model-in-typescript-here-is-the-path-that-worked.txt", "jsonld": "https://wpnews.pro/news/i-trained-a-small-language-model-in-typescript-here-is-the-path-that-worked.jsonld"}}