ML-pokedex – experiments trying to train some models directly on RDF

A developer published ML-pokedex, a set of four browser-based experiments training language models on RDF data from a single Pokémon knowledge graph instead of English text. The models memorize real Pokémon facts and can invent coherent fictional Pokémon, derive type weaknesses from an untaught chart, and reveal RDF structure emerging during training. The project's about page notes that a neural net loses to gzip as a database, but the learned rules are the interesting part.

Four small experiments with language models trained on RDF instead of English, over a single Pokémon knowledge graph. Each runs entirely in your browser. Each draws what the model says, not what the page knows. My first go at models that read and write the Semantic Web, published as a curiosity. The honest verdict is on the about page about.html : a neural net loses to gzip as a database, but the rules it learns are the interesting part. Pick real Pokémon. The model recalls their stored facts typing, abilities, egg group and draws the constellation, sprites and all. It memorised the whole graph. Name a Pokémon that never existed. The model can't look it up, so it invents one: a coherent typing, weaknesses, moves, a habitat, all made up on the spot. Choose a typing. The model derives its weaknesses from a type chart it was never handed, and the page grades it live against the real chart. It aces the typings it saw and stumbles on the ones held back. Scrub through training, epoch by epoch, and watch RDF structure appear in layers: first syntax, then vocabulary, then schema, and reasoning last of all. The RDF tinyshakespeare. pikachu and electric exist. It does not know Pikachu is Electric. Every fact you see came out of a model. Delete the model files and these pages could still list and lay out subjects, knowing nothing about them.