{"slug": "show-hn-tinytowns-a-small-model-to-generate-realistic-sounding-town-names", "title": "Show HN: Tinytowns, a small model to generate realistic sounding town names", "summary": "A developer released tinytowns, a hand-written character-level transformer in plain PyTorch that generates invented town names after training on 128,435 real place names from GeoNames' cities1000 dataset. The model ships in three sizes — nano at 4,572 parameters (20 KB, error score 2.47), small at 105,052 parameters (424 KB, 2.20) and tinytowns at 1,001,500 parameters (4 MB, 2.15) — measured on 14,271 held-out names against a 3.33 random-guess baseline, with more than 95% of each model's 10,000 test names coming out invented. The model runs entirely in the browser, downloading once from Hugging Face, and uses a 28-symbol vocabulary with a 21-character context window.", "body_md": "# tinytowns — a town name generator\n\nA very small AI model that makes up town names in the style of real ones. It guesses the next letter, one at a time. It does not know which places exist.\n\n**What it is for:** tinytowns makes up new town names in the style of real ones. It learned\nthe letter patterns of 128,435 real place names: it was trained to guess the next letter of a\nplace name as well as it can. **It does not know which places exist**, so most names it writes are\nnew. To find real places, look them up in a list of places, or ask a large AI model such as Claude. How it compares with large models:\n[Small model vs LLM](https://quirkyagents.com/wizard/projects/small-model-vs-llm-benchmark/).\n\n## Watch it\n\nA 37-second video. It shows the model's guesses, a real name, an invented name, and what you could use it for. The video has no sound.\n\n## What it is\n\ntinytowns is a very small AI model. It works like the models behind chat apps, but it is much smaller. You can read every number in it.\n\nThe model reads a name one letter at a time. Then it gives a chance to each symbol that can come next. There are 28 symbols: the 26 letters, a space, and \"end of name\". The bars show the five symbols with the highest chances.\n\n## How to use it\n\n1. Select a model. The larger models make better guesses.\n2. Type some letters, for example `san` .\n3. Look at the bars. To add a letter, tap its bar.\n4. To let the model write the rest of the name, tap **Finish it for me** .\n5. To end the name, tap **end of name** , type a full stop (.), or press Enter.\n\nWhen the name is complete, the page tells you if it is real or invented.\n\n## Temperature\n\nTemperature changes how the model selects letters. At a low temperature (\"safe\"), the usual letters get almost all the chances, and the names look like real names. At a high temperature (\"wild\"), unusual letters get more chances, and the names are stranger. Move the slider and look at the bars.\n\n## Real or invented\n\nThe model learned from a list of 142,706 real place names. When a name is complete, the page looks for it in this list. \"Real\" means that a place with this name exists. \"Invented\" means that the name is not in the list.\n\nMost names are invented, and that is the model's job. Each model made 10,000 names as a test. For each model, more than 95% of the names were invented. A \"real\" name can be one that the model copied from the list it learned from.\n\n## Where it runs\n\nThe model runs in your browser, on your device. The page downloads the model from Hugging Face one time. After that, your device calculates each guess. No server does this work.\n\n## Three sizes\n\n| Model | Numbers in the model | Download | Error score | \n|---|---|---|---|\n| nano | 4,572 | 20 KB | 2.47 | \n| small | 105,052 | 424 KB | 2.20 | \n| tinytowns | 1,001,500 | 4 MB | 2.15 | \n\nThe error score shows how bad the guesses are. A lower score is better. We measured it on 14,271 names that the model did not learn from. Random guesses get a score of 3.33.\n\nA model with more numbers makes better guesses, but each step up helps less. From nano to small, the score goes down by 0.26. From small to tinytowns, it goes down by only 0.05.\n\n## What it can do\n\nThese are ideas. We did not test them:\n\n- Make names for towns in games and on maps.\n- Make test data with place names that are not real.\n- Tell if a word looks like a place name. For example, it could find `asdfgh` in a city field.\n\n## Technical details\n\n- **Model:** a decoder-only, character-level transformer, written by hand in plain PyTorch.\nVocabulary of 28 (index 0`.` marks start and end of a name, then space,`a` –`z` ), context of 21.\nLearned token and position embeddings, pre-LayerNorm blocks (causal multi-head self-attention,\nthen a GELU MLP), a final LayerNorm and a separate unembedding. Biases on every linear layer, float32.\n- **Sizes** (d_model / layers × heads / MLP width): nano 16 / 1 × 1 / 64, small 64 / 2 × 4 / 256,\ntinytowns 128 / 5 × 4 / 512.\n- **Data:** GeoNames`cities1000` (places with 1,000+ people, 243 countries), lowercased, hyphens\nto spaces, apostrophes and names over 20 letters dropped: 142,706 names, split 90/10 with a fixed\nseed (128,435 train, 14,271 validation).\n- **Training:** next-symbol cross-entropy on`.` + name +`.` , padding ignored. AdamW, weight decay\n0.01, learning rate 3e-3 (nano, small) or 1e-3 (tinytowns), 6,000 steps, batch 64 on a CPU (nano)\nor 256 on a laptop RTX 3050 (small 135 s, tinytowns 550 s). The weights kept are from the step\nwith the lowest validation loss.\n- **Validation loss:** uniform 3.3322, bigram 2.6732, nano 2.4676, small 2.2049, tinytowns 2.1504\n(best at step 5,500; train loss 2.0085 there, and validation loss rises to 2.1543 by step 6,000).\n- **In the browser:** a hand-written JavaScript forward pass, with no runtime library. It parses the`.safetensors` header itself and recomputes the whole context for each guess (no KV cache). On all\n14,271 validation names it matches the PyTorch loss to six decimal places for all three sizes.\nMeasured in headless Chrome on a one-core cloud server: about 0.7 ms, 3 ms and 34 ms per guess.\n\nMore detail is in the [model card on Hugging Face](https://huggingface.co/dave8172/tinytowns) and the\nstep-by-step code on [GitHub](https://github.com/dave8172/tinytowns). The series\n[How an LLM runs](https://quirkyagents.com/wizard/blog/series/how-an-llm-runs/) explains the same parts in a full-size model.\n\nPlace names: [GeoNames](https://www.geonames.org), licensed\n[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).\n\n← [All work](https://quirkyagents.com/wizard/projects/#models)", "url": "https://wpnews.pro/news/show-hn-tinytowns-a-small-model-to-generate-realistic-sounding-town-names", "canonical_source": "https://quirkyagents.com/wizard/projects/tinytowns/", "published_at": "2026-10-22 08:57:59+00:00", "updated_at": "2026-10-22 09:11:52.632237+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-tools"], "entities": ["tinytowns", "Hugging Face", "GeoNames", "PyTorch", "Claude"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-tinytowns-a-small-model-to-generate-realistic-sounding-town-names", "markdown": "https://wpnews.pro/news/show-hn-tinytowns-a-small-model-to-generate-realistic-sounding-town-names.md", "text": "https://wpnews.pro/news/show-hn-tinytowns-a-small-model-to-generate-realistic-sounding-town-names.txt", "jsonld": "https://wpnews.pro/news/show-hn-tinytowns-a-small-model-to-generate-realistic-sounding-town-names.jsonld"}}