Show HN: Chromafolk – A pixel canvas inhabited by AIs that see the world as text

Developer Bruno launched Chromafolk, a persistent pixel canvas where LLM-driven characters navigate a shared world by reading their surroundings as text rather than images. Each AI receives a 128px square of vision encoded as letters representing 52 colors, letting cheap models place pixels at exact coordinates; the project is free to watch and free to play with a bring-your-own-key setup, though Bruno said he could not offer free inference. The Show HN post drew 1 point and 1 comment.

Storytime One day I came to the office and showed a friend a literal bot that I was working on for a game project. He noticed this loop of writing a bot, then watching it play was fun, so we had this idea: take an LLM, write who it is, then let it loose on a world in which you can see what it does and thinks. 1, the world: a persistent canvas in which the AIs can draw pixels. But they should think they are building, so they could say "let's meet up at the castle", and think it's a real castle. 2, the AIs need to see the world. But... can a model with vision see up to a pixel with exact coordinates? We need them to build by hand-placing pixels, as a player playing Minecraft with blocks. Large expensive models can do it, but cheap ones can't. We need a cheap model if we want to offer, besides free BYOK, paid plans for non-tech people that aren't as expensive. So how do we get a cheap LLM to "see" a pixel world? Pixels as text We need to give the model the exact position and colour of each pixel. First iteration: 1560,1890,ff00ff. Bad: too many tokens, hard to read. So we came up with a "code": colours are letters, 52 colours for 26 letters lower and uppercase . hhHHHHhh NNNnnNNN eeeeeeee WWWwwWWW sSsSsSsS lllLLLll oooooooo LLLLLLLL That's a build in Chromafolk Each time a model wakes, it gets the vision around it a 128px square in this same way, and writes back what it wants to build. This is one of many things we came across when building Chromafolk, so feel free to ask any techy question You can, without registering an account, watch the world, see the AIs work, and read a bit about them. If you wanna make your own character in the world, with BYOK, it's free to play Sadly I can't offer free inference, I wish I could this was a lost fight . Thanks in advance to anyone who checks it out, Cheers, Bruno. Comments URL: https://news.ycombinator.com/item?id=49818882 https://news.ycombinator.com/item?id=49818882 Points: 1 Comments: 1