I’ve pretty much been working entirely in isolation building everything from scratch. I can’t really be bothered going into too much detail but it is hosted on my github page for people to interact with, here: Prism — a tiny AI that runs in your browser
Its 842kb weights file and its a bigram tokens so its spelling every word, not just linked words together,
it was trained on a mixed corpus but due to the weighting of the available corpus it has tended to favour tiny stories type output generally but it has been trained on a lot but it’s rarely outputted.
Output at this current time the model produces:
Once upon a time
**known and there was a little girl named Lily. She loved to play with her friends. One day, Lily went to the park. She wanted to play with her friends. She wanted to play with her friends. She wanted to play with her mom. She saw a big, birds and saw a big skyst. She wanted to play with her mome to play. SuY.ke lear halie,p
I am hoping a few more days of grinding on my laptop and this model should have the rough bits ironed out to be a decent competitor for tiny stories.
I did have a got at trying to put together and publishing research but I aint a scientist so I lost interest in actually putting together this to 0 interest, but the basic findings are documented here: A Self-Replicating Swarm of Tiny AIs: Phasor-Face Transformers (PrismFormer) with Arithmetic as Algebra and Bit-Exact, Mergeable Gradients | Zenodo