Bonsai, a vector search company, has identified a pervasive issue it calls 'Float Bloat' where embedding vectors are cast from float32 to float64 during serialization, doubling storage and network cos…
Microsoft Research released BitNet b1.58 2B4T, the first open-source native 1-bit LLM at the 2-billion parameter scale, which uses ternary weights (-1, 0, +1) requiring about 1.58 bits per weight, red…
DeepGrove's Maple-Preview, a 20B-parameter mixture-of-experts reasoning model with ternary weights, achieves 120 tokens per second on an iPhone and 218 tok/s on a base M4 Mac mini, according to the co…
Bonsai is a toolkit for building AI chat apps with branchable conversations, allowing users to explore tangents without derailing the original thread. The library, which is not a finished app but a de…
A developer known as pcdeni has created CaSA, an architecture that runs PrismML's ternary Bonsai LLM models directly inside commodity DRAM by breaking DDR4 timing rules and using charge-sharing, bypas…
A new AI model called Bonsai with 27 billion parameters and 1-bit weights can now run directly in a web browser, as demonstrated by the WebML Community on Hugging Face. The model's extreme quantizatio…
NVIDIA Jetson Orin Nano Super 8GB benchmarks show 25W as the energy-efficiency sweet spot for sub-4B Bonsai LLMs, delivering 47-48% more tokens per second than 15W while maintaining or improving outpu…
NobodyWho has released the world's first standalone Apple Watch app that runs AI models locally and offline without requiring an iOS companion app. The app supports small language models like Prism ML…