Article URL: https://huggingface.co/spaces/pliny-the-prompter/obliteratus
Comments URL: https://news.ycombinator.com/item?id=49889532
Points: 1
source & further reading
huggingface.co — original article
llama.cpp v1.0.0 : A stronger foundation for the local AI ecosystem
Same A100, similar throughput: would both pass a 1-second first-token target?
.shn5: lossless BF16 model files ~1.5× smaller, and a 61 GB MoE on a 16 GB GPU