Show HN: PicoLM v1.0-rc1

PicoLM v1.0-rc1, an LLM inference engine written in C99, has been released, supporting llama-2, GPT-2, Qwen 3.6/3.8(+MoE), and Gemma-3n models. The engine features CPU SIMD acceleration, CUDA/HIP support, mmap()ed GGUFs for lower RAM usage, and an OpenAI/llama.cpp-compatible HTTP server, with optional live VNC visualization of activation heatmaps.

PicoLM is an LLM inference engine written in C99. It currently supports llama-2, GPT-2, Qwen 3.6/3.8 +MoE and Gemma-3n models. Significant amount of work went into CPU SIMD acceleration/testing/correctness, and wide cross-platform availability with constant testing to never lose portability from DOS through OS/X 10.4 to modernity . CUDA/HIP is supported, and accelerated IMMA kernels are available. More work needs to be done on prompt processing speed, but text generation is quite fast already. GGUFs are mmap 'ed, not preloaded, so it's much more friendly to RAM usage than llama.cpp. External LE/BE GGUF- FUGG utility available for the endian-handicapped. OpenAI/llama.cpp-compatible HTTP server, ready to use with harnesses. Eye candy: optional live VNC visualization of the per-layer activation heatmap. I do actual feature freeze and release cycles, unlike llama.cpp which did a grand total of zero in the past 3 years. In fact, v1.0-rc1 just got released. Comments URL: https://news.ycombinator.com/item?id=49547323 https://news.ycombinator.com/item?id=49547323 Points: 1 Comments: 0