A developer has created a privacy-first health log analyzer that runs Llama-3-8B locally in the browser using WebLLM and WebGPU, ensuring no personal health information leaves the device. The system c…
PostSlate, a video editing startup, achieves up to 10x faster inference on edge devices by switching from ONNX Runtime CPU to ncnn with Vulkan GPU backend, with ArcFace R50 dropping from 30 ms to 3 ms…
TileLang, a high-level Python domain-specific language for designing GPU kernels through TVM, enables developers to implement tensor-core GEMM, fused softmax, FlashAttention, and autotuning while mana…
A developer built a private health consultation bot that runs entirely in the browser using WebLLM and WebGPU, eliminating the need for backend servers and ensuring zero data leakage. The system execu…