DeepSeek open-sourced its full frontier-model training software stack on September 30, centered on TileLang, a Python-like domain-specific language that compiles GPU compute kernels to Huawei Ascend 9…
WebLLM, an open-source library from MLC AI, enables running large language models like Llama 3, Mistral, and Phi-3 directly in the browser via WebGPU, eliminating server-side latency and data privacy …
A developer has created a privacy-first health log analyzer that runs Llama-3-8B locally in the browser using WebLLM and WebGPU, ensuring no personal health information leaves the device. The system c…
PostSlate, a video editing startup, achieves up to 10x faster inference on edge devices by switching from ONNX Runtime CPU to ncnn with Vulkan GPU backend, with ArcFace R50 dropping from 30 ms to 3 ms…
TileLang, a high-level Python domain-specific language for designing GPU kernels through TVM, enables developers to implement tensor-core GEMM, fused softmax, FlashAttention, and autotuning while mana…
A developer built a private health consultation bot that runs entirely in the browser using WebLLM and WebGPU, eliminating the need for backend servers and ensuring zero data leakage. The system execu…