A developer released TurboQuant, a pip-installable adapter for mlx-lm on Apple Silicon that uses a randomized Hadamard transform for data-oblivious, calibration-free quantization of weights and KV cac…
Researchers introduced LearnStop, a cost-aware early-exit method for reasoning language models that learns when to stop computation based on online features like answer confidence and entropy. Across …
A non-programmer named Tim is setting up a local LLM on a MacBook M5 Max with 128GB unified memory, using Docker Desktop with Model Runner, Open WebUI, and models like Gemma 4 and Qwen3 30B-A3B-Q4_k_m…
A developer identified a structural cause for coding agents ignoring system-prompt rules mid-session: attention dilution as context grows. The rule becomes an old, low-weight token among thousands of …
Researchers at the University of Washington released TheoremGraph, a unified dependency graph spanning 18 million+ mathematical statements from arXiv papers and the Lean formal proof assistant. The gr…
HW-TSC submitted a cascaded system to the IWSLT 2026 Subtitling track, using a large-model-based streaming speech recognition framework with VAD, sliding-window context caching, long audio chunking, a…
DeepSeek-AI released DeepSpec, an open-source codebase for training and evaluating draft models for speculative decoding, supporting three draft model algorithms (DSpark, DFlash, Eagle3) and requiring…
Mixture-of-Experts (MoE) models like Qwen3-30B-A3B and DeepSeek-V3 separate total parameters (memory) from active parameters (compute), allowing a 30B-parameter model to run at the speed of a 3B model…
NVIDIA released NeMo AutoModel, a library that integrates Expert Parallelism and DeepEP into Hugging Face's API, achieving 3.4x to 3.7x higher training throughput and 29% to 32% lower GPU memory consu…
A developer reported a CPU bug in Hugging Face's text-embeddings-inference tool, causing accuracy issues during concurrent embedding tasks. The bug, related to attention mask handling for equal-length…
NVIDIA released NeMo AutoModel, an open-source library that accelerates fine-tuning of Mixture-of-Experts (MoE) transformer models by 3.4-3.7x in training throughput and reduces GPU memory usage by 29…
Researchers introduced RASC+, a retrieval-constrained LLM adjudication method for clinical value set authoring, achieving a candidate-pool recall of 0.730 on the RASC benchmark. Using GPT-5 as a const…
A developer describes building a complex 'loop engineering' pipeline in December 2024 to translate large Korean documents to English using local LLMs, but ultimately abandoned the project after weeks …
Alibaba's Qwen3-80B and Qwen3-4B models were successfully co-located on a single NVIDIA DGX Spark using vLLM containers behind a LiteLLM proxy, but the 80B model's inability to emit tool calls in auto…
Researchers from Hao AI Lab introduced JetFlow, a speculative decoding framework that breaks the scaling ceiling of autoregressive LLMs by combining one-forward drafting efficiency with branch-wise ca…
NVIDIA Research released cuTile Rust, a tile-based system for writing memory-safe, data-race-free GPU kernels in Rust. The project extends Rust's ownership model to GPU programming, achieving up to 92…
A developer released FlashQwen, a from-scratch CUDA inference engine for Qwen3-8B, built with C++ and CUDA. The project is hosted on GitHub and aims to provide efficient inference for the Qwen3 langua…
Researchers propose ASAG, a training-free method that monitors attention distributions to detect when a reasoning model has reached a conclusion, stopping generation early. Applied to DeepSeek-R1-Dist…
Open WebUI now supports the Model Context Protocol (MCP), enabling local AI assistants to use external tools like web search, file access, and code execution. Developers can set up a Docker-based envi…
A developer built a private RAG system using AnythingLLM and Ollama that runs locally on any machine, allowing users to drop in PDFs, Word docs, and code files and ask questions without cloud dependen…