Glad Labs fixed a GPU pinning issue where LiteLLM 1.89.2's global api_base override prevented per-model routing, causing vision tasks to cold-load onto the wrong GPU. The team also hardened content gu…
Google Cloud introduced a multi-node KV cache offloading solution using GKE and Managed Lustre for large language model inference, achieving over 50% TCO savings and nearly 60% reduction in GPU-hour r…
DeepSeek launched DSpark, an open-source speculative decoding module that boosts AI inference speed by up to 400% on existing chips, potentially reducing demand for Nvidia's high-end accelerators. The…
Chinese AI firm DeepSeek open-sourced DSpark, a speculative decoding system that accelerates large language model inference by up to 85% without altering output quality, releasing it under the MIT lic…
HackerRank open-sourced its AI résumé screener, hiring-agent, revealing that the same résumé scores between 66 and 99 across runs due to LLM nondeterminism, not parsing errors. The volatility means a …
Privatewhisper.ai launched a private AI voice dictation tool that runs on-device, in a hardware-secured enclave, or via anonymous cloud processing, with no account required and audio never stored or t…
HackerRank open-sourced its applicant tracking system (ATS) scoring tool, but testing reveals extreme score variability—ranging from 66 to 99 on the same resume—due to LLM non-determinism. The tool's …
Ollama provides a tool for running large language models locally on macOS, Linux, and Windows without requiring an API key or cloud service. The tool packages model weights, a runtime based on llama.c…
A developer's guide explains how to run open-source AI models locally on consumer hardware in 2026, highlighting that a mid-range laptop can now run models once considered frontier-class. The guide co…
DeepSeek-AI released DeepSpec, an open-source codebase for training and evaluating draft models for speculative decoding, supporting three draft model algorithms (DSpark, DFlash, Eagle3) and requiring…
Google announced a method to retrofit Multi-Token Prediction onto frozen Gemini Nano v3 models, accelerating on-device inference for Pixel 9 and 10 series devices. The approach appends a lightweight T…
LocalClicky, a new open-source voice assistant for Mac, runs entirely offline using local AI models like Whisper.cpp and Ollama, ensuring no voice data, screenshots, or commands leave the user's machi…
A researcher replicated Anthropic's concept-injection experiments on 14 open-weight language models and found that the models do not satisfy criteria for genuine introspection, instead exhibiting stat…
Off Grid AI Desktop is a free, open-source app that runs large language models locally on Apple Silicon Macs, leveraging unified memory for GPU inference. The app supports models like Gemma and Qwen, …
Researchers at arXiv found that jailbreak attacks on large language models can be detected by analyzing entropy dynamics in intermediate layers, rather than final outputs. The study shows that monoton…
Google's GKE Labs introduced OpenRL, an open-source self-hosted API for post-training and fine-tuning large language models on Kubernetes clusters. The project decouples reinforcement learning infrast…
AT&T and GSMA launched the Open Telco AI platform, using Google's open-source Gemma models to create domain-specific telecom AI models that achieved up to 91.74% accuracy in network operations. The in…
Ollama provides a REST API with 14 endpoints for running large language models locally. The API includes endpoints for text generation, chat, embeddings, model management, and OpenAI compatibility. De…
Hugging Face engineer Onur developed a real-time notification system for the OpenClaw repository using local open-weight models like Gemma and Qwen, running on an NVIDIA GB10 with 128 GB of unified me…
Google DeepMind researchers audited DiffusionGemma, a text diffusion model, and found it is not significantly less transparent than Gemma in terms of variable interpretability, but algorithmic transpa…