Tencent's Zhuque Lab released AI-Infra-Guard, an open-source security scanner for AI systems that fingerprints services such as Ollama, vLLM and ComfyUI, checks them against more than 1,600 known CVEs…
Crusoe, a vertically integrated AI infrastructure company, is hiring a Staff Applied AI Inference Engineer in San Francisco with a salary range of $215,000–260,000 per year, which sits 8% above the $2…
A developer built a local AI inference server for about $2,000 using ten-year-old hardware, including two NVIDIA Tesla P40 GPUs and an AMD EPYC processor. The system handles thousands of agent request…
TrustGraph 2.8 has been released, introducing an asynchronous architecture, hybrid retrieval, structured output, and a plugin-based workbench. The update replaces thread-per-consumer pub/sub with conf…
Gremlord, a new open-source tool announced on Hacker News, wraps Anthropic's Claude Code in a local router that lets users run the coding assistant on any model—including OpenAI, xAI, Ollama, vLLM, an…
Cohere released a serving engine for its North Mini Code model built around a decode megakernel that runs 1.25x to 1.41x faster than vLLM end-to-end on a single H100 with BF16 precision. The engine, a…
AMD and the vLLM project detailed five speculative decoding methods for vLLM on AMD GPUs, achieving up to 2.59x throughput improvement for certain large language models. The techniques enable producti…
Mistral raised €3 billion in a Samsung-led Series D at a €21 billion-plus valuation, marking the largest equity round ever for a European tech company, with funds allocated to compute, frontier resear…
ByteDance's open-source DeerFlow framework, which orchestrates sub-agents, memory, and sandboxes for complex AI tasks, reached the #1 spot on GitHub Trending in February 2026 following its version 2 l…
Ray Summit 2026, held August 24-26 at the San Francisco Marriott Marquis, drew more than 2,000 attendees to focus on reinforcement learning as a production engineering problem, with Robert Nishihara, …
AMD and the veRL project released a production-ready reinforcement learning post-training container for AMD Instinct GPUs, supporting MI300 and MI355 series on ROCm, with AITER-accelerated vLLM and SG…
OpenBMB released MiniCPM5-2B, a 2.52B-parameter dense causal language model with a native 131,072-token context, averaging 53.9 across 34 benchmarks and outperforming Qwen3.5-4B's 51.1, with notable s…
Superlinked has released the Superlinked Inference Engine (SIE), an open-source server designed to consolidate the many small models used in AI agent workflows into a single deployment, addressing wha…
A developer's source-code walkthrough of vLLM 0.22's V1 execution path traces a single offline inference request from the LLM.generate() API through inter-process communication, scheduling, GPU execut…
AMD's experiments with speculative decoding in vLLM on AMD Instinct MI300X and MI355X GPUs using the ROCm platform show that output-token throughput gains vary by drafting method, proposal length, mod…
Researchers introduced Sarathi-Serve, an LLM inference scheduler that uses chunked-prefills and stall-free scheduling to improve throughput-latency tradeoffs, achieving 2.6x higher serving capacity fo…
The Institute of Foundation Models (IFM), the frontier lab launched by MBZUAI in May 2025, released K2 Horizon, a fleet of six Apache 2.0-licensed open-source models ranging from 0.9B to 375B paramete…
Anshvardhan Shetty, a first-year EECS undergraduate at Imperial College London and research intern with Professor Juncheng Yang at Harvard SEAS, presents a study of KV cache admission on high-bandwidt…
Meta researchers developed Proteus, a system that uses AI agents to generate specialized GPU kernels, achieving 1.8–5.2× faster inference for Qwen 3.5 122B models compared to the best kernels in vLLM.…
NVIDIA announced that new compact open models, including Nemotron 3.5 Lightning and Qwen3.8-27B, can now run reasoning and agentic AI workloads on Jetson edge devices, eliminating the need for data ce…