MLOps news and analysis on Web Pulse: 3397 curated articles tracking the latest MLOps developments, tools, and research, updated continuously from vetted sources.
A former B2B SaaS web developer who switched to inference engineering about a month ago published a skills map arguing the field requires only three service categories: routing and scheduling via Nvidia Dynamo or llm-d, …
An open-source command line tool called Claude Task Master automates the full pull request workflow, planning tasks, writing code, running tests, pushing commits, opening pull requests, responding to CI failures and revi…
A developer released VernLLM, an open-source MIT-licensed LLM call framework for TypeScript, and stress-tested it with over 1,000 LLM calls. The framework offers adaptive AIMD rate-limiting, advanced circuit-breakers, mu…
Amazon SageMaker Inference launched prefix-aware routing, a routing strategy that directs requests sharing the same prompt prefix to the same instance so cached key-value pairs are reused. Benchmarks on Llama 3.1 70B Ins…
Osprey, a target-agnostic speculative decoding method detailed in arXiv:2609.09338v1, raises mean acceptance length by 16.1% on Qwen3-8B, 21.2% on Llama-3.3-70B-Instruct, and 22.7% on MiniMax-M2.5 (229B), with a 17.5% ga…
Amazon launched model caching for Amazon SageMaker Inference on HyperPod, a feature that pre-loads model weights and container images onto cluster nodes so inference pods can read from local NVMe storage at approximately…
ServiceNow is hiring a Staff Engineer for Machine Learning Systems & Reliability at its Moveworks unit in Mountain View, California, an on-site role posted without published pay. The job listing states Moveworks, acquire…
A pinned Docker recipe now serves the 0xSero/deepseek-v4-flash-0731-spark model on a single NVIDIA DGX Spark via Local Inference Lab's SparkInfer, exposing a 262,144-token limit with a K64 DSpark speculative draft and fi…
Neural Nova published GPU optimization benchmarks for LLM workloads, reporting throughput and cost gains across four models running on vLLM. Qwen3-235B-A22B on 8× NVIDIA H100-80GB posted +138.7% token/s and +58% cost sav…
Boundflow released Charter, an open-source platform for running production-safe AI agents on a company's own infrastructure, available at github.com/boundflow/charter. Charter lets users define agents, lifecycle, and run…
A developer building an enterprise generative AI platform on Oracle Cloud Infrastructure detailed the data pipeline behind a RAG system, describing how they moved 70,000+ documents through a staged ingestion process on a…
Inferact, founded by the creators and core maintainers of vLLM, is hiring University of Waterloo co-op students for an on-site Inference Engineering co-op in San Francisco, with pay not published. The role spans vLLM wor…
A developer published a step-by-step guide for deploying Google's 2B Gemma 4 model to Cloud Run with an NVIDIA L4 GPU, using vLLM for serving and a single-file Python MCP server to stage weights, deploy, health-check, be…
A developer detailed the "AI integration illusion," the systematic gap between AI features that work flawlessly in demo sandboxes and crash under production conditions. The writeup attributes failures to distribution shi…
NVIDIA's NIM (NVIDIA Inference Microservices) stack delivers 2.5x higher concurrency for the Nemotron 3 Ultra model through full-stack optimizations including PagedAttention-based KV cache management, optimized memory al…
NVIDIA's NIM 2.0.12 optimized serving stack delivered up to 2.5x higher output-token throughput than the open-source baseline when serving the Nemotron 3 Ultra model on four B200 GPUs, reaching 1,997 tok/s versus 718 tok…
A developer traced a silent vector search failure to pgvector's HNSW ef_search parameter, whose default of 40 limits the candidate list the graph walk keeps in flight, causing relevant chunks to be missed even when prese…
A solo operator running 42 domains and several SaaS platforms from Diakopto, Greece, has published a rulebook for letting AI coding agents work on a live production server without breaking it. The core principle is that …
Edge0-AI released edge0, an open-source streaming MoE inference framework that runs a 35B-parameter language model on Apple Silicon iPhones at roughly 2.9 GB peak active memory for the edge0-35b tier and about 1.0 GB for…
Futurum research cited in The Great Unification found 20% of enterprises were actively running the Model Context Protocol (MCP) in production in the first half of 2026, with another 26.9% building pilots, according to th…