Modular has built a new data layer for LLM inference routing that solves the problem of querying cached blocks across hundreds of pods in microseconds. The company's architecture uses a specialized da…
Ten strategies to reduce costs associated with LLM API usage, emphasizing that users pay for input and output tokens rather than the inference itself. Key recommendations include selecting appropriate…
Testing the MiniMax M2.7 model via API on three real-world workflows—refactoring a PyTorch project, drafting knowledge-base notes, and scaffolding a Kaggle competition entry—using Claude Code as the a…
Together AI launched Voice Finder, a search tool that indexes over 600 voices across multiple text-to-speech models to help developers quickly select the right voice for their applications. The tool a…
Modular announced that traditional HTTP-era load balancing algorithms like round-robin, consistent hashing, and least-connections are inadequate for large language model inference because GPU pods are…
Researchers analyzing weight files from over a dozen open-weight language models — spanning labs including Qwen, DeepSeek, Google, and OpenAI, from 0.6 billion to 1.4 trillion parameters — found that …
Anthropic capped a user's Claude Max plan at 15 daily routines after a scheduled batch job failed at 2:36 AM, revealing the platform's hidden constraints on heavy usage. The user calculated a $137 API…