Cerebras reported that disaggregating the prefill and decode stages of AI inference lifted throughput 5x in early results using the same number of its systems, without reducing token-generation speed,…
OpenAI previewed Ultrafast, a new API service tier for GPT-5.6 Sol that runs up to 14 times faster than the standard processing path and produces as many as 750 output tokens per second, enabled by Ce…
Inferact, a startup whose team includes vLLM maintainers, released open-source "megakernels" for Google's TPU v7 (Ironwood) that delivered 709 tokens per second for a single user on Moonshot AI's Kimi…
A developer tested Cloudflare Workers AI's free tier and found 24 open-source LLMs that accept inference requests through a single API endpoint, including OpenAI's gpt-oss-120b, Qwen2.5-Coder-32B, and…
A developer used Qwen-3.8-Flash-Next to reverse engineer and port the 1989 strategy game War of the Lance to modern browsers via a Three.js rendering engine, completing the work in about a week with l…
OpenAI is preparing a wider rollout of its Ultrafast API mode around its DevDay event on September 29, with new references appearing across the OpenAI Platform and API documentation, according to Test…
Aikido Security released Altar-1, an open-weight security model pruned from Z.AI's 753B-parameter GLM-5.3 MoE down to 328.0 GB of stored weights, a 78.2% reduction from BF16 and 32.8% smaller than its…
Cerebras reported a 22-second median across two successful runs of a dinner-booking AI agent using Qwen 3.8 27B on its inference hardware with Pi as the agent harness, which the company says is 19 tim…
Developer 0xSero released model-toolkit, a GitHub repository of standalone Python tools for evaluating, observing, pruning, and quantizing language models, drawn from the REAP and EXL3 experiments inc…
A developer building MCQplex, an exam-prep platform for Nepal's NEB curriculum, has open-sourced three small Node.js utilities extracted from that work. The packages include llm-free-cascade, which fa…
Cerebras co-founder and other chip industry leaders discussed AI infrastructure bottlenecks, efficiency, investment, and future innovation at an event held at Google Bay View, according to Forbes cont…
A 2025 study on predictive auditing of hidden tokens in LLM APIs found that invisible reasoning tokens can account for over 90% of a model's total token spend on complex tasks, driving billing surpris…
A developer built a minimal Python web server that turns Cerebras-hosted inference of Alibaba's Qwen 3.8 27B model into a live operating system with zero apps on disk, streaming tokens at 1,950 tokens…
AI inference has become the dominant focus of the AI industry in 2026, with Nvidia CEO Jensen Huang calling it the "inflection point of inference" at the company's GTC 2026 conference, according to Mo…
Software engineer Sean Goedecke argues that developer experience will become the bottleneck for AI coding agents as inference speeds rise, citing GPT-6-Astra at roughly 60 tokens per second versus Taa…
A developer built LogSentinel, a self-hosted AI-powered log forensics workbench that treats log files as hostile input to defend against prompt injection attacks embedded in log content. The tool sani…
Cerebras is now serving OpenAI's GPT-5.6 Sol at up to 750 output tokens per second, according to a Cerebras blog post, a speed the author says shifts the bottleneck from the model to the surrounding a…
The US Department of Justice has launched an antitrust probe into Nvidia's $20 billion acquihire of Groq, The New York Times reported this week, but The Register reports that unwinding the deal would …
Cerebras is now serving OpenAI's GPT-5.6 Sol at up to 750 output tokens per second, according to a Cerebras blog post, a speed the source frames as moving the performance bottleneck from the model to …
A developer has released Dario, a terminal-based AI coding assistant installed via npm, which offers 150 free queries per day without signup or API key. The tool routes requests across multiple provid…