Cisco's Foundation AI team released Antares-1B, a 1 billion parameter open-weight model that localizes software vulnerabilities in code repositories and runs on a single GPU, consuming under 5GB of VR…
Existing B70 users report that the best Intel LLM performance is achieved using a specific GitHub gist, with Qwen3.8-27B reaching 2313.29 tokens/s prefill and 30.76 tokens/s generation on a B70. One u…
FreeCodeCamp published a tutorial on scaling large language model inference for AI agents using vLLM, explaining GPU scheduling challenges and optimization techniques. The guide covers building intuit…
Red Hat's head of product for AI platforms, Tushar Katarki, says open source models and platforms are improving faster than enterprises planned, giving organizations more control over cost, data, and …
A new arXiv preprint (2608.13756v1) reports that swapping only the INT8 linear kernel (CUTLASS versus Triton) inside vLLM causes the two arms to agree on no sequence in any end-to-end comparison (0/8,…
Ray Direct Transport (RDT), a feature in Ray Core, enables RDMA-backed weight transfer in reinforcement learning for LLMs, achieving up to 7.5x speedup over naive RDMA implementations on GB200 nodes u…
Empero released Qwen3.8-9B, an unofficial community distillation of Alibaba's Qwen 3.8 model, compressing its reasoning into a 9B parameter model based on Qwen3.5-9B. Trained on roughly 70,000 reasoni…
Meta released Muse Glimmer on August 10, a 30B open-weight coding agent model that runs on consumer hardware, with quantized builds fitting in 24GB VRAM. The model, distilled from Meta's Muse Spark 1.…
Agent6, a new coding agent that jails model commands and uses editable state machines for long-running tasks, has been released. The agent restricts filesystem and network access via Landlock and secc…
Open source AI development is increasingly centered in China, with models outperforming GPT-4 in coding and math benchmarks while being released under permissive licenses, according to a technical ana…
Meta Superintelligence Labs released Muse Glimmer on August 10, a 30B-parameter open-weight model under Apache 2.0, designed for local agentic workflows and tool calling. The model, available on Huggi…
VLLM's KV cache and PagedAttention techniques reduce latency and cost in large language model inference by storing and efficiently managing key-value matrices, enabling higher throughput on existing G…
Alibaba's Qwen 3.8 27B dense model outperforms the larger Qwen 3.7 Plus in coding and office productivity tasks, according to a technical review. The 27B model features a 262,000-token context window …
Netflix's GenRec, an LLM-backed ranker in the 1B–10B parameter range, beat its mature production recommender system in a four-week A/B test on roughly 10% of live traffic, posting statistically signif…
Alibaba's open-source Qwen 2.5 series has surpassed 3 billion downloads, according to the company. The models are praised for stability across quantization levels, coding proficiency, long-context han…
Open-weight AI models have closed the performance gap with closed-source APIs, with specialized 7B to 30B models now outperforming older 175B models due to improved data quality and Mixture-of-Experts…
Alibaba released Qwen3.8-27B, a 27.78-billion-parameter open-weights multimodal model under Apache 2.0, with a 262,144-token native context window and configurable reasoning. Vendor-reported benchmark…
A developer detailed deployment recipes for Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter Mixture-of-Experts model with 95B active parameters, using vLLM. The guide covers verified GPU configurations, i…
Qwen has released Qwen3.8-27B, a 27-billion-parameter multimodal language model with integrated vision capabilities, built on the Qwen3.5 architecture. The model supports native image and video unders…
A developer argues that scaling up large language models alone does not make them smarter, and instead advocates for a practical blueprint combining prompt engineering, decoding parameters, retrieval-…