Doubleword, one of six companies in the first wave of UK Sovereign AI investments, achieved up to 3× the throughput of vLLM for DeepSeek-V4-Flash on a single node of Isambard-AI, the UK's national AI …
NVIDIA's NVLink and NVSwitch technologies form a scale-up fabric that connects GPUs tightly enough to behave as a single machine, contrasting with scale-out fabrics like InfiniBand and RoCE that link …
NVIDIA's CUDA compiler pipeline transforms a simple vector addition kernel from PTX virtual assembly to SASS machine code through multiple compilation stages, including LLVM-based cicc and ptxas, befo…
Researchers have developed adaptive speculative decoding, a method that dynamically selects draft lengths at runtime to optimize token generation efficiency in large language models. The approach addr…
InfiniBand, a high-performance interconnect technology designed for Remote Direct Memory Access (RDMA), has become critical for AI training and inference workloads that require direct data movement be…
Researchers at Doubleword developed UCCL-EP, an expert parallel communications kernel that enables GPU-initiated communication patterns like DeepEP to work across arbitrary NIC-accelerator pairs, incl…
A high-performance Expert Parallelism (EP) kernel is essential for running large Mixture-of-Experts (MoE) language models across multiple GPUs, as it handles the dynamic routing of tokens to experts l…
Speculative decoding, a lossless inference optimisation that predicts future tokens to reduce latency, faces new economic constraints as modern mixture-of-experts (MoE) architectures replace dense tra…
AMD's MI300X accelerator, with 192GB of HBM3 memory and roughly half the list price of NVIDIA's H100, remains underutilized due to software incompatibilities. As of early May 2026, running vLLM with D…
Researchers have developed Speculative KV coding, a lossless compression method that reduces the size of large language model KV caches by up to approximately 4x using a smaller, faster predictor mode…
Researchers analyzing weight files from over a dozen open-weight language models — spanning labs including Qwen, DeepSeek, Google, and OpenAI, from 0.6 billion to 1.4 trillion parameters — found that …