AWS is pairing its Trainium chips with Cerebras CS-3 systems to split transformer inference into prefill and decode phases, with Trainium handling prefill and Cerebras handling decode, shipping as a p…
Anthropic has begun watermarking text output from Claude, embedding imperceptible watermarks in text and attaching C2PA metadata to files, with models launched on or after 2 August 2026 supporting mar…
JAX's Pallas kernel system, which lowers through Triton on GPU and Mosaic on TPU, lets developers write custom kernels when XLA's automatic fusion is insufficient for operations like flash attention, …
JAX, a numerical computing library, is fundamentally a tracing machine that converts pure Python functions into a typed intermediate representation called a jaxpr, refusing to execute code with contro…
Google's open-source TPU compiler, shipped as the Mosaic TPU dialect inside JAX, documents eight explicit memory spaces that programmers must manage, including vector memory (VMEM), scalar memory (SME…
XLA, the compiler behind JAX, TensorFlow, and PyTorch/XLA, optimizes array programs by freezing shapes, statically allocating buffers, and fusing operations against a global cost model, which makes it…
XLA, the compiler under JAX, TensorFlow, and PyTorch/XLA, uses two intermediate representations: classic HLO (a hand-built C++ IR) and MLIR dialects such as StableHLO and CHLO, with a translation laye…
Torch.compile is not a traditional compiler but a system that intercepts Python bytecode at runtime, extracts compilable regions through a multi-stage pipeline, and stitches them back with eager Pytho…
Triton is a compiler with a Python frontend that parses a function's AST, runs it through an MLIR pipeline, and emits a GPU binary, never executing the Python function as Python. The compiler handles …
A cubin (CUDA binary) is a standard ELF64 file with NVIDIA-specific sections that encode everything needed to load and launch a kernel, including machine code, parameter layout, and register allocatio…
A survey of code repository hosting in 2026 finds that GitHub remains the default for most developers, but concerns over DMCA compliance, AI training on code, account suspension, and Microsoft ownersh…
PyTorch, JAX, and compiler-level tools like Enzyme use three different representations for automatic differentiation—runtime tape, functional trace-and-transform, and source (IR) transformation—each w…
MLIR, a compiler infrastructure framework, has become the foundation for numerous machine learning compilers including XLA, Triton, Mojo, Torch-MLIR, IREE, and ONNX-MLIR. It provides a reusable IR con…
VLLM, a large-model inference serving framework, uses Python for control flow but pushes arithmetic into compiled C++ and CUDA kernels to avoid interpreter overhead. The Python/C++ boundary crossing i…
VLIW architecture is experiencing a resurgence driven by machine learning workloads, which rely on regular, statically analyzable computation that VLIW exploits efficiently. Unlike out-of-order design…
A developer building vLLM from source on an AWS g5 instance with Ubuntu 26.04 and Python 3.14 encountered multiple version-skew, driver, and toolchain issues, including a pitfall where missing nvidia-…
Loop unrolling, a classic compiler optimization, is experiencing a resurgence in the machine learning era as a critical technique for maximizing throughput on modern compute architectures like SIMD ve…
VLLM, a model-serving engine for large language models, introduced a small op-level IR to resolve the tension between acting as a compiler target and a hand-tuned kernel dispatcher. The IR allows vLLM…
A developer explores the complexity of running a "Hello, World!" program on a heterogeneous system where a host processor launches code on a different target architecture, such as a DSP or AI accelera…