{"slug": "triton-notes", "title": "Triton Notes", "summary": "OpenAI-developed Triton, a programming language and compiler for writing custom GPU kernels, lets developers describe block-level operations on data tiles and have the compiler map them onto GPU threads, rather than managing individual threads as in CUDA. The introductory note walks through Triton's host/device split and kernel-launch pattern, covering shape, stride, dtype and device-feature validation, grid sizing via 1D, 2D or 3D launch tuples, and the non-persistent tiled launch style that assigns one program per output tile using triton.cdiv to compute ceil(a/b) so partial tiles still get a program. The note uses vector addition and matrix multiplication kernels to build intuition for the abstraction.", "body_md": "# Triton Notes\n\n**[Triton](https://github.com/triton-lang/triton)** is a programming language and compiler originally developed at OpenAI that makes custom GPU kernels easier to write. It makes efficient GPU code easier to express than in CUDA, especially for deep learning workloads.\n\nIn CUDA, you usually decide what each individual thread does and manually manage many low-level details. Triton provides a higher-level abstraction: you describe what one **program** does to a block, or tile, of data. Triton's compiler then maps those block operations onto GPU threads and generates the GPU code. We will see what this means later in this note.\n\nIt is also worth noting that Triton uses the same style of array programming found in PyTorch and NumPy. If that way of thinking is unfamiliar, these two introductions are useful refreshers to *think in shapes*:\n\nThis note is an introduction to Triton's abstractions. We will use two kernels—vector addition and matrix multiplication—throughout the note to build intuition.\n\n## [#](#overall-pattern-of-gpu-programming)Overall Pattern of GPU Programming\n\nAt a high level, GPU programming has two sides regardless of the [DSL](https://en.wikipedia.org/wiki/Domain-specific_language):\n\n1. **[Host (CPU)](https://modal.com/gpu-glossary/host-software) code** , which prepares the input tensors, decides how much parallel work to launch, and launches the kernel.\n2. **[Device (GPU)](https://modal.com/gpu-glossary/device-software) code** , or the kernel, describes the computation that each GPU program performs.\n\nSo the overall flow looks like this:\n\n1. Create host and device memory for tensor data.\n2. Copy tensor data from host to device.\n3. Load tensor data on-chip.\n4. Execute the operation.\n5. Store the tensor back to device memory.\n\nWe will divide this into two steps and look at each:\n\n1. Launching a kernel.\n2. Defining a kernel.\n\n## [#](#1-launching-a-kernel)1. Launching a kernel\n\nHow can we run a kernel?\n\nAs in CUDA, the host (CPU) launches the kernel before any device-side work begins. A single launch creates many instances of the kernel, called **programs** in Triton.\n\nWe will look at this in three steps:\n\n1. Validation.\n2. Setting the grid size.\n3. Launching the kernel.\n\n### [#](#validations)Validations\n\nBecause kernels provide considerable freedom over variables and memory, it is helpful to fail fast and loud when something is wrong. We therefore validate shapes, strides, dtypes, and device-specific features, such as those introduced with Blackwell.\n\nThese assertions largely serve two purposes: ensuring correctness and enabling specialized fast paths. Many kernels branch into completely different implementations based on the results of these checks.\n\n#### [#](#validation-example)Validation example\n\n### [#](#setting-grid-size)Setting Grid Size\n\nA launch grid is a 1D, 2D, or 3D tuple specifying how many program instances run along each axis. Inside the kernel, `tl.program_id(axis)` identifies the current program along that axis.\n\nPoor grid mapping can reduce parallelism or cache locality, so it's important to tune it.\n\nSo how do we set the grid size? Most kernels use one of two launch styles:\n\n#### [#](#1-non-persistent-tiled-launch-one-program-per-output-tile)1. Non-persistent tiled launch (one program per output tile)\n\nIn a non-persistent tiled launch, we divide the output into tiles of `BN` elements and launch one program for each tile. Program 0 processes the first `BN` elements, program 1 processes the next `BN` elements, and so on. Therefore, the grid size is the number of tiles required to cover all `N` elements.\n\nFor example,\n\n**What is happening here?**\n\n[`triton.cdiv(a, b)`](https://triton-lang.org/main/python-api/generated/triton.language.cdiv.html) computes `ceil(a / b)`. This ensures that a final partial tile still receives a program. The kernel must mask any lanes that fall outside the logical tensor bounds.\n\nWhen `grid` is a function, Triton passes it a dictionary containing the kernel's compile-time meta-parameters, such as `BN`. This lets the grid size depend on the selected block sizes, including values selected by [autotuning](#1-autotune).\n\n#### [#](#2-persistent-launch-multiple-output-tiles-per-program)2. Persistent launch (multiple output tiles per program)\n\nA persistent kernel launches a limited number of programs, often bounded by the number of output tiles and chosen relative to the GPU's SM count. Each program loops over tiles inside the kernel. See [when one program processes multiple tiles](#optional-when-one-program-processes-multiple-tiles) for how this works in the kernel.\n\n### [#](#what-parameters-do-we-send)What Parameters Do We Send?\n\n#### [#](#input-and-output-tensors)Input and output tensors\n\nWe pass the input tensors that the kernel reads and preallocated output tensors that it writes.\n\n#### [#](#number-of-elements-in-each-dimension)Number of elements in each dimension\n\nThe number of elements in a tensor (e.g., 1,000) may not be divisible by its tile size (e.g., 32). The kernel receives the logical dimension and creates a mask such as `offs_n < N`, preventing lanes in a partial tile from accessing memory outside the vectors.\n\n#### [#](#strides)Strides\n\nStride is the number of elements between adjacent positions along one tensor dimension. For vector addition, the stride is usually `1` because vectors are commonly contiguous, but a non-contiguous vector may have a different stride. We will talk more about strides in the two-dimensional case.\n\n#### [#](#static-arguments-tlconstexpr)Static arguments (`tl.constexpr`)\n\nAn argument annotated with `tl.constexpr` is known at compile time. Triton can use it to specialize generated code, size block-shaped tensors, or remove compile-time branches. Tile sizes are common autotuning targets, while `tl.constexpr` is also used for algorithm variants and feature flags.\n\n#### [#](#example-of-launching-a-kernel)Example of launching a kernel\n\n### [#](#end-to-end-example-launching-a-vector-add-kernel)End-to-end example: launching a vector add kernel\n\n## [#](#2-defining-a-kernel)2. Defining a kernel\n\nUntil now, we have seen how the host launches a kernel. Before looking at how a Triton kernel is defined, let's first go over the two most important abstractions in Triton: tile and program.\n\n### [#](#central-abstractions-tile-and-program)Central abstractions: Tile and Program\n\nConsider adding two vectors: C = A + B.\n\nIf the vectors contain one million elements, a Triton kernel usually does not process all one million elements in a single program. Instead, we divide the output into smaller regions and launch multiple instances of the kernel.\n\nTL;DR: A tile is a region of the data, while a program is an executing kernel instance that processes that region.\n\n#### [#](#tile)Tile\n\nA *tile* is a logical region of the overall problem that is processed together.\n\nFor example, one program might process a block of `1024` consecutive elements of the vector addition output `C`.\n\nA tile is represented by block-shaped offsets, pointers, and values inside the program.\n\nThis differs from CUDA, where the unit of code is a *thread*: values are scalars, and you index one element at a time.\n\n##### [#](#tile-vs-block)Tile vs Block\n\nIn Triton, *tile* and *block* are often used almost interchangeably. Both commonly refer to a group of tensor elements processed together by one program.\n\nTo be concrete,\n\n- **Tile** refers to the logical region of the overall tensor or problem.\n- **Block** refers to the block-shaped offsets, pointers, or values manipulated inside the program.\n\nFor example, a program may be responsible for the tile `C[1024:2048]`. Inside the program, that tile is represented by blocks of offsets and values containing up to 1024 elements.\n\nIn this note, we use them interchangeably.\n\n#### [#](#program)Program\n\nA Triton *program* is one executing instance of a compiled kernel. It is roughly analogous to a CUDA thread block, also called a [CTA](https://modal.com/gpu-glossary/device-software/cooperative-thread-array).\n\nThe CPU kernel launch shown above creates as many program instances as specified by the grid. For example, if `N = 4,096` and `BN = 1024`, four programs are launched.\n\nEach program receives a program ID (`pid`) and usually processes a different input or output tile. Triton launch grids have at most three dimensions, matching the grid defined by the host.\n\nAll four programs execute the same kernel code but operate on different data. In fancy terms, we call this [SPMD](https://en.wikipedia.org/wiki/Single_program,_multiple_data).\n\nNow that we've covered the two central abstractions, let's see how a kernel is defined and run on the GPU.\n\nAt a high level, each program follows these steps:\n\n1. Read the program ID and determine which output tile the program owns.\n2. Construct the tile’s global element offsets.\n3. Use those offsets to form input and output pointers.\n4. Load the input blocks.\n5. Compute the output block.\n6. Store the output block.\n\n### [#](#typical-kernel-parameters)Typical Kernel Parameters\n\nTriton kernels receive runtime arguments and configuration metadata from the host code described in the previous section, including:\n\n- **Input and output pointers:** PyTorch tensors passed at launch become pointers to their GPU storage.\n- **Logical dimensions:** sizes used for iteration and masks.\n- **Strides:** element offsets used to map multidimensional indices to memory locations.\n- **Runtime scalar values:** optional values such as a dropout probability or scaling factor.\n- **Compile-time meta-parameters:**`tl.constexpr` values such as tile sizes or feature flags.\n\n#### [#](#example)Example\n\n### [#](#1-read-the-program-id-and-determine-which-output-tile-the-program-owns)1. Read the program ID and determine which output tile the program owns\n\n#### [#](#program-id)Program ID\n\nA program ID, or `pid`, identifies a program along one launch-grid axis through `tl.program_id(axis)`. The kernel uses it to determine which tile that program processes.\n\n#### [#](#example-1)Example\n\nHere we use `axis=0` because the host launched a one-dimensional grid.\n\n### [#](#2-construct-the-tiles-global-element-offsets)2. Construct the tile’s global element offsets\n\nLocal offsets are indices within the tile, ranging from `0` to `tile_size - 1`. We simply use `tl.arange` (just like `np.arange` or `torch.arange`) to create the array.\n\nTo locate the tile within the full tensor, we incorporate the program ID. In practice, kernels commonly construct global offsets directly:\n\nWhat does this mean? Since each program processes a tile of size `BN`, we multiply it by `pid` to locate that program's block. So `offs_n` ranges from `pid * BN` to `(pid + 1) * BN - 1`.\n\nFor example, if `BN = 1024` and `pid = 3`, `offs_n` is:\n\nThese lines construct block-shaped tensors of output coordinates. The coordinates are later combined with base address and strides to construct pointers for `A`, `B`, and `C`.\n\n#### [#](#masking)Masking\n\nAs explained, masks allow a regular block-shaped program to safely handle the final partial tile. A mask is usually applied to the global offsets.\n\n#### [#](#example-2)Example\n\nThis way, even if `N` is not a multiple of `BN`, the final partial tile can be masked correctly.\n\n### [#](#3-use-those-offsets-to-form-input-and-output-pointers)3. Use those offsets to form input and output pointers\n\nOnce we have the global offsets, we map the logical indices to memory pointers. These are the final memory addresses we will use to load the tensor elements.\n\n### [#](#4-load-the-input-blocks)4. Load the input blocks\n\nInput tensors normally already reside in GPU global memory before the kernel begins. A Triton kernel reads values with `tl.load`, computes on them using on-chip resources (typically registers), and writes results back with `tl.store`. The compiler and hardware manage lower-level movement through the cache hierarchy.\n\nWe use the mask we've created here as well to mask off elements outside the bounds.\n\nThe `other` argument supplies a value for masked-out loads. We usually use `0.0` or `-float(\"inf\")` depending on the context. For example, vector addition can simply use `0.0`.\n\n**Example**\n\n### [#](#5-compute-the-output-block)5. Compute the output block\n\nIn Triton, computation is pretty high-level. More complex kernels may require additional operations, but many computations are expressed using basic arithmetic or matrix multiplication (`tl.dot`). Vector addition is dead simple:\n\n### [#](#6-store-the-output-block)6. Store the output block\n\nAfter we've loaded and computed the values, we need to store them back in device memory. We need the pointers that specify where to store them, the value to store (`c`), and the mask.\n\n### [#](#end-to-end-example-of-a-vector-addition-kernel)End-to-end example of a vector addition kernel\n\n## [#](#2d-and-beyond)2D and beyond\n\nWe've seen the most basic kernel as an exercise. Now let's move one step forward and deal with two-dimensional scenarios.\n\nMany operations in deep learning, including matmul, use tiles with two or more dimensions. We will go over the same path again while introducing additional things to consider when dealing with multi-dimensional arrays. For the parts already explained, I won't repeat myself.\n\n### [#](#launching-a-matmul-kernel)Launching a matmul kernel\n\nAs we've seen above, we validate, set grid size, and launch the kernel:\n\n#### [#](#strides-1)Strides\n\nNotice we pass two strides for each tensor because `A`, `B`, and `C` are all two-dimensional: one stride for moving between rows and another for moving between columns.\n\nConsider a [row-major](https://en.wikipedia.org/wiki/Row-_and_column-major_order) matrix `A` with shape `[M, K]`. Moving one column advances by one element, while moving one row skips over `K` elements. Therefore, when `A` is contiguous, its strides are `(K, 1)`.\n\nMost of the time, tensors sent to kernels are created contiguously. However, a transposed tensor or another view may have a different memory layout. Passing `a.stride(0)` and `a.stride(1)` allows the kernel to use the correct row and column jumps in either case. See the diagram below.\n\n### [#](#defining-a-matmul-kernel)Defining a matmul kernel\n\nAs usual, we first receive all the parameters sent by the host launch.\n\n#### [#](#1-read-the-program-ids-and-determine-which-output-tile-the-program-owns)1. Read the program IDs and determine which output tile the program owns\n\nSince we launched a two-dimensional grid, program instances are also coordinated along two dimensions.\n\n##### [#](#optional-when-one-program-processes-multiple-tiles)Optional: When one program processes multiple tiles\n\nThe matmul kernel above uses a straightforward mapping: the host launches one program for every output tile, and `pid_m` and `pid_n` identify that tile directly.\n\nAnother option is a **persistent-style launch**, where the host launches fewer programs than there are output tiles. Each program then processes several tiles in a loop. This is where `tl.num_programs(axis)` becomes useful: it returns the number of programs launched along an axis, which determines how far each program advances through the tile list.\n\nFor a 1D launch grid, we can flatten the two-dimensional output-tile grid into a single sequence:\n\nProgram `0` starts with tile `0`, program `1` starts with tile `1`, and so on. After processing its first tile, each program advances by `num_programs`.\n\nFor example, if four programs are launched for sixteen output tiles, program `0` processes tiles `0`, `4`, `8`, and `12`. The diagram below illustrates this assignment.\n\nThe host must explicitly choose this smaller launch grid. `tl.num_programs()` only reports how many programs were launched; it does not make the kernel persistent by itself.\n\n#### [#](#2-construct-the-tiles-global-element-offsets-1)2. Construct the tile’s global element offsets\n\nNext, get the global offsets for each dimension:\n\n#### [#](#3-use-those-offsets-to-form-input-and-output-pointers-1)3. Use those offsets to form input and output pointers\n\nThen form the addresses by adding those offsets to the pointers:\n\nWhat do `[:, None]` and `[None, :]` mean? To understand this, we should brush up on how broadcasting works in array programming. Feel free to skip this section if you're already fluent with the notation!\n\n##### [#](#broadcasting)Broadcasting\n\n**Broadcasting** allows tensors with different shapes to participate in the same element-wise operation. When a dimension of size `1` is matched with a larger dimension, its values behave as though they were copied across that dimension. Note that this expansion is logical—no larger tensor needs to be allocated or materialized in memory. NumPy, PyTorch, and Triton compare shapes from the rightmost dimension to the left.\n\nTwo dimensions are compatible when, comparing from the rightmost dimension:\n\n1. They are equal;\n2. One of them is `1` ; or\n3. One shape has fewer dimensions, in which case the missing leading dimensions are treated as `1` .\n\nBased on this rule, let's go through some examples:\n\nFor A of shape `[3, 4]`:\n\n- `[3, 4]` is compatible because the shapes are equal.\n- `[1, 4]` is compatible because the rightmost dimensions are equal and the leading`1` expands to`3` .\n- `[1, 1]` is compatible because each dimension of size`1` expands to match`[3, 4]` .\n- `[1, 2]` is incompatible because the rightmost dimensions,`4` and`2` , are neither equal nor`1` .\n\nIn a matmul kernel, broadcasting turns the M and K coordinate vectors into an M-by-K pointer tile for `A`:\n\nThe same idea constructs the K-by-N pointer tile for `B`:\n\nSince `[B0, 1]` and `[1, B1]` are compatible, Triton broadcasts both into `[B0, B1]`. No expanded index arrays need to be materialized in global memory.\n\n##### [#](#expanding-dimensions-for-broadcasting)Expanding dimensions for broadcasting\n\nIn Triton, you can explicitly use `tl.expand_dims(offs_m, axis)` or `tl.unsqueeze(offs_m, axis)`. However, we usually use `:` and `None` to express this:\n\nThis takes all M coordinates and inserts a new trailing dimension, producing one column.\n\nThis takes all K coordinates and inserts a new leading dimension, producing one row.\n\n##### [#](#strided-pointers)Strided Pointers\n\nAn intuitive way to understand multidimensional pointer arithmetic is to consider one axis at a time. The indices along an axis tell us how many steps to take, while its stride tells us how many memory elements each step moves. Multiplying them gives that axis’s contribution to the final memory offset. Broadcasting then combines the contributions from every axis to produce a pointer for each coordinate in the tile.\n\nFor example,\n\n`offs_m[:, None] * stride_a_m` computes the row contribution and broadcasts it across every column. Meanwhile, `offs_k[None, :] * stride_a_k` computes the column contribution and broadcasts it across every row. Adding them to `a_ptr` produces the complete M-by-K grid of pointers.\n\n#### [#](#4-load-the-input-blocks-1)4. Load the input blocks\n\nAt this point, the program owns a `BM × BN` tile of the output. Computing that tile requires accumulation across the entire K dimension. Rather than loading all of K at once, the program processes it in chunks of `BK` elements while keeping the partial result in `accumulator`. `accumulator` is set to `tl.float32` for numerical precision and later cast to `tl.float16` after the loop.\n\nEach iteration loads a `BM × BK` tile from `A` and a `BK × BN` tile from `B`. Their masks are constructed using the same broadcasting pattern as their pointers.\n\n#### [#](#5-compute-the-output-block-1)5. Compute the output block\n\n`tl.dot` adds the current K chunk’s contribution to `accumulator`, after which the pointers advance to the next K chunk.\n\n#### [#](#6-store-the-output-block-1)6. Store the output block\n\nWith these pieces in place, let's put the matmul host launch and Triton kernel together.\n\nThe host-side kernel launch looks like this:\n\nThe matmul kernel (without [L2 cache optimizations](https://triton-lang.org/main/getting-started/tutorials/03-matrix-multiplication.html#l2-cache-optimizations)) looks like:\n\nThat's it!\n\n### [#](#selecting-configurations)Selecting Configurations\n\nWhile Triton raises the level of abstraction so you don't need to deal with messy hardware, configuration selection remains hardware-adjacent. This is because performance still depends on choosing tile and launch parameters that fit the workload and target GPU. Common parameters include `BM`, `BN`, `BK`, `num_warps`, and `num_stages`.\n\nThere are two common ways to select configurations:\n\n#### [#](#1-autotune)1. [autotune](https://triton-lang.org/main/python-api/generated/triton.autotune.html)\n\nThe first uses `@triton.autotune` with a list of `triton.Config` objects.\n\nEach configuration can set compile-time keyword arguments such as block sizes, along with launch options such as `num_warps`, `num_stages`, and `num_ctas`. See the [` triton.Config` documentation](https://triton-lang.org/main/python-api/generated/triton.Config.html) for the full list.\n\nFor example, a kernel can use autotuning like this:\n\nExhaustively testing every possible configuration can take a long time. In practice, we provide a bounded set of plausible candidates. For larger search spaces, Triton can prune candidates through `prune_configs_by`, while `@triton.heuristics` or manual lookup tables can avoid runtime tuning when compilation and cold-start latency matter.\n\nThe `key` parameter in `@triton.autotune()` defines the runtime values that identify a tuning problem. Whenever one or more of those key values change, Triton evaluates the candidate configurations for the new key and caches the result.\n\n#### [#](#2-manual-lookup-tables-or-heuristics)2. Manual Lookup Tables or [Heuristics](https://triton-lang.org/main/python-api/generated/triton.heuristics.html)\n\nAlthough autotuning flexibly covers different shapes and dtypes, heuristics can sometimes select configurations with less runtime overhead.\n\nManual lookup tables or heuristics can be keyed on shape, dtype, device, and feature flags. They are useful in latency-sensitive inference serving, where runtime autotuning and compilation overhead matter.\n\n### [#](#a-few-more-triton-operations)A few more Triton operations\n\n#### [#](#tlwhere)`tl.where`\n\nIf the matmul fuses an activation such as [ReLU](https://en.wikipedia.org/wiki/Rectified_linear_unit), `tl.where` can be used as a per-element if-else on the output tile:\n\nBoth value branches of `tl.where` are evaluated, as noted in [the API documentation](https://triton-lang.org/main/python-api/generated/triton.language.where.html). Use masks on `tl.load` and `tl.store`, rather than `tl.where`, when a branch would otherwise perform an unsafe memory access.\n\n#### [#](#axis-0-and-axis-1)Axis = 0 and Axis = 1\n\nThe `axis` argument identifies the dimension to reduce, and that dimension is removed from the result. For a two-dimensional tensor shaped `[B0, B1]`, reducing along `axis=0` produces `[B1]`, while reducing along `axis=1` produces `[B0]`.\n\n**Example**\n\nsums each row across columns:\n\nSo it turns `[B0, B1]` into `[B0]`.\n\n## [#](#testing)Testing\n\nCorrectness comes before performance. Before benchmarking a Triton kernel, test it against a trusted reference implementation, typically PyTorch.\n\nTry testing with:\n\n- A representative shape.\n- A shape that is not divisible by the tile size, to verify the tail masks.\n- Every dtype and memory layout that the kernel claims to support.\n\n### [#](#example-3)Example\n\nFor example, a matmul test should use `M`, `N`, and `K` independently instead of testing only square matrices.\n\n`torch.testing.assert_close()` checks both absolute and relative error:\n\n- `atol` is the allowed absolute difference, which matters for values near zero.\n- `rtol` scales the allowed difference relative to the expected value.\n\nWe keep both `atol` and `rtol` because Triton and PyTorch may evaluate operations in a different order or at different intermediate precisions, producing small numerical differences.\n\n## [#](#performance-benchmarking)Performance Benchmarking\n\nOnce the kernel passes its tests, we can compare it against a meaningful baseline using the same shapes, dtypes, and device.\n\nGPU work is asynchronous, so timing a launch with a normal CPU timer can measure only the time required to enqueue the work. [triton.testing.do_bench](https://triton-lang.org/main/python-api/generated/triton.testing.do_bench.html) performs warmup and repeated GPU timing and returns the runtime in milliseconds.\n\n**Example**\n\nHere is a compact benchmark for matrix multiplication:\n\n**Latency (`ms` or `µs`)** is the clearest metric for small operations. **Throughput (`TFLOP/s`)** reports how quickly the matrix multiplication's floating-point work completes.\n\nFor a larger shape sweep with tables and plots, Triton also provides [triton.testing.Benchmark](https://triton-lang.org/main/python-api/generated/triton.testing.Benchmark.html) and [triton.testing.perf_report](https://triton-lang.org/main/python-api/generated/triton.testing.perf_report.html).\n\n## [#](#debugging)Debugging\n\n[Interpreter mode](https://triton-lang.org/main/programming-guide/chapter-3/debugging.html) simulates Triton programs sequentially on the CPU using NumPy equivalents, which makes intermediate values inspectable with Python `print()` or `pdb`.\n\nBecause programs execute sequentially on the CPU, interpreter mode does not reproduce GPU parallelism, scheduling, or performance and should not be used for benchmarking. It also has unsupported cases, including operations on `bfloat16` values and indirect memory-access patterns. Use it to inspect correctness, then test the kernel on the target GPU as well.\n\n## [#](#wrapping-up)Wrapping Up\n\nThis was a friendly introduction to Triton. Before finishing this note, let's review the core ideas:\n\n1. A Triton program is one instance of a compiled kernel. It owns one tile, or sometimes a sequence of tiles, loads the corresponding values, computes on them, and stores its outputs.\n2. `grid` defines how many programs launch along each axis. For example,`(1, 1, 1)` launches one program, while`(3, 2, 1)` launches six. In the latter grid,`tl.program_id(0)` ranges from`0` to`2` , and`tl.program_id(1)` ranges from`0` to`1` .\n3. Most non-persistent kernels map one program to one logical output tile. Persistent kernels may assign several tiles to each program. See [when one program processes multiple tiles](#optional-when-one-program-processes-multiple-tiles) .\n4. Each program uses its program ID to identify the tile it contributes to the output tensor.\n5. Before loading data, a program determines which logical tensor elements it owns and maps those coordinates to memory pointers.\n6. As a practical rule of thumb, expressions derived from program IDs, block coordinates, `tl.arange()` , and strides usually calculate indices, memory offsets, or pointers. The actual tensor values are obtained by reading from those locations with`tl.load()` , and computations on the data typically follow.", "url": "https://wpnews.pro/news/triton-notes", "canonical_source": "https://junupark.xyz/triton-notes/", "published_at": "2026-10-20 02:41:19+00:00", "updated_at": "2026-10-20 03:15:58.241289+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-infrastructure", "developer-tools"], "entities": ["Triton", "OpenAI", "CUDA", "PyTorch", "NumPy", "Blackwell"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/triton-notes", "markdown": "https://wpnews.pro/news/triton-notes.md", "text": "https://wpnews.pro/news/triton-notes.txt", "jsonld": "https://wpnews.pro/news/triton-notes.jsonld"}}