{"slug": "mlir-to-rtl-simulation-flow-from-linalg-matmul-to-a-systolic-array", "title": "MLIR-to-RTL Simulation Flow: From linalg.matmul to a Systolic Array", "summary": "A developer built hw-mlir-lab, an open MVP that lowers a linalg.matmul operation in MLIR through tiling and padding into a custom standalone.systolic_matmul op, then to a C ABI call executed against a SystemVerilog 8x8 systolic array simulated with cocotb/Verilator. The project demonstrates a full compiler-to-RTL path for prototyping on-chip AI accelerator subsystems, and the developer plans to expand it into a framework with MLIR pass libraries, IP blocks, runtime interfaces and CIRCT exploration.", "body_md": "Hi! I'd like to share the MVP of my [hw-mlir-lab](https://github.com/Dima365/hw-mlir-lab/tree/matmul_demo) project, where I use MLIR to lower matrix multiplication (`matmul`) to a systolic array simulated in Verilator.\n\nIn modern systems-on-chip, more and more computation is moving to specialized hardware blocks for AI, communications, DSP, image processing, scientific computing, and other workloads. In these systems, the CPU core often focuses on control, while dedicated accelerator IP blocks handle the heavy computation.\n\nBut designing a fast hardware block is only half the challenge. The software stack also needs to use it efficiently. Overall performance depends not only on clock frequency and RTL microarchitecture, but also on tiling, padding, data layout, command formats, the runtime interface, memory organization, and how the compiler flow decides which operations to offload to the accelerator.\n\nThis creates an awkward gap. The hardware team typically focuses on the RTL block, interfaces, timing, and resource usage. The software team focuses on the compiler, runtime, APIs, and application integration. Yet many decisions sit right between these two worlds. For example, it is difficult to choose a good tile size or data format by looking only at the compiler side or only at the hardware side.\n\nI wanted to build a minimal project that would let me explore the entire chain: from a high-level operation in MLIR to its execution on an RTL model of an accelerator.\n\nThe current MVP validates one specific end-to-end path: `linalg.matmul` in MLIR is transformed to fit a fixed-size 8x8 systolic array, replaced with a custom MLIR operation, lowered to a C ABI call, and executed through a cocotb/Verilator simulation of the RTL block.\n\nMLIR provides a convenient foundation for this compiler flow: start with a high-level operation, apply transformation passes, introduce a hardware-specific operation, and then lower it to a call to a specific accelerator.\n\nGoing forward, I want to develop the project into an open framework for prototyping on-chip accelerator subsystems, with a library of MLIR passes, IP blocks, runtime interfaces, RTL simulation, tests, integration with synthesis tools, and exploration of CIRCT.\n\nThe project currently implements one specific end-to-end path: from `linalg.matmul` in MLIR to matrix multiplication on an RTL model of a systolic array in a cocotb/Verilator simulation.\n\nA simplified view of the flow looks like this:\n\n```\nMLIR input\n  |\n  v\nMLIR transform pipeline\n  |\n  v\ncustom standalone dialect / passes\n  |\n  v\nLLVM lowering\n  |\n  v\nnative executable\n  |\n  v\nC interface bridge\n  |\n  v\ncocotb / Verilator\n  |\n  v\nSystemVerilog systolic array RTL\n```\n\nThe input is an MLIR program containing `linalg.matmul`. This is a standard high-level linear algebra operation that is not yet tied to any particular hardware block.\n\nThe MLIR transformation pipeline then brings it into a form suitable for the current demo accelerator: the matmul is split into 8x8 tiles, with padding applied to handle boundary cases. This is necessary because the RTL systolic array in the MVP expects fixed-size inputs.\n\nNext, matching operations are replaced with a custom MLIR operation:\n\n```\nstandalone.systolic_matmul\n```\n\nThis operation serves as an explicit marker: this part of the computation must run on an external hardware block rather than as a regular CPU matmul.\n\nFor project-specific transformations, I use `standalone-opt`. This is an extended version of the MLIR standalone example: MLIR provides an example out-of-tree dialect along with an opt-like tool that can serve as a starting point for your own dialect and passes. In my case, I extended this template with the operations and transformations needed to turn matching `linalg.matmul` operations into calls to the systolic array.\n\nNext, `standalone.systolic_matmul` is lowered to a standard C-callable interface:\n\n```\nsystolic_matmul_8x8(...)\n```\n\nThis is an important boundary in the project. To the generated program, the accelerator looks like an ordinary C function. At the moment, that function is backed by a simulation bridge, but a different runtime mechanism could take its place in the future: MMIO, DMA, an FPGA driver, or an interface to a physical ASIC/FPGA prototype.\n\nThe current C bridge passes data to the cocotb testbench through a Unix socket. It sends two input tiles of type `i8[8][8]` and an accumulator of type `i32[8][8]`, then receives the updated result. The cocotb testbench accepts the request, feeds the data into the SystemVerilog systolic array module, waits for the computation to finish, and returns the result to the program.\n\nThe MVP therefore explicitly separates three levels:\n\n```\nMLIR level     — representing and transforming the computation\nC ABI level    — the boundary between generated code and accelerator calls\nRTL level      — executing the operation on a hardware block model\n```\n\n`linalg.matmul` Becomes a Call to the Systolic Array\nNow let's look at the central part of the project: how a regular `linalg.matmul` gradually becomes a call to a specific hardware block.\n\nThe MLIR pipeline in the MVP is divided into several explicit stages. I deliberately save intermediate files after each step so that it is possible to inspect how the IR changes:\n\n```\n01_tiled.mlir\n02_padded.mlir\n03_bufferized.mlir\n04_entry_wrapped.mlir\n05_systolic_memref.mlir\n06_systolic_call.mlir\n07_llvm.mlir\n08_llvm.ll\n```\n\nThe input is an MLIR program containing `linalg.matmul`. At this level, the operation knows nothing about the systolic array, RTL, cocotb, or the C ABI. It is simply a high-level description of matrix multiplication.\n\nThe first stage is tiling. Using the Transform dialect schedule in `matmul_tile.mlir`, the original `linalg.matmul` is split into 8x8x8 tiles. This size is intentional: the current demo systolic array operates on a fixed-size 8x8 block, so the compiler pipeline must bring the computation into a form that the hardware can accept.\n\nThe next stage is padding. If the original matrix dimensions are not multiples of 8, boundary tiles appear. The transformation schedule in `matmul_pad.mlir` handles these cases by padding small or boundary tiles to the fixed size. As a result, the runtime can always send data of the same shape to the systolic array.\n\nBufferization comes next. Up to this point, the IR may use a tensor-level representation. The `one-shot-bufferize` pass converts the data to the memref level: the operands are now represented as buffers in memory. This is an important transition because the flow ultimately needs to reach the C ABI and a regular native executable.\n\nA C interface entry wrapper is then created. It allows the C driver—the file containing `main`—to call the generated MLIR function through a stable external entry point. In the current project, a wrapper is created for `matmul_entry`; it accepts the input memrefs and an output memref, calls the main MLIR function, and returns the result through the supplied output buffer.\n\nOnce the memref-level IR is ready, the project-specific stage begins: conversion to a systolic operation.\n\nThe custom pass:\n\n```\n--convert-linalg-matmul-to-systolic\n```\n\nfinds matching `linalg.matmul` operations and replaces them with:\n\n```\nstandalone.systolic_matmul\n```\n\nThe pass currently expects a specific operation shape:\n\n```\nlhs: memref<8x8xi8>\nrhs: memref<8x8xi8>\nacc: memref<8x8xi32>\n```\n\nIn other words, this is no longer a general matmul of arbitrary size and type. It is a hardware-specific operation prepared for the current RTL block.\n\nThe next custom pass is:\n\n```\n--lower-systolic-to-func-call\n```\n\nIt removes `standalone.systolic_matmul` and replaces it with a regular function call:\n\n```\nsystolic_matmul_8x8(...)\n```\n\nThis is the key point in the entire flow. Before this stage, the operation is still represented in MLIR as an accelerator-specific operation. After this stage, it is a regular call through a C-compatible interface that can be linked with the runtime/interface code.\n\nIn brief, the central part of the pipeline looks like this:\n\n```\nlinalg.matmul\n  |\n  | tiling: 8x8x8\n  v\ntiled linalg.matmul\n  |\n  | padding\n  v\nfixed-size matmul tiles\n  |\n  | bufferization\n  v\nmemref-level IR\n  |\n  | convert-linalg-matmul-to-systolic\n  v\nstandalone.systolic_matmul\n  |\n  | lower-systolic-to-func-call\n  v\nfunc.call @systolic_matmul_8x8\n```\n\nAfter that, the remaining MLIR dialects are lowered to the LLVM dialect, and `mlir-translate` converts it to standard LLVM IR. The compilation pipeline then compiles this LLVM IR into an object file, compiles `interface.c`, and links everything together with the C driver into a native executable.\n\nAt runtime, the `systolic_matmul_8x8` function sends the data to the cocotb/Verilator testbench and returns the result.\n\nThe generated program therefore looks like a regular native application, but some of its matmul operations are actually executed on an RTL model of the accelerator.\n\nTo make the example easy to run and experiment with, the project is set up to run in Docker: LLVM/MLIR, Clang, Verilator, cocotb, and the custom `standalone-opt` all run inside a preconfigured environment.\n\nThe main demo is launched with a single command:\n\n```\nmake demo\n```\n\nThis builds the MLIR tools, runs the demo program through the compiler pipeline, compiles a native executable, and launches it alongside the cocotb/Verilator simulation of the RTL systolic array.\n\nAfter the run, you can inspect the intermediate artifacts from the MLIR pipeline:\n\n```\nbuild/mlir-pipeline/01_tiled.mlir\nbuild/mlir-pipeline/02_padded.mlir\nbuild/mlir-pipeline/03_bufferized.mlir\nbuild/mlir-pipeline/04_entry_wrapped.mlir\nbuild/mlir-pipeline/05_systolic_memref.mlir\nbuild/mlir-pipeline/06_systolic_call.mlir\nbuild/mlir-pipeline/07_llvm.mlir\nbuild/mlir-pipeline/08_llvm.ll\n```\n\nThis is useful for debugging: you can see where `linalg.matmul` was tiled, where `standalone.systolic_matmul` appeared, and where it became a regular function call.\n\nTo check multiple cases, run the tests:\n\n```\nmake test\n```\n\nThe tests generate several matmul cases with different dimensions and data patterns, run them through the same flow, and compare the results against the expected outputs.\n\nThe current MVP is deliberately limited to a single path: `linalg.matmul` → 8x8 systolic array → RTL simulation. This makes it possible to validate the entire flow end to end more quickly.\n\nThe first limitation is the hardware target itself. The project currently supports a fixed-size systolic array operating on 8x8 tiles. The inputs use `i8`, while the accumulator and result use `i32`. This is enough to demonstrate the idea, but it does not yet cover other data types, quantization scenarios, or more complex memory layouts.\n\nThe second limitation is the runtime layer. The `systolic_matmul_8x8` function currently communicates with the cocotb testbench through a Unix socket. This is a convenient simulation transport for the MVP, but another system could use MMIO, DMA, a driver layer, a command queue, or a different accelerator communication mechanism in its place.\n\nThe project also does not yet include a full memory model, an interconnect, a DMA scheduler, a cost model, or automatic selection of operations that would benefit from offloading. The pipeline currently selects only matching `linalg.matmul` operations that conform to the expected shape after tiling and padding.\n\nThere are several directions in which the project could develop.\n\nThe first is expanding the set of supported operations and IP blocks. Right now, the only hardware target is the demo systolic array for matmul. Other accelerators could be integrated in the future: DSP blocks, FFT engines, stencil-compute blocks, hash cores, and other computational IP. Each block could have its own MLIR operation, lowering pass, runtime interface, and cocotb/Verilator tests.\n\nThe second is a more advanced compiler flow. This includes hardware constraints, a cost model, selection between CPU and accelerator paths, and support for different data types and memory layouts.\n\nThe third is the runtime and integration with physical hardware. The socket bridge can be viewed as the initial simulation transport. The next step is a runtime interface that can be backed by MMIO, DMA, or an FPGA prototype.\n\nThe fourth is synthesis and CIRCT. The RTL is currently handwritten and verified in simulation. I would like to add a synthesis sanity check to obtain basic resource and timing reports, and explore how CIRCT could help generate wrappers, interfaces, control logic, or other parts of the hardware infrastructure.\n\nMore broadly, I want to develop the project into an open framework for prototyping on-chip accelerator subsystems. Not as a replacement for HLS or a ready-made ASIC compiler, but as a reproducible environment for experimenting with compiler passes, IP blocks, runtime interfaces, and RTL simulation within a single flow.\n\nI would welcome feedback from anyone working with MLIR/LLVM, RTL, FPGA/ASIC, cocotb/Verilator, EDA tooling, or hardware accelerators. Contributions, issues, architecture discussions, and ideas for new IP blocks or compiler/runtime use cases are all welcome from anyone interested in getting involved.", "url": "https://wpnews.pro/news/mlir-to-rtl-simulation-flow-from-linalg-matmul-to-a-systolic-array", "canonical_source": "https://dev.to/dima365/mlir-to-rtl-simulation-flow-from-linalgmatmul-to-a-systolic-array-1gme", "published_at": "2026-10-17 13:22:49+00:00", "updated_at": "2026-10-17 13:34:19.614826+00:00", "lang": "en", "topics": ["ai-chips", "ai-infrastructure", "mlops", "developer-tools", "ai-research"], "entities": ["MLIR", "Verilator", "cocotb", "LLVM", "CIRCT", "hw-mlir-lab"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/mlir-to-rtl-simulation-flow-from-linalg-matmul-to-a-systolic-array", "markdown": "https://wpnews.pro/news/mlir-to-rtl-simulation-flow-from-linalg-matmul-to-a-systolic-array.md", "text": "https://wpnews.pro/news/mlir-to-rtl-simulation-flow-from-linalg-matmul-to-a-systolic-array.txt", "jsonld": "https://wpnews.pro/news/mlir-to-rtl-simulation-flow-from-linalg-matmul-to-a-systolic-array.jsonld"}}