{"slug": "what-is-mlir-and-why-does-it-exist", "title": "What Is MLIR and Why Does It Exist?", "summary": "Chris Lattner created MLIR (Multi-Level Intermediate Representation) in 2018 at Google to solve the problem of fragmented compiler infrastructure across different hardware targets and programming models. MLIR, released publicly in 2019 under the LLVM umbrella, provides a common way to represent and transform code, reducing the need to build separate compilers for each new chip, language, or ML framework. The project lives inside the LLVM monorepo to leverage existing battle-tested building blocks.", "body_md": "If you've never written a compiler, the word \"MLIR\" probably looks like alphabet soup. This article is for you. By the end you'll understand, in plain language, *what* problem MLIR solves and *why* it had to exist at all.\n\nLet's start with the origin story — because where something comes from tells you almost everything about what it's for.\n\nThe story of MLIR starts in 2018 at Google. Chris Lattner, one of the most influential figures in compiler engineering, set out to solve a problem that had been bothering the industry for years — there was no common way to represent and transform code across different hardware targets and programming models. MLIR was his answer, and it went public in 2019 under the LLVM umbrella.\n\nImagine you work on TensorFlow, Google's machine learning library. Your job is to take a model someone wrote in Python and make it run *fast* — on a laptop CPU, on a phone, on a GPU, and on Google's custom TPU chips. To do that, the model has to be translated, step by step, into instructions each piece of hardware understands. That translation-and-optimization process is, fundamentally, a **compiler**.\n\nThe trouble was that there wasn't *one* compiler. There were many. One team built a tool to optimize graphs. Another built a separate tool to target TPUs. Another for mobile. Another for a specific hardware accelerator. Each tool had its own way of representing the program internally, its own bugs, its own optimization tricks that couldn't be shared with the others. The ecosystem was **siloed** — a pile of separate, half-overlapping compilers all reinventing the same wheels.\n\nAnd this wasn't unique to Google. Across the industry, the same pattern kept repeating: a new chip, a new language, or a new ML framework would appear, and someone would sit down to build *yet another* compiler from scratch to support it. Everybody was paying the same enormous bill, over and over.\n\nChris Lattner moved to Google in 2017 to lead the TensorFlow infrastructure team, walked straight into that fragmentation mess, and built MLIR to fix it.\n\nMLIR stands for **Multi-Level Intermediate Representation**. Hold onto that name — every word in it is doing real work, and we'll unpack it as we go. The official paper describes the goals directly: reduce software fragmentation, improve compilation for the wild variety of modern hardware, dramatically lower the cost of building domain-specific compilers, and help existing compilers connect to one another.\n\nA small but telling detail:MLIR doesn't live in its own separate project. It was addedinsidethe LLVM monorepo (`llvm-project`\n\n) in a folder literally called`mlir/`\n\n. Why? Because LLVM already had two decades of battle-tested, reusable building blocks — data structures, error handling, a testing framework — and Lattner knew that codebase better than anyone alive. Starting from zero would have meant rebuilding all of that. Sitting inside the monorepo, MLIR could borrow it on day one.\n\nBefore we get to the machine-learning payoff, we need a shared mental model of what a compiler actually *does*. Let's build that with the simplest possible program.\n\nWhen you compile a program, your code goes on a journey through several stages:\n\n```\nSource code\n   → Frontend (parsing)\n      → AST (a tree of your program)\n         → IR (intermediate representation)\n            → Optimization passes (run in a loop)\n               → Lowering (toward the machine)\n                  → Backend (per-CPU details)\n                     → Code generation (actual machine code)\n```\n\nDon't worry about memorizing it. The three ideas that matter are:\n\nLet's trace a single expression — `x = 1 + 2`\n\n— through all three.\n\nFor instance, when you run a `.py`\n\nfile, the very first thing CPython does is break raw text into **tokens** — the smallest meaningful chunks of the language.\n\n``` python\nimport tokenize, io\n\nsource = \"x = 1 + 2\"\n\ntokens = tokenize.generate_tokens(io.StringIO(source).readline)\nfor tok in tokens:\n    print(tok)\n```\n\nOutput:\n\n```\nTokenInfo(type=1  (NAME),   string='x',  ...)\nTokenInfo(type=54 (OP),     string='=',  ...)\nTokenInfo(type=2  (NUMBER), string='1',  ...)\nTokenInfo(type=54 (OP),     string='+',  ...)\nTokenInfo(type=2  (NUMBER), string='2',  ...)\n```\n\nSo `x = 1 + 2`\n\nstops being an opaque string and becomes a flat list of typed pieces. The tokenizer doesn't care about *meaning* yet — it just answers: **\"what kind of thing is this character sequence?\"**\n\nNext, the **parser** takes that flat list of tokens and builds an **AST** (Abstract Syntax Tree) — a nested structure that captures the *grammar* of your program.\n\n``` python\nimport ast\n\ntree = ast.parse(\"x = 1 + 2\")\nprint(ast.dump(tree, indent=2))\n```\n\nOutput:\n\n```\nModule(body=[\n  Assign(\n    targets=[Name(id='x')],\n    value=BinOp(\n      left=Constant(value=1),\n      op=Add(),\n      right=Constant(value=2)))])\n```\n\nThe flat sequence `1 + 2`\n\nbecame a `BinOp`\n\nnode with an `Add`\n\noperator and two children. The structure of the expression is now *explicit* in the shape of the tree — not buried in the order of characters. This tree is what gets handed off to the next stage. The compiler never looks at your source text again.\n\nNext, `compile()`\n\ntakes the AST and produces **bytecode** — CPython's IR. The optimizer runs between the two, applying any transformations it can find. Here it applied **constant folding**: since both operands are literals, `1 + 2`\n\ncan be solved at compile time. The runtime never sees the addition at all.\n\n``` python\nimport ast, dis\n\nsource = \"x = 1 + 2\"\n\ntree = ast.parse(source)       # Stage 1 — AST\ncode = compile(source, \"<string>\", \"exec\")  # Stage 2 — bytecode\ndis.dis(code)\n```\n\nOutput:\n\n```\n  1           0 RESUME          0\n              2 LOAD_CONST      0 (3)   ← already computed\n              4 STORE_NAME      0 (x)\n              6 RETURN_CONST    1 (None)\n```\n\n`1`\n\nand `2`\n\nare gone. Only `3`\n\nremains.\n\nThe backend is the most complex part of any compiler and deserves its own article. For now, just one thing worth seeing: after all the stages above, `x = 1 + 2`\n\neventually becomes exactly **two x86 instructions**:\n\n```\nmov eax, 3   ; load the result (already computed at compile time)\nret          ; return it\n```\n\nThat's it. The CPU never sees `1`\n\nor `2`\n\n— only `3`\n\n.\n\nCPython itself doesn't go this far. It stops at bytecode and interprets it via a virtual machine in\n\n`ceval.c`\n\n. JIT compilers like PyPy or Numba go all the way to machine code like the snippet above.\n\nThe Python example showed the pipeline from the outside. Let's now watch the optimizer do something slightly more interesting — remove code that will never matter.\n\nHere's a small C++ program with a deliberate mistake:\n\n```\n#include <iostream>\n#include <string>\n\nint main() {\n    std::string dead = \"I am never used\";  // created, then never read\n    std::cout << \"Hello world\\n\";\n    return 0;\n}\n```\n\nThat `dead`\n\nvariable is **dead code**: we build it, then never read it. A human reviewer would say \"just delete that line.\" We're going to watch the compiler figure that out on its own.\n\nThe AST captures the *structure* of your code with all the punctuation and formatting stripped away. For brevity, the `#include`\n\nmachinery is omitted — it expands into a lot of generated declarations. The meaningful structure of `main`\n\nlooks like this:\n\n``` php\nFunctionDecl: main -> int\n└── CompoundStmt\n    ├── DeclStmt\n    │   └── VarDecl: dead : std::string = \"I am never used\"\n    ├── CallExpr: operator<<\n    │   └── (std::cout << \"Hello world\\n\")\n    └── ReturnStmt\n        └── IntegerLiteral: 0\n```\n\nThe tree is faithful to what you wrote — warts and all. The `dead`\n\nvariable is still there. Cleanup comes later.\n\nThe compiler then converts the AST into **Intermediate Representation (IR)**. Real IR for a `std::string`\n\nprogram is genuinely noisy, so let's switch to a simpler version of the same idea:\n\n```\nint compute() {\n    int unused = 99;   // dead variable\n    int a = 2;\n    int b = 3;\n    return a + b;\n}\n```\n\nWith optimizations **off**, the LLVM IR looks like this (simplified):\n\n```\ndefine i32 @compute() {\nentry:\n  %unused = alloca i32\n  %a      = alloca i32\n  %b      = alloca i32\n  store i32 99, i32* %unused   ; unused = 99\n  store i32 2,  i32* %a        ; a = 2\n  store i32 3,  i32* %b        ; b = 3\n  %0   = load i32, i32* %a\n  %1   = load i32, i32* %b\n  %add = add i32 %0, %1        ; a + b\n  ret i32 %add\n}\n```\n\nVerbose, but readable: reserve some slots, store numbers, add two of them, return the result. Every line of your source has a faithful echo — including the pointless `unused = 99`\n\n.\n\nNow we turn optimizations **on**. The compiler runs a series of **optimization passes** — small, focused transformations applied in a loop until nothing more can be improved. Two run here:\n\n`2 + 3`\n\nis always `5`\n\n. No reason to compute it at runtime.`unused`\n\nis written but never read. No one depends on it, so it's deleted.The result:\n\n```\ndefine i32 @compute() {\nentry:\n  ret i32 5\n}\n```\n\nThe whole function became \"return 5.\" The dead variable vanished and the arithmetic was solved at compile time. *That* is what the compiler's middle stage is for — and it's exactly the kind of work MLIR is built to make easy across many different kinds of programs.\n\nGo to ** godbolt.org**. Paste in C++ (or dozens of other languages), pick a compiler, and watch the output update in real time as you toggle between\n\n`-O0`\n\n(no optimization) and `-O2`\n\n(optimize hard). Watching dead code evaporate is the fastest way to build intuition for everything above. It's the single best companion to this article.So if LLVM is such a great compiler infrastructure, why couldn't TensorFlow just *use* it directly?\n\nHere's the catch. LLVM's IR was designed to describe programs at the level of **CPU instructions** — load this number, add these two registers, jump to that address. That's the right level for compiling C or Rust. But it's far too *low* for machine learning.\n\nA neural network doesn't think in \"add two registers.\" It thinks in operations like **\"do a 2D convolution\"** or **\"apply softmax\"** or **\"multiply these two matrices.\"** If you flatten all of that down to individual CPU instructions too early, you throw away the high-level meaning — and with it, the chance to do the *big* optimizations that only make sense when you can still see \"oh, these two matrix multiplications could be fused together.\"\n\nThis is the core insight behind the **\"Multi-Level\"** in MLIR. Instead of one fixed IR, MLIR lets you have **many IRs at different levels of abstraction**, and lower your program gradually:\n\n```\nHigh level:   \"matmul\", \"convolution\", \"softmax\"   ← ML-shaped operations\n    ↓\nMid level:    loops, array indexing, linear algebra\n    ↓\nLow level:    LLVM IR  →  actual CPU / GPU / TPU instructions\n```\n\nEach level is called a **dialect** in MLIR — a self-contained vocabulary of operations suited to one kind of reasoning. You optimize at the level where it's natural, *then* lower to the next. The philosophy in one sentence: a big compiler should be broken into many small compilers between intermediate languages, each designed to make one kind of optimization easy to express.\n\nLLVM couldn't be stretched to do this: it was designed for CPUs, sat at too low a level of abstraction, and carried years of incidental baggage. But it had all those reusable pieces worth keeping. MLIR is what you get when you keep the good parts and add the missing \"multi-level\" idea on top.\n\nLet's make it concrete. Suppose we're training a network to recognize handwritten **letters of the alphabet** (26 classes, A–Z). In Keras the model is just a few lines:\n\n``` python\nimport tensorflow as tf\n\nmodel = tf.keras.Sequential([\n    tf.keras.layers.Flatten(input_shape=(28, 28)),\n    tf.keras.layers.Dense(128, activation='relu'),\n    tf.keras.layers.Dense(26, activation='softmax'),\n])\n```\n\nInnocent-looking. But under the hood, running this model is a chain of math operations on large grids of numbers. To make it fast on real hardware, a compiler has to take it through exactly the kind of multi-level lowering we just described.\n\nQuick detour, because the word is everywhere (it's literally in \"TensorFlow\"). A **tensor** is just a container of numbers with a shape:\n\n`7`\n\n) → `[1, 2, 3]`\n\n) → For our purposes: **a tensor is a matrix of numbers, and in a neural network, those numbers are the weights the model learned during training.** When the model recognizes a letter, your input image (a tensor) gets multiplied by weight tensors, over and over, until it produces 26 scores — one per letter.\n\nWhen that Keras model is fed into an MLIR-based compiler, the high-level operations get represented in a dialect with explicit tensor types. Below is a simplified but syntactically real sketch of the `Dense`\n\nlayer — a matrix multiply followed by a bias add:\n\n```\n// Input: one flattened image (784 = 28×28 numbers)\nfunc.func @dense(%input:   tensor<1x784xf32>,\n                 %weights: tensor<784x128xf32>,\n                 %bias:    tensor<1x128xf32>) -> tensor<1x128xf32> {\n\n  %0 = \"tosa.matmul\"(%input, %weights)\n        : (tensor<1x784xf32>, tensor<784x128xf32>) -> tensor<1x128xf32>\n\n  %1 = \"tosa.add\"(%0, %bias)\n        : (tensor<1x128xf32>, tensor<1x128xf32>) -> tensor<1x128xf32>\n\n  return %1 : tensor<1x128xf32>\n}\n```\n\nLook at the types: `tensor<1x784xf32>`\n\nmeans \"a tensor shaped 1 × 784 of 32-bit floats.\" The compiler can *see* the shapes and the high-level operations (`matmul`\n\n, `add`\n\n), which means it can reason about them — fuse operations, reorder them, choose the optimal memory layout for a TPU — all *before* lowering everything down to LLVM IR and finally to machine code.\n\nThat's the whole point. The dead-code-elimination trick we watched earlier was a tiny optimization on a tiny program. MLIR is the framework that lets you apply that same *style* of optimization to machine-learning-shaped programs, at the right level of abstraction, for whatever hardware you're targeting — without building a brand-new compiler from scratch every single time.\n\nWe've covered the *why* — deliberately staying at altitude:\n\nIn the next articles we'll get our hands dirty: setting up an MLIR project, reading and writing real dialects, running an actual lowering pass, and seeing the `mlir-opt`\n\ntool transform code live.\n\nIf you want a head start, the [MLIR tutorial series by Jeremy Kun](https://www.jeremykun.com/2023/08/10/mlir-getting-started/) and the [official MLIR docs](https://mlir.llvm.org/) are excellent next stops.\n\n**The one idea worth keeping:** MLIR exists because the world kept building the same compiler over and over. It's the reusable, multi-level foundation that makes that stop.", "url": "https://wpnews.pro/news/what-is-mlir-and-why-does-it-exist", "canonical_source": "https://dev.to/frodo/what-is-mlir-and-why-does-it-exist-4d78", "published_at": "2026-07-01 09:00:00+00:00", "updated_at": "2026-07-01 09:19:36.399980+00:00", "lang": "en", "topics": ["machine-learning", "developer-tools", "ai-infrastructure"], "entities": ["Chris Lattner", "Google", "MLIR", "LLVM", "TensorFlow", "TPU"], "alternates": {"html": "https://wpnews.pro/news/what-is-mlir-and-why-does-it-exist", "markdown": "https://wpnews.pro/news/what-is-mlir-and-why-does-it-exist.md", "text": "https://wpnews.pro/news/what-is-mlir-and-why-does-it-exist.txt", "jsonld": "https://wpnews.pro/news/what-is-mlir-and-why-does-it-exist.jsonld"}}