{"slug": "catching-nan-at-the-mlir-pass-boundary", "title": "Catching NaN at the MLIR Pass Boundary", "summary": "An MLIR pass can fold 0 * Inf into a quiet NaN attribute during constant folding, producing structurally valid but numerically meaningless IR that surfaces later as a distant accuracy regression in an NPU runtime, according to a technical account of the failure. The IEEE 754 standard defines 0 * Inf as NaN, and because NaN compares unequal to itself, a range check such as x < min || x > max accepts it as in-range. The recommended fix is to trace backward to the first bad pass, correct its arithmetic, and add a local operation verifier that enforces the numeric contract after every pass, since a runtime check observes only the symptom after the compiler/runtime boundary.", "body_md": "A Multi-Level Intermediate Representation (MLIR) pass can create a not-a-number (NaN) attribute during constant folding. This can happen even when every operand starts as a valid value.\n\nThe compiler should reject that value at the earliest intermediate representation (IR) boundary. Otherwise, the runtime exposes it later as a distant accuracy regression.\n\nThe shortest reliable workflow traces the first bad pass and fixes its arithmetic. A local operation verifier then turns the operation's numeric contract into a check after every pass.\n\nI write fusion passes that reduce work in a neural processing unit (NPU) runtime. A later regression reports a lower accuracy metric on a held-out evaluation set, but the report does not identify the fusion pass.\n\nI trace the bad values backward through the model and runtime. A fused operation's output attribute already contains a quiet NaN before the runtime touches it.\n\nThe fusion pass folds `0 * Inf` into that attribute. The runtime consumes the\nattribute and propagates the NaN through the remaining computation.\n\nThe operation still type-checks, and its operands and results still align. The compiler therefore produces structurally valid but numerically meaningless IR.\n\nLLVM represents compile-time floating-point values with [`APFloat`](https://llvm.org/doxygen/classllvm_1_1APFloat.html). This type\nsupports multiple floating-point formats and explicit rounding modes, so\ncompiler code does not need to depend on the host machine's native\nfloating-point behavior.\n\nThe Institute of Electrical and Electronics Engineers (IEEE) 754 standard\ndefines `0 * Inf` as NaN. Each operand can carry a valid meaning on its own, but\ntheir product has no numeric result.\n\nOther arithmetic paths can create the same class of value:\n\n`Inf - Inf` produces NaN because the difference has no defined value.`Inf / Inf` and `0 / 0` produce NaN because neither ratio has a defined value.\nCommon arithmetic operations propagate an existing NaN. One bad fold can therefore spread through many downstream operations before the runtime reports a visible failure.\n\nNaN also compares unequal to itself. Every ordered comparison against NaN\nreturns false, so a range check such as `x < min || x > max` accepts NaN as\nthough it falls inside the range.\n\nA direct finiteness check catches both NaN and infinity. A range check cannot replace that contract.\n\nA hardware description language (HDL) simulator uses `X` to represent an unknown\nlogic state. An uninitialized register or timing violation can introduce one\n`X`, and downstream logic can propagate it far from its source.\n\nA NaN follows the same debugging shape. The final observation provides propagation evidence, while the first transition from a valid value to NaN identifies the defect.\n\nThe analogy stops at propagation. `X` represents simulator uncertainty rather\nthan a physical third logic value, while IEEE 754 defines NaN as a\nfloating-point value with specified comparison and arithmetic behavior.\n\nThis distinction does not change the debugging rule. Trace backward to the first source.\n\n[Synopsys describes](https://www.synopsys.com/blogs/chip-design/debugging-x-can-be-difficult.html) the same process for register-transfer-level (RTL) and\ngate-level X propagation. An engineer follows drivers and fan-in signals until\nthe earliest `X` occurs.\n\nThe quiet NaN sits in an operation attribute before the graph reaches the runtime. The compiler can inspect the value at the exact boundary where the fusion pass creates it.\n\nA runtime numeric check observes the symptom after the value crosses the compiler/runtime boundary. It cannot identify which compiler pass first writes the attribute.\n\nThe [MLIR developer guide](https://mlir.llvm.org/getting_started/DeveloperGuide/#ir-verifier) defines a contract for every pass. Each pass can assume\nvalid input IR, and each pass must return valid output IR.\n\nThe pass manager enforces this contract between passes by default. Callers can disable per-pass verification, but the valid-input, valid-output convention still defines correct pass behavior.\n\nPass-boundary verification checks a pass's final output. A rewrite can use a transient invalid state internally, but it must restore all invariants before it returns.\n\nMLIR also tells operation verifiers to inspect local properties. A verifier can check the value of the operation's own attribute without following producers or consumers.\n\nThis local rule preserves transformation freedom. It also limits rejection to an invariant that the operation itself defines.\n\nAn MLIR pass pipeline runs in a fixed order. The first pass whose output contains NaN marks the source boundary.\n\nDuring initial triage, `-mlir-print-ir-after-all` prints IR after every pass.\nOnce the output reveals the suspect pass, targeted flags show the two states\nthat matter:\n\n```\n-mlir-print-ir-before=<pass>\n-mlir-print-ir-after=<pass>\n```\n\nThe `-mlir-print-ir-after-change` flag suppresses output for passes that leave\nthe IR unchanged. This option reduces noise in pipelines where many passes do\nnot affect a given input.\n\nThe private operation, attribute, and pass names stay private. The following fictional quantization-rescale operation preserves the relevant structure:\n\n```\n// Last-good IR before the fusion pass.\n%0 = \"quant.rescale_fuse\"(%input) {scale = 2.500000e-01 : f32}\n     : (tensor<1x64x56x56xf32>) -> tensor<1x64x56x56xf32>\n\n// First-bad IR after the fusion pass folds 0 * inf.\n%0 = \"quant.rescale_fuse\"(%input) {scale = 0x7FC00000 : f32}\n     : (tensor<1x64x56x56xf32>) -> tensor<1x64x56x56xf32>\n```\n\nThe operand, result type, and shape remain identical. Only the `scale` attribute\nchanges from a finite quarter-scale factor to the bit pattern for a quiet NaN.\n\nA type checker or shape-inference pass accepts both forms. The operation needs a numeric invariant to reject the second form.\n\nFixing the `0 * Inf` fold removes the current source. It does not stop another\npass, importer, or rewrite pattern from assigning NaN to the same attribute.\n\nAn operation verifier states the lasting contract. For the fictional\n`quant.rescale_fuse` operation, `scale` represents a multiplicative quantization\nfactor in the finite positive reals.\n\nThat meaning makes finiteness and positivity intrinsic to the operation. Every producer must honor the same constraints.\n\nIf an operation permits NaN in general but one fusion pass must not produce it, the pass owns the check instead. An operation verifier must not reject values that the operation's semantics allow.\n\nThe Operation Definition Specification (ODS) enables a custom verifier with one declaration:\n\n``` js\ndef RescaleFuseOp : Quant_Op<\"rescale_fuse\", []> {\n  let arguments = (ins AnyTensor:$input, F32Attr:$scale);\n  let results = (outs AnyTensor:$output);\n  let hasVerifier = 1;\n}\n```\n\nThe C++ implementation checks the complete attribute contract:\n\n``` js\nLogicalResult RescaleFuseOp::verify() {\n  const llvm::APFloat &scale = getScaleAttr().getValue();\n  if (!scale.isFinite() || scale.isNegative() || scale.isZero())\n    return emitOpError() << \"scale attribute must be a finite, positive value\";\n  return success();\n}\n```\n\n`APFloat::isNaN()` rejects quiet and signaling NaN encodings, but it accepts\npositive and negative infinity. `APFloat::isFinite()` rejects NaN and both\ninfinities.\n\nThe operation's semantics determine which query fits. A quantization scale cannot use infinity, so this verifier checks finiteness and positivity.\n\nThe [ODS documentation](https://mlir.llvm.org/docs/DefiningDialects/Operations/#custom-verifier-code) defines the verification order. Structural traits run\nfirst, generated invariant checks validate attributes and types next, and the\ncustom verifier runs after those checks.\n\nThis order lets `verify()` call `getScaleAttr().getValue()` directly. The\ngenerated checks already establish the attribute's presence and type.\n\nThe [MLIR testing guide](https://mlir.llvm.org/getting_started/TestingGuide/#diagnostic-tests) documents `-verify-diagnostics` tests for operation\ninvariants. One negative case proves that the verifier rejects NaN and preserves\nthe diagnostic:\n\n```\n// RUN: mlir-opt %s -split-input-file -verify-diagnostics\n\nfunc.func @rescale_rejects_nan(%arg0: tensor<4xf32>) -> tensor<4xf32> {\n  // expected-error@+1 {{scale attribute must be a finite, positive value}}\n  %0 = \"quant.rescale_fuse\"(%arg0) {scale = 0x7FC00000 : f32}\n       : (tensor<4xf32>) -> tensor<4xf32>\n  return %0 : tensor<4xf32>\n}\n```\n\nThe surrounding operation tests should retain at least one valid scale case. Together, the cases preserve both acceptance and rejection behavior.\n\nThe last-good/first-bad comparison locates the offending pass. It does not remove unrelated structure from the reproducer.\n\n[`mlir-reduce`](https://mlir.llvm.org/docs/Tools/mlir-reduce/) minimizes a valid input while preserving a user-defined\ninterestingness test. It validates each candidate before applying a reduction.\n\nThe tool keeps a reduction only while the test reproduces the failure.\n\nFor this incident, the candidate must contain valid IR from before the fusion\npass. The interestingness test runs the suspect pipeline and succeeds only when\nthe verifier reports `scale attribute must be a finite, positive value`.\n\nDo not feed `mlir-reduce` the already-invalid, post-fusion operation. That input\nfails verification before the tool can reduce it.\n\nFeed the tool valid, pre-fusion IR instead. It can then reduce the conditions that cause the pass to create the invalid attribute.\n\nThe concrete command depends on the private dialect and pipeline:\n\n```\n# Run mlir-reduce with test script\nmlir-reduce first-good-input.mlir \\\n  -reduction-tree=\"traversal-mode=0 test=check_for_nan_diagnostic.sh\"\n```\n\nReproduce the failure with pass-manager verification enabled.\n\nLocate the first bad pass with `-mlir-print-ir-after-all`.\n\nCapture that pass's before-and-after IR with the targeted print flags.\n\nFix the arithmetic that creates the invalid value.\n\nAdd a local operation verifier when the invariant belongs to the operation.\n\nAdd one negative diagnostic regression test.\n\nReduce the valid pre-pass input with the same failure predicate.\n\nRuntime numeric checks remain useful when invalid values exist only in execution data. Input-dependent instability may never appear in a compile-time attribute.\n\nThis incident has a different boundary. The compiler already holds the invalid value, so the operation verifier stops compilation next to the broken contract instead of letting the runtime report a distant accuracy regression.", "url": "https://wpnews.pro/news/catching-nan-at-the-mlir-pass-boundary", "canonical_source": "https://thecloudlet.github.io/technical/compiler/catching-nan-at-the-mlir-pass-boundary/", "published_at": "2026-08-26 00:00:00+00:00", "updated_at": "2026-09-16 08:40:42.865654+00:00", "lang": "en", "topics": ["ai-chips", "ai-infrastructure", "mlops", "developer-tools"], "entities": ["MLIR", "LLVM", "APFloat", "IEEE 754", "Synopsys", "NPU"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/catching-nan-at-the-mlir-pass-boundary", "markdown": "https://wpnews.pro/news/catching-nan-at-the-mlir-pass-boundary.md", "text": "https://wpnews.pro/news/catching-nan-at-the-mlir-pass-boundary.txt", "jsonld": "https://wpnews.pro/news/catching-nan-at-the-mlir-pass-boundary.jsonld"}}