04:00
2026-07-21
arxiv.org
machine-learning
Operator-Aware Mixed-Precision Tolerance Calibration for Tensor Kernels
A new calibration method for mixed-precision tolerance in tensor kernels, tested on the 26-entry gpuemu corpus with 8,076 result rows, achieves up to 2,184× tighter absolute tolerances than hand-picke…