LLT: Local Linear Transformer for PDE Operator Learning

Researchers introduced the Local Linear Transformer (LLT), a neural operator for learning PDE solution maps that combines linear global attention with local spatial mixing and incorporates coordinate and geometry information. LLT achieves competitive or lower relative L2 error across multiple PDE problems including elasticity, plasticity, airfoil flow, pipe flow, and Darcy flow, while reducing training time by factors of 1.8 to 2.5 compared to Transolver on matched structured discretizations. The architecture addresses limitations of standard attention in PDEs by scaling linearly with computational nodes and providing an explicit local interaction bias.

arXiv:2607.07718v1 Announce Type: new Abstract: Neural operators have become a common approach for learning PDE solution maps and accelerating numerical simulations. Transformer-based neural operators are of particular interest, since attention can learn long-range dependencies in the computational domain. However, standard attention has two major limitations when applied to PDEs: it scales quadratically with the number of computational nodes, and it lacks an explicit bias toward local interactions. To address these issues, we introduce Local Linear Transformer LLT for PDE operator learning. The architecture combines linear global attention with local spatial mixing, and incorporates coordinate and geometry information. We evaluate LLT on several PDE problems, including elasticity, plasticity, airfoil flow, pipe flow, and Darcy flow. The reference data for these problems span finite-element, finite-volume, and finite-difference discretizations on structured and unstructured meshes. Compared with other neural-operator and transformer baselines from prior studies, LLT achieves competitive or lower relative $L 2$ error across these problems. On matched structured discretizations, wall-clock time per training iteration is reduced by factors of 1.8 to 2.5 relative to Transolver. We also scale the approach and apply it to a three-dimensional car aerodynamics dataset with 32,186 unstructured mesh points per sample. Together, these results indicate that LLT provides an accurate and computationally efficient operator for PDE problems across discretizations, mesh types, and problem settings.