{"slug": "automatic-differentiation-from-scratch-how-pytorch-computes-gradients-in-physics", "title": "Automatic Differentiation from Scratch: How PyTorch Computes Gradients in Physics-Informed Neural Networks", "summary": "A new arXiv paper traces how PyTorch's automatic differentiation engine computes gradients for Physics-Informed Neural Networks, detailing the two-level differentiation process and verifying results against hand derivations.", "body_md": "arXiv:2607.13042v1 Announce Type: new\nAbstract: This paper traces, with explicit numerical values, how PyTorch's automatic differentiation (AD) engine computes gradients for Physics-Informed Neural Network (PINN) training -- a setting that requires two levels of differentiation: computing the physics derivative $\\hat{y}'(t)=d\\hat{y}/dt$ through the network, and computing parameter gradients $\\nabla_\\theta L$ of a loss that itself depends on $\\hat{y}'(t)$. Using a 1-3-3-1 multilayer perceptron and the initial value problem $y'(t)+y(t)=0$, $y(0)=1$, we trace the complete pipeline at every node: the computational graph built during the forward pass, the reverse-mode backward traversal that computes all 22 parameter gradients in a single pass, and the graph-on-graph mechanism by which \\texttt{create\\_graph=True} enables correct differentiation through the physics-informed residual. Every adjoint value is verified against the hand derivations of Tahimi (2026), connecting the $P/Q$ sensitivity framework to the vector--Jacobian products used by PyTorch's autograd engine.", "url": "https://wpnews.pro/news/automatic-differentiation-from-scratch-how-pytorch-computes-gradients-in-physics", "canonical_source": "https://arxiv.org/abs/2607.13042", "published_at": "2026-07-16 04:00:00+00:00", "updated_at": "2026-07-16 04:28:34.673604+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "artificial-intelligence", "ai-research", "developer-tools"], "entities": ["PyTorch", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/automatic-differentiation-from-scratch-how-pytorch-computes-gradients-in-physics", "markdown": "https://wpnews.pro/news/automatic-differentiation-from-scratch-how-pytorch-computes-gradients-in-physics.md", "text": "https://wpnews.pro/news/automatic-differentiation-from-scratch-how-pytorch-computes-gradients-in-physics.txt", "jsonld": "https://wpnews.pro/news/automatic-differentiation-from-scratch-how-pytorch-computes-gradients-in-physics.jsonld"}}