Defeating vanishing gradients in deep neural networks: Quotient Tree Arithmetic

A new computational framework called Quotient Tree Arithmetic (QTA) prevents gradient underflow in deep neural network training by representing values as deferred quotient pairs evaluated lazily at a materialization boundary, according to a paper submitted to arXiv on 15 Jun 2026. QTA provides O(1)-cost gradient computation via chain-rule tape collapse and reduces DAG storage from O(BLd) to O(L+Bd) for a batch of B examples through L layers, with a single IEEE division introducing at most one-half ULP of rounding error versus O(m) ULP for eager evaluation.

Computer Science Symbolic Computation Submitted on 15 Jun 2026 Title:Quotient Tree Arithmetic: Deferred-Division Computation with Bounded Symbolic Depth and Cross-Subtree Cancellation View PDF /pdf/2607.22612 HTML experimental https://arxiv.org/html/2607.22612v1 Abstract:We introduce Quotient Tree Arithmetic QTA , a computational substrate in which values are represented as deferred quotient pairs N, D whose ratio is evaluated lazily at a designated materialization boundary. The framework applies to any domain: IEEE 754 doubles used as exact integer containers give exact rational arithmetic within the 2^53 exactness window; arbitrary IEEE doubles extend coverage to transcendental values including machine learning activations such as exp x and sqrt x . Three structural theorems underpin QTA. 1 Bounded Depth Growth: each arithmetic operation increases tree depth by at most 1, giving O m tree size after m operations with no combinatorial explosion. 2 Cross-Subtree Cancellation: subtrees appearing in both numerator and denominator positions cancel via reference identity without arithmetic, including transcendental values computed once and shared. 3 Deferred Stability: a single IEEE division at the materialization boundary introduces at most one-half ULP of rounding error, versus O m ULP for eager evaluation. For machine learning training, QTA provides: structural prevention of gradient underflow to zero; O 1 -cost gradient computation via chain-rule tape collapse when intermediate activations are reference-identical; shared-weight batch compression reducing DAG storage from O BLd to O L+Bd for a batch of B examples through L layers; and tracked factor cancellation replacing O log n GCD with O 1 trial division when denominators are known. We propose a vectorized hardware normalization instruction RatCleanup for SIMD-parallel rational pair reduction. The algebraic foundation is the localization of a ring at its multiplicative set, connecting QTA to algebraic structure theory while grounding it in hardware-native IEEE arithmetic. Current browse context: cs.SC References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .