{"slug": "empirical-minimal-realisation-compression-of-deep-neural-networks-via-tests", "title": "Empirical Minimal-Realisation Compression of Deep Neural Networks via Controllability-Observability Tests", "summary": "Researchers proposed a controllability-observability framework for compressing deep neural networks by viewing them as nonlinear dynamical systems. The method achieved up to 83% parameter compression on CIFAR-10 with negligible accuracy loss and 3x inference speedup. The approach provides a principled criterion for designing compact neural architectures.", "body_md": "arXiv:2607.05457v1 Announce Type: new\nAbstract: Deep neural networks often contain substantial hidden-state redundancy, but most compression methods operate directly on weights, neurons, or quantised representations without explicitly characterising the dynamical role of internal states. This paper proposes a controllability-observability framework for empirical state-order reduction of deep neural networks. By viewing a trained network as a depth-indexed nonlinear dynamical system, we construct data-driven reachability, observability, and balanced Gramians from hidden-state snapshots and output Jacobians. The resulting A/B/C tests estimate layer-wise reachable, observable, and jointly reachable--observable ranks. These ranks are then used not only as diagnostic measures of hidden-state redundancy, but also as actual compressed layer widths for realised reduced networks. Experiments on MNIST and CIFAR-10 compare the proposed balanced realisation against projection-based reduction, unstructured pruning, structured pruning, low-rank SVD, dynamic INT8 quantisation, and linear baselines. On MNIST, a four-layer SiLU DNN is reduced from state order 1024 to 277, giving 72.95% state compression and 73.48% parameter compression, while maintaining 95.45% accuracy compared with 96.60% for the full model. On CIFAR-10, a larger SiLU DNN is reduced from state order 4608 to 1339, giving 70.94% state compression and 83.09% parameter compression, while preserving accuracy from 54.45% to 54.44% and reducing CUDA inference latency by approximately 3X. The results show that balanced reachable-observable ranks provide a principled empirical minimal-realisation criterion for designing compact neural architectures with little or no loss in accuracy.", "url": "https://wpnews.pro/news/empirical-minimal-realisation-compression-of-deep-neural-networks-via-tests", "canonical_source": "https://arxiv.org/abs/2607.05457", "published_at": "2026-07-08 04:00:00+00:00", "updated_at": "2026-07-08 04:16:22.540259+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "ai-research"], "entities": ["MNIST", "CIFAR-10", "SiLU"], "alternates": {"html": "https://wpnews.pro/news/empirical-minimal-realisation-compression-of-deep-neural-networks-via-tests", "markdown": "https://wpnews.pro/news/empirical-minimal-realisation-compression-of-deep-neural-networks-via-tests.md", "text": "https://wpnews.pro/news/empirical-minimal-realisation-compression-of-deep-neural-networks-via-tests.txt", "jsonld": "https://wpnews.pro/news/empirical-minimal-realisation-compression-of-deep-neural-networks-via-tests.jsonld"}}