arXiv:2611.23242v1 Announce Type: new Abstract: Spiking Neural Networks (SNNs) offer theoretical energy-efficiency advantages over conventional deep networks, but realizing these advantages on vision tasks requires sparse spike representations that naive rate-coding methods fail to provide. We combine biologically motivated spike-encoding schemes with two training improvements: Rouser adaptive-threshold surrogate gradients and Kronecker-Factored Approximate Curvature (K-FAC) second-order optimization. The spike encoding schemes experimented with are rate coding, latency coding, DVS-style delta modulation, and Difference-of-Gaussians (DoG) filtering. Experiments on MNIST and Fashion-MNIST demonstrate that biologically inspired encoders yield the largest accuracy gains in capacity-constrained regimes and serve as an effective parameter-efficiency lever. DoG and delta modulation encoders produce significantly sparser spike trains than rate coding. K-FAC yields modest but consistent accuracy improvements over Adam-trained baselines, and Rouser's learnable per-neuron thresholds improve Fashion-MNIST accuracy from 87.37% to 88.13% by enabling homeostatic firing-rate regulation. Biologically grounded pre-processing methods can help close the accuracy gap between SNNs and non-spiking baselines without increasing model size.
Biologically-Inspired Spike Encoding and Optimization for Spiking Neural Networks
A new arXiv paper (2611.23242v1) reports that combining biologically motivated spike-encoding schemes with Rouser adaptive-threshold surrogate gradients and Kronecker-Factored Approximate Curvature (K-FAC) second-order optimization improves spiking neural network accuracy on vision tasks without increasing model size. On Fashion-MNIST, Rouser's learnable per-neuron thresholds raised accuracy from 87.37% to 88.13% by enabling homeostatic firing-rate regulation, while Difference-of-Gaussians (DoG) filtering and DVS-style delta modulation encoders produced significantly sparser spike trains than rate coding. The authors conclude that biologically grounded pre-processing yields the largest gains in capacity-constrained regimes and can help close the accuracy gap between SNNs and non-spiking baselines.
For capacity-constrained SNNs, replace rate coding with DoG or delta-modulation encoders and add Rouser learnable per-neuron thresholds, which raised Fashion-MNIST accuracy from 87.37% to 88.13%.
more in playbook: Fine-tuning and dataRun your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.