{"slug": "learnable-spectral-activations", "title": "Learnable Spectral Activations", "summary": "Researchers introduced learnable spectral activations (LSA), a method that replaces fixed neuron-level nonlinearities with a residual truncated Fourier series whose harmonic amplitudes are learned during training, according to arXiv paper 2610.07419v1. LSA does not expand the asymptotic function class but changes the representation's factorization, separating feature selection by linear weights from spectral shaping by activation coefficients updated via separate gradients. Across audio, image, neural radiance field, and neural acoustic field tasks, LSA improved reconstruction quality and concentrated more target-signal energy in the leading eigenmodes of the neural tangent kernel.", "body_md": "arXiv:2610.07419v1 Announce Type: new \nAbstract: Implicit neural representations (INRs) are shaped by the spectral structure induced by their input encodings and activation functions. Existing methods improve fitting primarily by modifying which frequencies are available to the network, through coordinate encodings or periodic nonlinearities. However, frequency access is not the only bottleneck: signals with localized or spatially varying structure require the network to efficiently compose frequencies into multi-harmonic internal responses. We introduce learnable spectral activations (LSA), which replace fixed neuron-level nonlinearities with a residual truncated Fourier series whose harmonic amplitudes are learned during training. LSA does not expand the asymptotic function class. Instead, it changes the factorization of the representation: linear weights select features while activation coefficients control spectral shaping, and the two are updated by separate gradients. Because the activation output is affine in the coefficients given fixed pre-activations, spectral tuning becomes a more direct subproblem compared to architectures where it is entangled with feature selection. Empirically, this factorization concentrates more target-signal energy in the leading eigenmodes of the neural tangent kernel, consistent with improved optimization behavior. Across audio, image, neural radiance field, and neural acoustic field tasks, LSA also improves reconstruction quality.", "url": "https://wpnews.pro/news/learnable-spectral-activations", "canonical_source": "https://www.machinebrief.com/news/learnable-spectral-activations-yt22", "published_at": "2026-10-07 04:00:00+00:00", "updated_at": "2026-10-07 05:48:25.790862+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "ai-research", "computer-vision"], "entities": ["arXiv", "learnable spectral activations", "neural tangent kernel"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/learnable-spectral-activations", "markdown": "https://wpnews.pro/news/learnable-spectral-activations.md", "text": "https://wpnews.pro/news/learnable-spectral-activations.txt", "jsonld": "https://wpnews.pro/news/learnable-spectral-activations.jsonld"}}