{"slug": "spiking-neural-networks-and-neuromorphic-hardware", "title": "Spiking Neural Networks and Neuromorphic Hardware", "summary": "A developer explains that spiking neural networks (SNNs) and neuromorphic hardware offer an energy-efficient alternative to traditional neural networks by using binary spike events instead of continuous values. The key efficiency gain comes from replacing multiply-accumulate operations with additions, but this advantage depends on low spike rates and few timesteps, which are undermined by conversion-based approaches that require many timesteps to encode dense inputs.", "body_md": "A spiking neuron does not output a number. It accumulates input into a membrane potential and, when that crosses a threshold, emits a single binary event and resets. Because the event is a 1, a synapse becomes an addition rather than a multiplication — and that substitution is the entire energy argument for the field.\n\nThe standard model is leaky integrate-and-fire, and it fits in four lines:\n\n```\nV_t = beta * V_{t-1} + I_t - S_{t-1} * V_threshold\nS_t = 1 if V_t > V_threshold else 0\n\n  V    : membrane potential, one scalar per neuron\n  beta : leak, = exp(-dt / tau), typically 0.9 to 0.99\n  I_t  : input current = sum of weights of the synapses that\n         received a spike this step\n  S_t  : the output. A single bit.\n```\n\nCompare it with a [recurrent unit](https://multigrid.ai/learn/rnn-lstm-explained) and the relationship is clear: this is a recurrent neuron whose state update is linear with a leak, and whose activation function is a step at the threshold. Its memory is one number, its output is one bit, and its nonlinearity is the hardest possible one.\n\nThe information is in *when* spikes happen and how often, not in their amplitude — there is no amplitude. That is the sense in which these networks are event-driven, and it is why they pair naturally with event cameras and other sensors that emit changes rather than frames.\n\nA pixel intensity is a number and the network only accepts events, so something has to convert. The choice is not a detail — it decides how many timesteps you need, and timesteps are what the efficiency argument is spent on.\n\n| Scheme | Description |\n|---|---|\n| Rate coding | Emit spikes with probability proportional to the value over T steps. Simple and robust to noise, and wasteful: T steps distinguish only T + 1 levels, so 8-bit precision would want 255 steps. This is what conversion-based approaches rely on, and it is why they need so many steps. |\n| Latency coding | Encode the value as when a single spike arrives: brighter fires earlier. One spike per neuron, which is as sparse as it gets, and information arrives progressively so a classification can often be made before the last inputs have come in. Fragile to timing jitter and harder to train. |\n| Population coding | A bank of neurons with overlapping tuning curves; the value is read from which of them fire. Buys precision at small T by spending neurons instead of timesteps. This is roughly what biology does. |\n| No encoding at all | The important case. An event camera emits a stream of per-pixel brightness-change events natively; a silicon cochlea emits events per frequency band. There is nothing to convert, no timestep budget spent on encoding, and the sparsity is a property of the world rather than an artefact. |\n\nThat last row is the through-line of this page. Where the input is already events, the architecture matches the data and the efficiency argument is at its strongest. Where the input is a dense tensor that has to be converted into events first, you have spent timesteps to create sparsity that was not there, and the argument gets much weaker.\n\nA standard network evaluates a layer once per input. A spiking network simulates it for `T`\n\ntimesteps, because a single binary spike carries almost no information and the signal is in the pattern across time.\n\n```\nStandard layer:   1 evaluation\nSpiking layer:    T evaluations, T typically 4 to 256\n\nThe saving has to come from sparsity: if only 5% of neurons spike\nat each step, the work per step is 5% of dense.\n\nNet effect at T = 32 and 5% activity:\n  32 * 0.05 = 1.6x the synaptic operations of the dense layer\n  ...but each is an ADD, not a MULTIPLY-ACCUMULATE.\n```\n\nThat arithmetic is the honest version of the efficiency claim, and it shows how tightly the two parameters bind. Low spike rates and few timesteps are the entire game. Conversion-based approaches that need 100 or more timesteps to approximate a rate code give the whole advantage back before they start.\n\nThe argument is real, and it is best made precisely. In a conventional network, a synapse computes `weight * activation`\n\nand adds it to an accumulator: a multiply-accumulate. In a spiking network the activation is 0 or 1, so a synapse either does nothing or adds the weight: an accumulate.\n\nIn digital CMOS, a floating-point multiply-accumulate costs substantially more energy than an integer addition of the same width — roughly an order of magnitude at 32 bits, on the figures usually cited from circuit-level analyses. Add the sparsity, and add that on neuromorphic hardware a neuron that does not spike consumes almost nothing because nothing is clocked, and you have an argument for a large energy reduction on the right workload.\n\nThe important qualification: most published efficiency comparisons count synaptic operations and multiply by a per-operation energy figure. That is an estimate of the arithmetic, not a measurement of a system. Real energy includes memory movement, the host processor, conversion of inputs into spikes, and idle draw — and on general-purpose hardware a spiking network is usually *slower and less efficient* than the network it replaces, because simulating `T`\n\nsequential steps of a sparse update is exactly what a GPU is bad at. The energy claim is a claim about matched hardware, and it should be read that way.\n\nTraining is the hard part, and the reason is one line of calculus. The output is a step function of the membrane potential:\n\n```\nS = 1 if V > V_th else 0\n\ndS/dV = 0     everywhere except at V = V_th\ndS/dV = undefined at V = V_th\n\nBackpropagation multiplies by dS/dV. Multiplying by zero\neverywhere means no gradient reaches any earlier layer.\n```\n\nTwo workarounds are in use, and the difference between them is worth knowing:\n\n`T`\n\nsteps. The conversion is faithful only as `T`\n\ngrows, and the timestep counts needed for accuracy are often in the hundreds, which as shown above deletes the efficiency argument. Recent work reduces this considerably; the tension is structural.Note what the first option means philosophically. The architecture is motivated by biological plausibility, and it is trained with a method that is a deliberate lie about the derivative, run through global backpropagation, which is the thing biology does not do. That is not an objection to the method — it works — but it does undercut the framing.\n\n| Platform | Description |\n|---|---|\n| Intel Loihi 2 | Research chip, second generation, with programmable neuron models and an on-chip mesh. Available to academic and industrial partners through Intel's research community rather than as a product. |\n| IBM TrueNorth / NorthPole | TrueNorth was the landmark low-power spiking chip; NorthPole is the later inference-focused design in the same lineage, emphasising keeping weights on chip to avoid memory movement. |\n| SpiNNaker / SpiNNaker 2 | A large many-core platform built for real-time simulation of large spiking networks, used primarily in computational neuroscience. |\n| BrainChip Akida | A commercial edge inference part aimed at always-on sensing, and one of the few in this list you can buy as a component. |\n\nAll of these are real silicon. None is a platform on which a general-purpose model is deployed at scale, and the tooling is fragmented in a way that compounds the problem: each has its own framework, and a model developed for one does not move to another.\n\nHonestly, and without either dismissing it or overselling it:\n\n**The trade:** binary events buy an accumulate instead of a multiply-accumulate and activity-proportional energy on hardware built for it. They cost a nonexistent gradient, an extra time axis that multiplies the work, poor mapping onto the hardware nearly everybody actually has, and an ecosystem that is a rounding error next to the alternative. Whether that is a good trade depends almost entirely on whether your input is a stream of events or a tensor, and [whether your power budget is measured in milliwatts](https://multigrid.ai/learn/edge-inference).", "url": "https://wpnews.pro/news/spiking-neural-networks-and-neuromorphic-hardware", "canonical_source": "https://dev.to/multigrid/spiking-neural-networks-and-neuromorphic-hardware-30p5", "published_at": "2026-08-12 17:35:12+00:00", "updated_at": "2026-08-12 17:46:53.725447+00:00", "lang": "en", "topics": ["artificial-intelligence", "neural-networks", "ai-research", "ai-infrastructure"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/spiking-neural-networks-and-neuromorphic-hardware", "markdown": "https://wpnews.pro/news/spiking-neural-networks-and-neuromorphic-hardware.md", "text": "https://wpnews.pro/news/spiking-neural-networks-and-neuromorphic-hardware.txt", "jsonld": "https://wpnews.pro/news/spiking-neural-networks-and-neuromorphic-hardware.jsonld"}}