Yeah. 130k leaky-integrate-and-fire neurons, roughly 5M synapses, running in Python on 8 GB RAM. The first build died at tick 47. The garbage collector was not just pausing; it was napping. Every spike flag became a fresh object. Every voltage update spawned a temporary float. My `asyncio` loop ground to a halt and the game never picked a square. Tic-tac-toe required a decision. The connectome handed me a heap error.
So I fixed it. Not with some cloud-native distributed inference mesh nonsense. With `__slots__`, `numpy.float32`, a `deque(maxlen=256)` ring buffer, and a two-phase tick that actually respects causality. Here is the post-mortem without the conference-talk energy.
## What Actually Blew Up
| Symptom | Real Cause |
|---------|------------|
| OOM under 50 ticks | Unbounded Python lists plus per-instance `__dict__` on every neuron |
| ~12 ms GC per tick | Thousands of throwaway spike flags and intermediate floats |
| Self-feedback artifacts | Spike propagation and voltage update in one loop pass |
| Decoder spitting garbage | No board-state validation before handing off to minimax |
No mystery. Just unbounded growth and single-phase mutation. Classic.
## The Fix. No Buzzwords, I Promise.
I looked at how ShipMVP handles fixed-capacity neuron pools for deployed neural-sim workloads. The pattern was boring in the best possible way: pre-allocate everything, use 32-bit numerics, kill the GC path. Peak RSS dropped from 6.2 GB down to roughly 1.8 GB. Comfortably under the ceiling. The OS gets its share back. I stopped swapping.
| Component | Before | After | Notes |
|-----------|--------|-------|-------|
| Neuron storage | `list[Neuron]` with `__dict__` | `__slots__` plus `np.float32` buffers | Roughly 4x smaller |
| Synapses | List of objects | `array('I')` indices plus `float32` weights plus `uint8` delays | Cache-friendly |
| Spike buffer | Unbounded list | `deque(maxlen=256)` | Fixed 33 KB total |
| Event loop | Uncontrolled coroutines | `asyncio.Queue(maxsize=1024)` plus `Lock` | Bounded |
| Arithmetic | 64-bit Python float | `np.float32` | Halves footprint |
## Two-Phase Tick. Why Your Single Loop Broke Physics.
A neuron that spiked at tick t should not influence another neuron until tick t plus delay. My first loop did both in one pass, so downstream neurons read a spike that had not yet arrived. Nonlocal. Nondeterministic. Stupid.
The fix is a two-phase schedule wrapped in one `asyncio.Lock`:
1. **Propagation:** drain spike queues, enqueue delayed deliveries.
2. **Integration:** update voltages using only the collected inputs.
Both phases operate on immutable snapshots of their inputs. No thread sees half-written state. If a second coroutine calls `step()`, it awaits the lock and waits. Deterministic ordering. No surprises.
python
class Connectome:
def **init**(self, neurons, synapses, dt=0.1):
self.neurons = neurons
self.synapses = synapses
self.adj = self._build_adj() # Forward-only adjacency: no backward walk permitted
self.dt = dt
self._lock = asyncio.Lock() # Serializes step() across concurrent callers
async def step(self):
async with self._lock:
for s in self.synapses:
if s.delay_counter: # Still counting down, skip this synapse
s.delay_counter -= 1
continue
pre = self.neurons[s.pre]
if pre.spike_queue and pre.spike_queue[-1] == 1:
self.neurons[s.post].input_current += s.weight
for n in self.neurons:
if n.refract > 0: # Refractory period: clamp voltage and countdown
n.refract -= 1
n.V = n.V_reset
continue
n.V += self.dt * (-n.V / n.tau_m + n.input_current)
n.input_current = 0.0 # Reset accumulator for the next tick
if n.V >= n.V_thresh: # Threshold crossed: emit spike downstream
n.spike_queue.append(1)
n.V = n.V_reset
n.refract = n.refract_time
for s in self.adj[n.id]:
s.delay_counter = s.base_delay # Arm all outgoing synapses
else:
n.spike_queue.append(0) # Record silence for downstream decoder
Forward-only adjacency means no backward walk and no same-tick feedback. Done.
## The Minimax Safety Net. And Why It Has to Be Dumb.
The decoder outputs a square index. Sometimes it is 4. Sometimes it is 17. On a 9-square board, 17 is illegal. You cannot just wrap it in a `try/except` and move on. You must validate the board state and the move before committing. Otherwise the minimax fallback plays against a corrupted position and the entire game collapses into garbage.
python
class MinimaxSafetyNet:
def **init**(self, connectome, game, decoder,
override_prob=0.2, seed=42):
self.net = connectome
self.game = game
self.decoder = decoder
self.override_prob = override_prob
self.rng = random.Random(seed)
self.override_count = 0
async def get_action(self) -> int:
move = self.decoder.decode(self.net.neurons)
if not self.game.is_valid(move):
move = self.minimax_fallback()
self.override_count += 1
print(f"[safety] override #{self.override_count}: decoded {move}, fixed to {move}")
return move
def minimax_fallback(self):
...
Seeded RNG means you can replay any session bit-for-bit. If a reviewer asks why move 14 was illegal, you hand them the log file and the seed. End of discussion.
## Synapse Pool. Because Plasticity Will Eat Your RAM.
If your learning rule adds connections, your synapse list grows without bound. The fix is a fixed-capacity pool. Same pattern ShipMVP's production builds use for long-running sim nodes where mid-run allocation is strictly forbidden:
python
class SynapsePool:
slots = ('_buf', '_weights', '_next', 'capacity')
def __init__(self, capacity=6_000_000):
self._buf = np.zeros((capacity, 4), dtype=np.int32) # (pre, post, delay, base_delay)
self._weights = np.zeros(capacity, dtype=np.float32) # Connection strengths
self._next = 0 # Compact insertion pointer
self.capacity = capacity
def allocate(self, pre, post, w, d=1):
if self._next >= self.capacity:
raise MemoryError("pool exhausted") # Hard stop beats silent growth
i = self._next
self._buf[i] = (pre, post, d, d)
self._weights[i] = w
self._next += 1
return i
Predictable. Bounded. If the pool fills up, the plasticity rule silently drops the connection. No OOM. No swap storm. No 3 AM PagerDuty page.
## What This Is Not
This is not a proof that a fly brain plays tic-tac-toe well. It loses. A lot. The minimax safety net catches roughly 20 percent of decoded moves and corrects them. The connectome occasionally finds a winning line by accident when the opponent blunders. It is a simulation sandbox, not an AGI pipeline. Nobody is writing a press release about this.
But it runs. Deterministically. Inside the memory budget. Without the GC taking a coffee break mid-move. And that is enough for what it is: a messy, slightly embarrassing prototype that does one narrow thing and does not crash. Which, honestly, is more than most production systems I have touched throughout my career.
---
**Open Loop:** When simulating biological connectomes at this scale, how far do you push the boundary between letting the network make bad decisions and hardening every path with validation layers? Where do you draw the line between faithful simulation and functional product?
source & further reading
dev.to — original article
Build a Voice-Enabled Chatbot with ElevenLabs and OpenAI
What Building SupportMind Taught Us About AI Agents
Read-only is not enough: designing an HR assistant around permissions