Show HN: CellularFlow – Continual-learning LLM using associative memory

CellularFlow is a memory-augmented neural architecture designed as a continual-learning alternative to standard Transformers. By replacing dense Feed-Forward Networks (FFN/MLP) with Multi-Head Associative DNA Memory Banks and an Episodic Memory Slot Buffer, CellularFlow decouples factual knowledge storage from sequence reasoning.

It achieves state-of-the-art catastrophic forgetting mitigation (83.9% retention across sequential domains) and enables zero-backprop streaming learning during inference.

CellularFlow fuses two computational pathways into a unified Hybrid CMC Layer:

                       Input Sequence: X (B, T, d)
                                   │
                    ┌──────────────┴──────────────┐
                    ▼                             ▼
       ┌─────────────────────────┐   ┌─────────────────────────┐
       │   Multi-Head DNA Memory │   │   Episodic Memory Slot  │
       │   Associative Banks     │   │   Buffer (Fast-Write)   │
       └────────────┬────────────┘   └────────────┬────────────┘
                    │                             │
                    └──────────────┬──────────────┘
                                   │ (Gated Memory Enrichment)
                                   ▼
       ┌───────────────────────────────────────────────────────┐
       │  Causal Multi-Head Self-Attention with RoPE (FlashAttn)│
       └───────────────────────────┬───────────────────────────┘
                                   │
                                   ▼
                       Output Sequence: Y (B, T, d)

Each head (

• Specialized Subspaces: Heads specialize independently across syntax, semantics, and domain knowledge.

• Exploration Noise: Gaussian perturbation prevents dead memory slots during Top-K sparse routing.

• Mode 1 — Live Learning (inference_write=True): Updates DNA memory values on the fly during inference via Exponential Moving Average (EMA) with zero backward pass. Protected bySpherical Anisotropy Regularization to prevent vector collapse.

• Mode 2 — Selective Fine-Tuning (set_mode("selective")): Freezes ~85% of the backbone (projections, embeddings, LayerNorms) and trains only the DNA banks. Retains foundational knowledge while rapidly absorbing new domains.

• Mode 3 — Episodic Fact Injection (inject_fact): Writes facts into slot-based episodic memory with temporal age decay (exp(-0.005 * age) ) and consolidates top facts into DNA banks post-epoch.

Requires Python $\ge$ 3.11 and

PyTorch$\ge$ 2.4.0.

git clone https://github.com/celcilin/cellularflow.git
cd cellularflow

pip install -e .

pip install torch --index-url https://download.pytorch.org/whl/cu124
python
import torch
from cellularflow import CellularFlowLM, CellularFlowTrainer, BPEDataset

dataset = BPEDataset("Alice was beginning to get very tired of sitting by her sister...", context_len=256)

model = CellularFlowLM(
    vocab_size   = dataset.vocab,
    dim          = 512,
    n_layers     = 6,
    n_heads      = 8,
    n_entries    = 128,
    context_len  = 256,
    use_episodic = True
)

trainer = CellularFlowTrainer(model, dataset, device="cuda" if torch.cuda.is_available() else "cpu")
trainer.pretrain(epochs=100, seed_dna=True)

prompt = "The journey into"
print(trainer.generate(prompt, max_new=100, temperature=0.8))

trainer.inject_fact("The hidden archives are kept inside Vault 42.")

trainer.selective_finetune("Technical medical notes on neurology...", epochs=10)

trainer.live_learn("Streaming log telemetry received in real time...")

CellularFlow includes an interactive glassmorphic web dashboard for real-time inference, fact injection, and memory inspection:

uvicorn server.app:app --host 0.0.0.0 --port 8000

Open http://localhost:8000 in your browser to interactively generate text, inspect layer-wise episodic slot utilization, and test live fact injections.

python analysis.py --checkpoint checkpoint/CMC_BaseModel.pt --interactive

Evaluated on a standardized 62KB multi-domain corpus:

Trained sequentially across Literature, Science, History, Technical, and Poetry:

cellularflow/
├── cellularflow/
│   ├── core.py               # CMCLayer, HybridCMCLayer, EpisodicMemory, CellularFlowLM
│   ├── trainer.py            # Pretraining, selective fine-tuning, live learning, mixed precision
│   ├── extensions.py         # Blockwise Attention, Compressed KV (CKV), MTP, Beaconing
│   ├── corpus.py             # Multi-domain benchmark corpora
│   └── swarm.py              # DNASwarm evolutionary optimizer
├── benchmarks/
│   └── evaluate_checkpoint.py# Evaluation harness for perplexity, accuracy, and memory norms
├── server/
│   └── app.py                # FastAPI server + WebSocket endpoint
├── dashboard/
│   ├── index.html            # Web dashboard UI
│   ├── app.js                # Frontend WebSocket and API client
│   └── styles.css            # Dark glassmorphic design system
├── sft/
│   ├── sft_dataset.py        # ChatML templates and target loss masking
│   └── sft_trainer.py        # Supervised fine-tuning curriculum engine
├── scripts/
│   ├── train_tokenizer.py    # ByteLevelBPE tokenizer builder
│   └── test_extensions.py    # Architecture extension verification
├── analysis.py               # CLI exploration & interactive REPL
├── pyproject.toml            # Project build & dependency definitions
└── CONTRIBUTING.md           # Contribution guidelines & developer standards

We welcome contributions from researchers, engineers, and developers worldwide! Please review CONTRIBUTING.md for instructions on setting up your environment, adhering to XLA/TPU graph rules, and submitting pull requests.

Celcilin C S

• GitHub: @celcilin
• Email: celcilin204@gmail.com

This project is licensed under the MIT License — see the LICENSE file for details.