"Every timeline you've ever seen is a lie of omission. It shows you WHEN things happened. It never shows you WHY. ChronoWeave is our attempt to fix that β and along the way, you're going to learn what actually happens inside a neural network when you call.backward()
."
This is not a tutorial you skim. It's a companion for a 3β5 month build. Read one section, do the exercise, hit the wall, climb over it, then come back. Every chapter follows the same rhythm:
And threaded throughout:
Let's begin.
Open any history textbook's timeline. You'll see a horizontal line, some dots, some dates. 1914. 1917. 1929. 1939. It tells you when. It is silent on why.
The assassination of Archduke Franz Ferdinand didn't cause World War I in a vacuum β it was the spark that landed on a decade of alliance treaties, arms races, and colonial tension that had already turned the continent into dry kindling. A flat timeline shows you the spark. It hides the kindling.
What historians actually think in is a causal graph: event A enabled event B, which, combined with condition C, triggered event D. That graph is tangled, non-linear, and multi-dimensional β which is exactly why nobody draws it by hand. It's too much cognitive load.
ChronoWeave's bet: if a machine can extract the causal graph from raw text, and then lay it out so that causally-connected events cluster together and pull each other into readable arrangement, you get something a static timeline can never give you β a map you can explore, drag, and reorganize, where the layout itself is meaningful. Events that influence each other stay near each other. That's not a UI nicety. That's the entire value proposition.
Here's the complete system, before we write a single line of code:
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β THE USER'S BROWSER β
β ββββββββββββββββ ββββββββββββββββββββββββββββββββββββββ β
β β Text Input ββββββββΆβ React + D3.js Canvas β β
β β "Paste β β (renders nodes = events, β β
β β history β β edges = causal links) β β
β β here" β β β β
β ββββββββββββββββ β User drags a node ββββββββ β β
β βββββββββββββββββββββββββββββββΌββββββββββ β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββΌβββββββββββββββ
β POST /extract β WS: node_moved
βΌ βΌ
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β FASTAPI BACKEND β
β β
β ββββββββββββββββββββββ βββββββββββββββββββββββββββββββββββββ β
β β EXTRACTION ENGINE β β SPATIAL INTELLIGENCE ENGINE β β
β β (Chapter 2) β β (Chapter 3 β the heart of it) β β
β β β β β β
β β 1. Split into β β 1. Build a graph: nodes=events, β β
β β sentences β β edges=causal relations β β
β β 2. Run NER ββββββΆβ 2. Init x,y for every node as a β β
β β (BERT) to find β β TRAINABLE TENSOR β β
β β events, dates, β β (requires_grad=True) β β
β β people, places β β 3. Define "Map Clarity Loss": β β
β β 3. Run Relation β β - causally linked nodes should β β
β β Extraction to β β be CLOSE β β
β β find "A causes B" β β - unrelated nodes should be β β
β β 4. Output: list of β β FAR (repulsion) β β
β β (event, date, β β - nothing should overlap β β
β β relation, event) β β 4. Run backprop on the β β
β β triples β β COORDINATES (not the model's β β
β β β β weights!) using a hand-written β β
β β β β Adam optimizer β β
β β β β 5. When user drags a node, PIN β β
β β β β that node's coords, re-run β β
β β β β optimization on everyone else β β
β ββββββββββββββββββββββββββ βββββββββββββββββββββββββββββββββββββ β
β β β β
β βΌ βΌ β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β PostgreSQL + pgvector (stores triples + embeddings) β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Walk through it end to end, because this sequence is the skeleton every later chapter hangs off of:
/api/extract
.CAUSES
, ENABLED_BY
, PRECEDES
, PART_OF
, or NONE
.(Assassination of Franz Ferdinand, CAUSES, Austria-Hungary ultimatum to Serbia)
. These get written to Postgres, and each event also gets a requires_grad=True
. This is the part that makes ChronoWeave unusual: the .backward()
, get gradients (node_id, x, y)
positions back to the frontend over the initial REST response (or WebSocket, for the live re-layout case).requires_grad=False
for it, fixed at the drag target). The backend re-runs the optimizer on every That last step is the entire reason this project exists. A physics engine (force-directed graph layout, e.g. D3's built-in forceSimulation
) can also do node repulsion and edge attraction β but it can't easily express arbitrary, learned, semantically-weighted objectives (e.g., "cluster by decade AND by causal chain AND penalize edge crossings, with weights that adapt based on graph density"). Gradient descent on a custom loss can. That's the pitch. We'll come back to defending it rigorously in Chapter 3.
| Layer | Choice | Why |
|---|---|---|
| ML core | PyTorch (from raw tensors, then nn.Module later) |
|
| You need autograd exposed at the tensor level to do coordinate optimization; PyTorch's dynamic graph makes this natural | ||
| NER / Relation Extraction | Fine-tuned BERT-base, later maybe REBEL for joint extraction | BERT is small enough to fine-tune on a single GPU and well documented; REBEL is a strong upgrade path for joint entity+relation extraction |
| Backend | FastAPI + WebSockets | Async-native, typed, and WebSocket support is first-class for the live re-layout streaming |
| Database | PostgreSQL + pgvector | Relational structure for triples, vector column for semantic similarity search on event embeddings |
| Frontend | React + D3.js | D3 gives you full control over force/position rendering; React manages component state and drag events |
| Deployment | Docker Compose β cloud (Render/Fly.io/AWS) | Reproducibility first, then scale |
Four phases, each roughly a month, building strictly bottom-up:
π§ The Mentor Says: Don't let the ambition of the final product distract you from the ugliness of the early steps. In week 2 you'll be manually computing gradients for a two-layer network with a pencil-and-paper feel to it, and it will seem impossibly far from "beautiful interactive graph app." That gap is supposed to be there. Every person who deeply understands PyTorch went through exactly this tunnel. You don't skip it by using nn.Module
early β you skip understanding by doing that.
(Weeks 1β4 Β· Prerequisite: comfort with Python, basic linear algebra β matrix multiply, dot products β and derivatives from a first calculus course)
A tensor is just a multi-dimensional array with two superpowers bolted on: it knows how to run on a GPU, and it can remember the sequence of operations that produced it, so it can later compute how a tiny nudge to its inputs would change its output. That second superpower is autograd, and it's the single idea Phase 1 exists to demystify.
Think of a tensor with requires_grad=True
as a spreadsheet cell that doesn't just hold a number β it holds a number and a formula referencing other cells. Change an input cell, and every downstream cell knows exactly how much it would change, without you re-deriving the formula by hand. That "how much would it change" is the gradient.
Set up your environment first:
poetry init
poetry add torch numpy matplotlib jupyter
poetry shell
Then in a notebook, don't build anything yet β just observe autograd:
import torch
x = ...
y = ...
...
x.grad
should equal 2*x + 2 = 8.0
β exactly the calculus-class derivative, computed automatically. The click isn't "wow it did calculus," it's realizing PyTorch didn't look up a symbolic formula for x**2 + 2*x + 1
β it recorded the sequence of primitive operations (pow
, mul
, add
) as you executed them, and walked that sequence backward, applying the chain rule at each step. This recorded sequence is called the computation graph, and it's rebuilt fresh every time you run forward β which is why PyTorch is called "define-by-run."
Every gradient ChronoWeave ever computes β whether it's a loss with respect to a BERT weight, or a loss with respect to a node's (x, y) position on the canvas β goes through this exact mechanism. There is no different code path for "coordinates" versus "weights." This is the whole trick behind Phase 3, twelve weeks from now.
π³οΈ The Rabbit Hole: Read about torch.autograd.grad()
vs .backward()
β the former lets you compute gradients without accumulating them into .grad
, important later when you want gradients with respect to coordinates specifically, while leaving model weights untouched.
β
Checkpoint 1.1: Explain, without looking anything up, why calling .backward()
twice on the same graph (without retain_graph=True
) throws an error. (Answer: the graph is freed after the first backward pass to save memory.)
nn.Module
nn.Module
is a convenience wrapper. Underneath, a "layer" is nothing more than: take an input, multiply by a weight matrix, add a bias, pass through a nonlinearity. A "network" is several of those chained. "Training" is: compute a loss, get its gradient with respect to every weight, nudge each weight opposite to its gradient. Building this by hand once is the single highest-leverage exercise in this project.
Fit y = sin(x)
on x
in [-Ο, Ο]
with manual parameters:
import torch
torch.manual_seed(0)
W1 = ... # TODO: (1,32), requires_grad=True, small random init
b1 = ... # TODO: (32,)
W2 = ... # TODO: (32,1)
b2 = ... # TODO: (1,)
def forward(x):
...
return z2
def mse_loss(pred, target):
...
lr = 0.05
for step in range(2000):
x = torch.linspace(-3.14, 3.14, 200).unsqueeze(1)
y_true = torch.sin(x)
pred = forward(x)
loss = mse_loss(pred, y_true)
...
if step % 200 == 0:
print(step, loss.item())
Loss will not decrease if you forget to zero gradients β PyTorch accumulates them into .grad
by default. Once burned by this, you'll never forget zero_grad()
again β you'll understand why, not just cargo-cult it. Then plot forward(x)
against sin(x)
: a sine wave emerges from 4 matrices you initialized as noise.
This loop β forward, loss, zero grad, backward, manual update β is structurally identical to the loop you write in Phase 3 for optimizing node coordinates. Only what the "parameters" represent changes.
π€ The Struggle: You'll likely hit RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
, usually from overwriting a leaf tensor without torch.no_grad()
. Print .requires_grad
/ .is_leaf
on every parameter right before the failing line.
An optimizer answers: given a gradient, how exactly should I change the parameter? Plain SGD (subtract lr*grad
) works but is slow and oscillates on ravine-shaped losses.
class ManualSGD:
def __init__(self, params, lr=0.01, momentum=0.0):
self.params = list(params)
self.lr = lr
self.momentum = momentum
self.velocities = ... # TODO: zeros_like buffer per param
def step(self):
with torch.no_grad():
for p, v in zip(self.params, self.velocities):
if p.grad is None: continue
...
def zero_grad(self):
for p in self.params:
if p.grad is not None: p.grad.zero_()
class ManualAdam:
def __init__(self, params, lr=0.001, betas=(0.9, 0.999), eps=1e-8):
self.params = list(params)
self.lr, self.b1, self.b2, self.eps, self.t = lr, *betas, eps, 0
self.m = ... # TODO: zeros_like per param (first moment)
self.v = ... # TODO: zeros_like per param (second moment)
def step(self):
self.t += 1
with torch.no_grad():
for i, p in enumerate(self.params):
if p.grad is None: continue
g = p.grad
...
def zero_grad(self):
for p in self.params:
if p.grad is not None: p.grad.zero_()
Re-run the Week 2 sine-fit with ManualAdam
, compare convergence speed to vanilla SGD.
Adam converges in hundreds of steps where SGD needed thousands, and is far less sensitive to learning rate choice. This robustness is why you'll reach for it again in Phase 3, where the loss surface (Map Clarity Loss over 2D positions) is even messier.
Your Phase 3 CoordinateAdam
class will be a near copy of this one β same math, different thing being optimized.
π‘ The Innovation (early preview): "I wrote my own Adam optimizer for coordinate updates because I needed to freeze individual coordinates mid-optimization when a user grabs a node β the standard torch.optim API doesn't expose that cleanly." Keep this for Chapter 3.
β Checkpoint 1.2: Plot loss curves for SGD, SGD+momentum, RMSprop, Adam β same problem, same init, same step count. If Adam isn't fastest, check bias-correction first.
A Transformer's core operation: for every token, compute a weighted average of every other token's representation, where weights ("attention") are learned and reflect relevance. Everything else β multi-head, positional encoding, layer norm, feedforward β is scaffolding.
Analogy: a room full of people (tokens) forming an opinion on a topic β you weight each person's input by relevance. Query = what you're looking for; Key = what each token offers; Value = what it contributes if attended to.
import torch, math
import torch.nn.functional as F
def scaled_dot_product_attention(Q, K, V, mask=None):
d_k = Q.shape[-1]
scores = ... # TODO: Q @ K.T / sqrt(d_k)
if mask is not None:
scores = scores.masked_fill(mask == 0, float('-inf'))
weights = ... # TODO: softmax(scores, dim=-1)
output = ... # TODO: weights @ V
return output, weights
python
class TinyMultiHeadAttention(torch.nn.Module):
def __init__(self, d_model, n_heads):
super().__init__()
assert d_model % n_heads == 0
self.d_k, self.n_heads = d_model // n_heads, n_heads
... # TODO: four Linear layers W_q, W_k, W_v, W_o
def forward(self, x, mask=None):
... # TODO: project, reshape to heads, attend per head, concat, W_o
class TinyEncoderLayer(torch.nn.Module):
def __init__(self, d_model, n_heads, d_ff):
super().__init__()
self.attn = TinyMultiHeadAttention(d_model, n_heads)
self.norm1 = torch.nn.LayerNorm(d_model)
self.ff = torch.nn.Sequential(
torch.nn.Linear(d_model, d_ff), torch.nn.ReLU(),
torch.nn.Linear(d_ff, d_model))
self.norm2 = torch.nn.LayerNorm(d_model)
def forward(self, x, mask=None):
... # TODO: x = norm1(x + attn(x)) ; x = norm2(x + ff(x))
Train on next-character prediction over a small corpus (a few historical-text paragraphs fits the theme).
Visualize attention weights for one sentence as a heatmap β rows should light up on semantically relevant tokens (e.g. a pronoun attending to the noun it refers to). Attention stops being a diagram and becomes something you watch your model do.
In Chapter 2 you'll swap this toy encoder for a pretrained BERT, understanding exactly what happens inside every layer because you built the smallest version yourself.
π³οΈ The Rabbit Hole: Attention has no inherent sense of token order. Read the sinusoidal positional encoding and ask why sine/cosine rather than a learned embedding.
β
Checkpoint 1.3 β End of Phase 1: From memory, sketch a Transformer encoder layer's forward pass, and explain what breaks if you remove residual connections, layer norm, or the sqrt(d_k)
scaling.
By end of Week 4, from raw tensor operations, you have: (1) intuitive autograd understanding, (2) a hand-built 2-layer network with manual gradient updates, (3) three hand-built optimizers, (4) a working tiny Transformer with visualized attention. Your Week 3 optimizers become the Phase 3 coordinate optimizer almost verbatim; your Week 4 Transformer understanding is what lets you debug BERT rather than treat it as a black box.
(Weeks 5β8 Β· Prerequisite: Phase 1 complete, comfort reading model architecture diagrams)
Your tiny Transformer from Week 4 learned character prediction from scratch on almost no data. BERT is the same architecture, scaled up (12 layers, 12 heads, 768-dim), pretrained on billions of words. Fine-tuning means: keep the pretrained weights as a strong starting point, attach a small task-specific head on top (here, a linear layer mapping each token's final hidden state to a label like B-EVENT
, I-EVENT
, B-DATE
, O
), and train the whole thing (or just the head) on your labeled data for a few epochs.
The reason this works with so little labeled data compared to training from scratch: BERT already learned general-purpose language structure from pretraining (grammar, word relationships, some world knowledge). Fine-tuning just teaches it to route that existing knowledge toward your specific labeling scheme.
from transformers import BertTokenizerFast, BertForTokenClassification
import torch
tokenizer = ...
model = ...
def tokenize_and_align_labels(text, word_labels):
"""
text: list of words, e.g. ["The", "assassination", "of", "Franz", "Ferdinand"]
word_labels: list of label ids, one per word
Returns tokenized input + labels aligned to WordPiece subtokens
(subtokens after the first get label -100 so they're ignored in the loss)
"""
...
Run inference on a sentence the model never saw during fine-tuning and watch it correctly tag "the outbreak of war" as an EVENT span and "1914" as a DATE span, purely from ~100 examples. That's the pretraining transfer working β this would be essentially impossible to get right training from random initialization on 100 sentences.
This NER output β spans tagged as EVENT, DATE, PERSON, PLACE β is exactly what Week 6's relation extraction will pair up into causal triples.
π€ The Struggle: Label alignment between words and WordPiece subtokens is the single most common source of silent bugs in NER fine-tuning β a misaligned label doesn't crash, it just quietly trains the model on garbage. Always spot-check tokenizer.convert_ids_to_tokens()
against your aligned label array for a handful of examples before you trust any training run.
NER tells you what the entities are. Relation extraction tells you how they relate. The simplest approach: for every pair of entities that co-occur in the same sentence (or a short window of sentences), feed their contextualized representations (plus the sentence itself) into a classifier that predicts one of a fixed set of relation labels: CAUSES
, ENABLED_BY
, PRECEDES
, PART_OF
, or NONE
.
A more powerful approach β REBEL β does entity and relation extraction jointly, generating the full set of triples as a structured text sequence in one pass, rather than requiring you to first extract entities, then classify every pair. It's a strong upgrade path once your simpler pairwise classifier is working and you understand why the joint approach is more efficient (it doesn't blow up combinatorially with the number of entities in a sentence).
Start with the pairwise classifier β it's the better learning exercise, even though REBEL is more production-capable:
The output for that Franz Ferdinand sentence should be CAUSES
with high confidence. Then feed it a sentence with two entities that are merely mentioned near each other with no causal link ("The war began in 1914. Assassinations were common in the region.") and watch it correctly predict NONE
. That contrast β the model discriminating relevance, not just proximity β is the whole point of relation extraction over naive "entities near each other are related" heuristics.
Every (entity_1, relation, entity_2)
triple this produces becomes an edge in the causal graph that Phase 3's spatial engine will lay out.
π³οΈ The Rabbit Hole: Read about "distant supervision" for relation extraction β a technique for auto-generating noisy training labels by aligning a knowledge base (like Wikidata) against text, instead of hand-labeling everything. Useful if your ~100 hand-labeled examples aren't enough once you scale up.
A pipeline is only as trustworthy as its weakest stage boundary. This week isn't about new ML β it's about wiring Weeks 5 and 6 together into something that reliably takes raw pasted text and emits a clean, deduplicated, database-ready set of triples, with sane error handling for the inevitable garbage input (empty text, non-English text, text with no extractable events).
class ExtractionPipeline:
def __init__(self, ner_model, relation_model, tokenizer):
...
def extract(self, raw_text: str) -> list[dict]:
"""
Returns: [{"subject": "...", "relation": "CAUSES", "object": "...",
"subject_date": "1914-06-28" or None, ...}, ...]
"""
...
Set up PostgreSQL with pgvector:
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE events (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
text TEXT NOT NULL,
date_text TEXT,
embedding VECTOR(768), -- BERT's hidden size
created_at TIMESTAMPTZ DEFAULT now()
);
CREATE TABLE relations (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
source_event_id UUID REFERENCES events(id),
target_event_id UUID REFERENCES events(id),
relation_type TEXT NOT NULL,
confidence FLOAT
);
Paste in three unrelated paragraphs of history (say, one on WWI causes, one on the French Revolution, one on the fall of Rome) and watch the pipeline correctly keep the triples from each topic separate, with no spurious cross-topic relations β because relation extraction is confidence-thresholded and entity embeddings from unrelated topics don't get spuriously matched.
This pipeline is the entire backend of the /api/extract
endpoint from Chapter 0's data flow diagram. Everything downstream (Phase 3, Phase 4) consumes its output.
β Checkpoint 2.1: Run your pipeline on a genuinely messy input (a Wikipedia paragraph with footnote markers, inconsistent date formats, nested clauses) and verify it degrades gracefully β no crash, just fewer/lower-confidence triples β rather than throwing an unhandled exception.
An extraction pipeline that "looks right" on your three favorite test paragraphs is not the same as one that's reliable. This week is about building a small held-out evaluation set (hand-labeled, separate from training data) and computing real metrics: precision, recall, and F1 for both entity spans and relation classification.
def evaluate_ner(model, eval_sentences, eval_labels):
...
def evaluate_relations(model, eval_pairs, eval_labels):
...
Your confusion matrix will likely show CAUSES
and ENABLED_BY
bleeding into each other β this is expected and informative, not a bug. It tells you those two categories are genuinely hard to distinguish from surface text alone (a human annotator would disagree with themselves on some of these too), which should inform how confidently the frontend displays that distinction later (e.g., maybe you merge them into one edge style with a subtler visual difference rather than two starkly different arrow types).
Phase 2 milestone: a real, measured, imperfect-but-quantified extraction system, feeding real triples into a real database.
You now have a fine-tuned BERT NER model, a relation extraction classifier, a hardened pipeline connecting them, a Postgres+pgvector store, and honest evaluation metrics on held-out data. This is a legitimate, demoable NLP system on its own β worth pausing to appreciate before diving into Phase 3, which is where the project becomes genuinely novel.
(Weeks 9β12 Β· Prerequisite: Phase 1 and 2 complete. This is the conceptual core of ChronoWeave β take this chapter slowly.)
Everything before this point β the Transformer, the NER model, the relation extractor β is, architecturally, "standard" deep learning applied to a specific domain. Impressive to build from scratch, but conceptually well-trodden. This chapter is where ChronoWeave does something genuinely uncommon: using gradient descent as a general-purpose layout algorithm, with coordinates as first-class trainable parameters. Take this slowly. It's the part of the project you'll actually be excited to explain in an interview.
Every optimization problem in ML has the same shape: parameters, a loss function measuring how bad the current parameters are, and an optimizer that nudges parameters to reduce that loss. So far, "parameters" has meant neural network weights. Nothing in that shape requires the parameters to be weights. A parameter is just a tensor with requires_grad=True
that appears somewhere in a differentiable computation whose output is your loss.
So: what if the parameters are the (x, y) positions of graph nodes on a canvas, and the loss is a hand-designed function that scores how "good" a layout is?
This reframes graph layout β traditionally solved with physics simulations (force-directed layout, spring-embedder algorithms) β as an optimization problem solvable with the exact same machinery you built in Phase 1.
Set up the skeleton, no loss function yet β just prove coordinates can be optimized at all:
import torch
num_nodes = 10
positions = ...
def toy_loss(positions):
return (positions ** 2).sum()
optimizer = torch.optim.Adam([positions], lr=0.1)
for step in range(100):
optimizer.zero_grad()
loss = toy_loss(positions)
loss.backward()
optimizer.step()
if step % 20 == 0:
print(step, loss.item())
Watching ten random points converge toward the origin using torch.optim.Adam
β the exact same optimizer class you'd use to train a neural network β is the moment this project's central idea stops being abstract. You just used backpropagation to solve a geometry problem. No physics, no forces, no simulation of springs β just calculus.
toy_loss
is a stand-in. Next week you replace it with the real Map Clarity Loss β but the training loop around it doesn't change at all.
π§ The Mentor Says: Resist the urge to jump straight to the full Map Clarity Loss this week. Build up loss terms one at a time, verify each one does what you expect in isolation (attraction only, then repulsion only, then combined), and only then combine everything. Debugging a five-term loss function that's never converged correctly at any point is miserable. Debugging a five-term loss function where you've verified each term separately is straightforward.
A good layout satisfies several competing objectives simultaneously:
Each of these becomes a differentiable term, and the total loss is a weighted sum:
L_total = w1 * L_attraction + w2 * L_repulsion + w3 * L_overlap + w4 * L_temporal
The weights (w1..w4
) are hyperparameters you'll tune empirically β this is exactly analogous to loss weighting in multi-task neural network training, another well-known hard problem, so don't be surprised if getting a visually pleasing balance takes real iteration.
def attraction_loss(positions, edges):
"""
edges: list of (i, j) node index pairs that are causally linked
Pulls linked nodes together β squared distance, like a spring
"""
i_idx = torch.tensor([e[0] for e in edges])
j_idx = torch.tensor([e[1] for e in edges])
...
def repulsion_loss(positions, min_distance=2.0):
"""
Pushes ALL pairs of nodes apart if they're closer than min_distance.
This is O(n^2) -- fine for the node counts ChronoWeave targets
(tens to low hundreds of events per graph), but know that this term
is the one that would need approximating (e.g. Barnes-Hut) at scale.
"""
n = positions.shape[0]
dist_matrix = ...
...
def overlap_loss(positions, box_size=1.0):
...
def temporal_loss(positions, dates_normalized):
"""
dates_normalized: tensor of shape (n,), each event's date mapped to
[0, 1] (earliest event=0, latest=1)
Soft-encourages x-coordinate to correlate with date
"""
...
def map_clarity_loss(positions, edges, dates_normalized, weights):
w1, w2, w3, w4 = weights
return (w1 * attraction_loss(positions, edges)
+ w2 * repulsion_loss(positions)
+ w3 * overlap_loss(positions)
+ w4 * temporal_loss(positions, dates_normalized))
Run the full optimization on a real extracted graph (from your Phase 2 pipeline!) with all four terms weighted, and watch causally-connected clusters visually separate from unrelated clusters, while individual nodes within a cluster stay legibly spaced apart. This is the actual "map" in ChronoWeave β and unlike a force-directed layout from a library, you can explain precisely why every node ended up where it did, because you wrote every term of the objective yourself.
This is the function whose gradient β with respect to positions
, not any neural network weight β drives literally everything the user sees on the canvas, and everything that happens when they drag a node in Chapter 4.
π€ The Struggle: The most common failure mode here is loss term imbalance β e.g. repulsion dominating so hard that attraction can't pull anything together, producing a uniform grid-like scatter with no visible clustering. When this happens, don't guess-and-check weights blindly. Log each individual loss term's value (not just the weighted sum) every N steps, and look at their relative magnitudes. If repulsion_loss
is naturally 50x the scale of attraction_loss
just from how you defined it (e.g. squared distances over many more pairs), your weights need to correct for that scale difference before they can express your actual priorities between the terms.
torch.cdist
docs: β Checkpoint 3.1: Take a graph with two clearly separate causal clusters (e.g. WWI causes and, unrelated, causes of the 2008 financial crisis, extracted from two different pasted texts) and verify optimization produces two visually distinct, non-overlapping clusters β not because you told it to, but because attraction pulls each cluster's internal nodes together while repulsion pushes the two clusters apart as aggregate masses.
Two refinements this week. First: gradient hooks. PyTorch lets you register a function that runs every time a gradient is computed for a specific tensor, which is how you'll implement per-node learning rate decay, gradient clipping for numerically unstable nodes, or debugging instrumentation (logging exactly which nodes have the largest gradients at each step β useful for diagnosing which part of the layout is "fighting" hardest).
Second: formalize your Phase 1 ManualAdam
into a purpose-built CoordinateAdam
that supports freezing individual coordinates β critical for Chapter 4's drag-and-reoptimize feature, where the dragged node must stay exactly where the user put it while every other node reoptimizes around it.
def make_logging_hook(node_names):
def hook(grad):
...
return grad # must return grad unchanged (or modified) -- don't return None
return hook
positions.register_hook(make_logging_hook(node_names))
class CoordinateAdam:
"""
Like your Phase 1 ManualAdam, but supports freezing a subset of
coordinates (e.g. a node the user just dragged).
"""
def __init__(self, positions: torch.Tensor, lr=0.05, betas=(0.9, 0.999), eps=1e-8):
self.positions = positions
self.lr, self.b1, self.b2, self.eps, self.t = lr, *betas, eps, 0
self.m = torch.zeros_like(positions)
self.v = torch.zeros_like(positions)
self.frozen_mask = torch.zeros(positions.shape[0], dtype=torch.bool)
def freeze(self, node_idx):
self.frozen_mask[node_idx] = True
def unfreeze(self, node_idx):
self.frozen_mask[node_idx] = False
def step(self):
self.t += 1
with torch.no_grad():
g = self.positions.grad
g = g.clone()
g[self.frozen_mask] = 0.0
self.m = self.b1 * self.m + (1 - self.b1) * g
self.v = self.b2 * self.v + (1 - self.b2) * g ** 2
m_hat = self.m / (1 - self.b1 ** self.t)
v_hat = self.v / (1 - self.b2 ** self.t)
update = self.lr * m_hat / (v_hat.sqrt() + self.eps)
update[self.frozen_mask] = 0.0
self.positions -= update
def zero_grad(self):
if self.positions.grad is not None:
self.positions.grad.zero_()
Freeze one node mid-optimization (simulate a user drag by just calling .freeze(node_idx)
and manually setting that row of positions
to a fixed target), re-run .step()
for another 100 iterations, and watch every other node smoothly re-flow into a new stable configuration around the pinned node β while the pinned node itself doesn't move a single pixel, even though it still has a nonzero gradient every step.
This freeze/unfreeze mechanism is exactly the backend logic Chapter 4's WebSocket drag handler calls into. You are not building new logic in Phase 4 β you're wiring this class up to a live event stream.
π‘ The Innovation: This is the strongest, most concrete "why is this novel" answer for the whole project: "Standard force-directed layout libraries treat a dragged node as an external constraint bolted onto a physics simulation β usually by literally fixing its position and letting the simulation tick forward. My approach treats the drag as freezing one parameter's gradient within the exact same optimization loop used to generate the original layout, which means the re-layout objective is provably the same objective function, just re-solved under an added constraint β not a different, ad hoc mechanism." That's a real distinction, not a marketing line, and you can defend it because you wrote both halves.
register_hook
API: Gradient-based layout fails in ways that are different from β and often more confusing than β a standard training loop failing, because there's no "accuracy" metric to sanity check against, only a loss value and a visual result. This week is a deliberate practice week: you'll intentionally break your Week 10β11 system in several ways and learn to recognize the symptoms.
lr
by 10x before assuming it's a logic bug)torch.cdist
produces a zero distance for a node compared to itself; if your repulsion loss divides by distance anywhere instead of just using squared distance, you'll get a divide-by-zero on the diagonal. Always double-check you've masked the diagonal.)positions
and print per-node gradient norms; the runaway node almost always has an anomalously large gradient from a bug specific to its edges, e.g. it participates in zero attraction edges so repulsion has nothing to balance against)After deliberately inducing and then fixing all four failure modes above, run your full pipeline end-to-end β real text in, real triples extracted, real layout optimized β and watch it just... work, cleanly, without you needing to touch a single hyperparameter mid-run. That reliability is earned specifically by having debugged each failure mode once on purpose, rather than encountering them for the first time under deadline pressure in Phase 4.
Phase 3 milestone: a working, debugged spatial intelligence engine that takes a causal graph and produces a legible 2D layout via gradient descent, with the ability to freeze nodes and re-optimize live.
π§ The Mentor Says: Keep a running "failure log" as a markdown file next to your code: every bug you hit, what the symptom looked like, and what actually fixed it. Six months from now when this happens again in a different project, that log is worth more than any Stack Overflow search, because it's your debugging vocabulary for this specific class of problem (gradient-based systems with non-standard parameters), which almost nobody else has written down anywhere.
β Checkpoint 3.2 β End of Phase 3: Explain to a rubber duck (or a friend, or a mirror) why gradient descent on coordinates is a valid approach to graph layout at all β i.e., why the loss function's gradient with respect to position tells you a meaningful direction to move a node, given that "position" isn't something with an obvious ground truth the way a classification label is. (The core answer: there's no ground truth position, but there's a well-defined relative preference β closer is better for linked nodes, farther is better for unlinked ones β and gradient descent is a general algorithm for finding local optima of any differentiable preference function, not just supervised losses with ground truth targets.)
(Weeks 13β16 Β· Prerequisite: Phase 3 complete)
Two distinct interaction patterns need two different transport mechanisms. The initial "paste text, click Generate" flow is a classic request/response β REST fits fine. But the drag-and-reoptimize flow needs the server to push a stream of intermediate frames to the client as optimization runs (so the user sees nodes flow smoothly into place rather than jumping instantly), which REST can't do β that's what WebSockets are for.
from fastapi import FastAPI, WebSocket
from fastapi.responses import JSONResponse
app = FastAPI()
@app.post("/api/extract")
async def extract(payload: dict):
text = payload["text"]
...
@app.websocket("/ws/relayout/{graph_id}")
async def relayout(websocket: WebSocket, graph_id: str):
await websocket.accept()
drag_event = await websocket.receive_json()
for step in range(200):
if step % 5 == 0:
...
await websocket.close()
Open your browser's network tab, trigger a drag, and literally watch a stream of JSON frames arrive over the WebSocket connection every few milliseconds β the exact same optimization loop from Chapter 3, just now visible as it happens, one HTTP-adjacent message at a time.
This is the wiring that turns Chapter 3's offline optimization script into a live, interactive feature.
D3 excels at binding data to SVG elements and handling enter/update/exit transitions smoothly β exactly what you need when node positions update every few WebSocket frames. React manages component state and lifecycle; D3 manages the actual SVG manipulation and smooth transitions between positions. The common pattern: let React own the DOM structure (which nodes/edges exist), let D3 own the transitions (how a node's x/y animates from old position to new).
// TODO: a GraphCanvas component that:
// 1. Receives `nodes` and `edges` as props (from the /api/extract response)
// 2. Renders SVG circles for nodes, lines for edges, using D3 scales
// to map data coordinates to screen coordinates
// 3. Uses d3.transition() to animate position changes smoothly whenever
// the `nodes` prop updates (i.e., whenever a new WebSocket frame arrives)
// 4. Attaches d3.drag() behavior to each node, which on 'end' fires a
// callback (e.g. onNodeDragged(nodeId, newX, newY)) that the parent
// component uses to open the /ws/relayout WebSocket and stream in
// the live re-layout frames from Week 13
function GraphCanvas({ nodes, edges, onNodeDragged }) {
// TODO: useRef for the svg element, useEffect that runs D3 rendering
// logic whenever `nodes` or `edges` change
}
Drag a node and watch not just that node move, but every connected node smoothly animate to its new position over the following second or two β not teleporting, not choppy, an actual fluid re-settling. This is the "magic moment" the entire project overview promised in Chapter 0, now real in a browser.
Nothing further β this is the product. Everything from Chapter 1 onward has been building toward this specific ten seconds of interaction.
π€ The Struggle: React re-rendering and D3 both wanting to own the DOM is a classic source of fights β D3 mutates elements directly, React expects to control them via its virtual DOM diff. The cleanest fix most people converge on: let React render the SVG container and static structure, but hand D3 a ref to manipulate node/edge elements directly inside a useEffect
, and never let React's render also try to set those same attributes. Pick one owner per DOM property.
Docker Compose ties your FastAPI backend, Postgres+pgvector database, and (optionally) a separate model-serving container into one reproducible unit you can run locally exactly as it'll run in the cloud β closing the "works on my machine" gap before it costs you a demo day.
services:
db:
image: pgvector/pgvector:pg16
environment:
POSTGRES_PASSWORD: ...
volumes:
- pgdata:/var/lib/postgresql/data
backend:
build: ./backend
depends_on:
- db
environment:
DATABASE_URL: postgresql://...@db:5432/...
ports:
- "8000:8000"
frontend:
build: ./frontend
ports:
- "3000:3000"
volumes:
pgdata:
docker-compose up
on a completely fresh machine (or a cloud VM) and the whole system β model inference, database, backend, frontend β comes up correctly with zero manual steps beyond that one command. That's the difference between "a project on my laptop" and "a project."
Deploy to Render, Fly.io, or a similar platform for a public demo URL β essential for the hackathon-ready, portfolio-ready version of this project.
The last week is deliberately not about new features. It's about the unglamorous 20% that makes a demo feel finished: states while extraction/optimization runs, graceful handling of bad input, a couple of pre-baked example texts a judge or interviewer can one-click try instead of needing to paste their own, and a README that explains the project in 30 seconds.
/api/extract
is in flight (this can take several seconds β a live optimization loop is not instant, and users need to know it's working, not frozen)Show the finished product to someone who has never seen it, say nothing, and watch them paste in their own text and drag a node without prompting. If they intuitively understand what happened without you explaining it, the product design succeeded.
A live, deployed, full-stack application: paste historical text, get an extracted causal graph laid out via gradient descent, drag any node and watch the rest of the graph re-optimize in real time β all running on infrastructure you understand end to end, built from raw tensor operations up.
Lead with the live demo, not the tech stack list. The first ten seconds should be someone pasting text and dragging a node β let the magic moment sell itself before you explain any internals.
The "gradient descent as general-purpose graph layout" idea, generalized beyond history specifically, touches on active research areas: differentiable graph drawing, learned layout objectives, and combining symbolic graph constraints with continuous optimization. A write-up comparing your Map Clarity Loss approach against classical force-directed layout on layout-quality metrics (edge crossing count, node overlap, cluster separation) would be a legitimate small research contribution, or at minimum a strong technical blog post.
Current design targets tens to low-hundreds of nodes per graph (the O(nΒ²)
repulsion term is the binding constraint). Scaling further means either approximating repulsion (Barnes-Hut / quadtree methods, the same trick classical force-directed layout libraries use at scale) or moving to a hierarchical layout where clusters are optimized independently and then composed.
curl -sSL https://install.python-poetry.org | python3 -
poetry --version # verify install
mkdir chronoweave && cd chronoweave
poetry init --name chronoweave-ml -n
poetry add torch numpy matplotlib jupyter transformers
poetry run python -c "import torch; print(torch.cuda.is_available())"
poetry run jupyter notebook
If you don't have a GPU: everything in Phase 1 and most of Phase 3 runs fine on CPU (small networks, small graphs). Phase 2's BERT fine-tuning is the one place a GPU meaningfully helps β Google Colab's free tier is sufficient for the scale of fine-tuning this project needs.
| Symptom | Likely Cause | Fix |
|---|---|---|
| Loss doesn't decrease at all | Forgot zero_grad() before .backward() |
|
Gradients are accumulating across steps β add zero_grad() at the top of the loop |
||
RuntimeError: ...does not require grad and does not have a grad_fn |
||
A leaf tensor got overwritten outside torch.no_grad() |
||
Check every in-place parameter update is inside with torch.no_grad(): |
||
Loss becomes NaN after N steps |
||
| Divide-by-zero, usually in a pairwise-distance loss term where the diagonal (self-distance = 0) isn't masked | Mask the diagonal before dividing by any distance term | |
| Everything collapses to a point during layout optimization | Repulsion term isn't actually contributing (masking bug or weight of 0) | Log each loss term separately, verify repulsion is nonzero and comparable in scale to attraction |
NER model predicts O for everything |
||
| Label misalignment between words and WordPiece subtokens | Spot-check tokenizer.convert_ids_to_tokens() against your label array on several examples |
|
| WebSocket disconnects mid-reoptimization | Sending too many frames too fast, or an unhandled exception mid-loop killing the coroutine | Throttle to sending every 5th step, wrap the optimization loop in try/except with a graceful websocket.close() on error |
| D3 nodes flicker or don't animate smoothly | React re-render fighting with D3's direct DOM manipulation | Ensure only one of React/D3 sets a given DOM attribute; do D3 updates inside useEffect , not render |
| Docker Compose backend can't reach the database | Using localhost instead of the service name in DATABASE_URL |
|
Compose networking resolves service names (db ), not localhost , between containers |
Week: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Phase 1 ββ ββ ββ ββ
Autograd ββ
Manual NN ββ
Optimizers ββ
Tiny Xformer ββ
Phase 2 ββ ββ ββ ββ
NER fine-tune ββ
Relation extract ββ
Data pipeline ββ
Evaluation ββ
Phase 3 ββ ββ ββ ββ
Coords as params ββ
Map Clarity Loss ββ
Grad hooks + optimizer ββ
Debugging practice ββ
Phase 4 ββ ββ ββ ββ
FastAPI + WS ββ
React + D3 ββ
Deployment ββ
Polish + demo ββ
Phase 1 (correctness of understanding, not the app):
torch.optim.Adam
on the same toy problemPhase 2 (quantitative, on your held-out eval set):
CAUSES
specifically (the class you care most about); lower is acceptable for rarer/harder classes like ENABLED_BY
Phase 3 (behavioral, verified visually + numerically):
Phase 4 (product-level):
The 30-second version:
"I built a system that reads historical text, extracts cause-and-effect relationships between events using a fine-tuned Transformer, and lays those events out on a 2D map using gradient descent instead of a physics engine β meaning the layout is generated by backpropagating a custom loss function with respect to node coordinates, the same way you'd train a neural network, except the 'parameters' are positions on a canvas instead of weights."
Anticipated follow-up questions and how to answer them:
"Why not just use a force-directed graph library?" β Because I needed the layout objective to express more than physical forces can naturally capture: e.g., a soft temporal-ordering constraint alongside causal clustering, with tunable relative weights between competing goals, and the ability to freeze arbitrary subsets of nodes mid-optimization while re-solving the same objective for everyone else. Gradient descent on a custom differentiable loss generalizes to all of that; a spring simulation would need a bespoke mechanism bolted on for each one.
"How did you handle relation extraction being noisy?" β I measured it rather than assumed it: held-out precision/recall/F1 per relation class, a confidence threshold before a triple gets displayed, and I designed the frontend to reflect uncertainty where the model itself is uncertain (e.g., visually softer distinction between relation types the model confuses most, per the confusion matrix).
"What was the hardest bug?" β Talk about one specific failure mode from Chapter 3 Week 12 in detail (e.g., the NaN-from-unmasked-diagonal bug) β specificity here is what separates "I used ML" from "I understand ML."
"Why build the optimizer from scratch instead of using torch.optim.Adam directly?" β Because I needed per-node freezing during live re-layout, which meant controlling exactly how the moment buffers and update step interacted with a frozen mask β not something the standard API exposes, and writing it myself meant I fully understood what I was modifying.
"What would you do differently at scale?" β The repulsion loss term is O(nΒ²); past a few hundred nodes I'd move to an approximate method (Barnes-Hut quadtree, the same technique classical force-directed layouts use), or a hierarchical approach that optimizes clusters independently before composing them.
End of guide. This document covers the full architecture, all four learning phases, and the operational appendices for building ChronoWeave. Treat each chapter's code skeletons as a starting point to fill in and argue with β the goal was never for you to run code someone else wrote, but to be the one who could have written PyTorch's autograd yourself, at least once, in miniature.