ChronoWeave: The Documentary

ChronoWeave is a system that extracts causal graphs from historical text and lays them out as interactive maps, using a neural network to position causally-connected events close together. The project's spatial intelligence engine treats node coordinates as trainable tensors and defines a 'Map Clarity Loss' to optimize layout.

"Every timeline you've ever seen is a lie of omission. It shows you WHEN things happened. It never shows you WHY. ChronoWeave is our attempt to fix that — and along the way, you're going to learn what actually happens inside a neural network when you call .backward ." This is not a tutorial you skim. It's a companion for a 3–5 month build. Read one section, do the exercise, hit the wall, climb over it, then come back. Every chapter follows the same rhythm: And threaded throughout: Let's begin. Open any history textbook's timeline. You'll see a horizontal line, some dots, some dates. 1914. 1917. 1929. 1939. It tells you when . It is silent on why . The assassination of Archduke Franz Ferdinand didn't cause World War I in a vacuum — it was the spark that landed on a decade of alliance treaties, arms races, and colonial tension that had already turned the continent into dry kindling. A flat timeline shows you the spark. It hides the kindling. What historians actually think in is a causal graph : event A enabled event B, which, combined with condition C, triggered event D. That graph is tangled, non-linear, and multi-dimensional — which is exactly why nobody draws it by hand. It's too much cognitive load. ChronoWeave's bet: if a machine can extract the causal graph from raw text, and then lay it out so that causally-connected events cluster together and pull each other into readable arrangement, you get something a static timeline can never give you — a map you can explore, drag, and reorganize, where the layout itself is meaningful. Events that influence each other stay near each other. That's not a UI nicety. That's the entire value proposition. Here's the complete system, before we write a single line of code: ┌─────────────────────────────────────────────────────────────────────┐ │ THE USER'S BROWSER │ │ ┌──────────────┐ ┌────────────────────────────────────┐ │ │ │ Text Input │──────▶│ React + D3.js Canvas │ │ │ │ "Paste │ │ renders nodes = events, │ │ │ │ history │ │ edges = causal links │ │ │ │ here" │ │ │ │ │ └──────────────┘ │ User drags a node ───────┐ │ │ │ └─────────────────────────────┼─────────┘ │ └──────────────────────────────────────────────────────────┼──────────────┘ │ POST /extract │ WS: node moved ▼ ▼ ┌─────────────────────────────────────────────────────────────────────┐ │ FASTAPI BACKEND │ │ │ │ ┌────────────────────┐ ┌───────────────────────────────────┐ │ │ │ EXTRACTION ENGINE │ │ SPATIAL INTELLIGENCE ENGINE │ │ │ │ Chapter 2 │ │ Chapter 3 — the heart of it │ │ │ │ │ │ │ │ │ │ 1. Split into │ │ 1. Build a graph: nodes=events, │ │ │ │ sentences │ │ edges=causal relations │ │ │ │ 2. Run NER │────▶│ 2. Init x,y for every node as a │ │ │ │ BERT to find │ │ TRAINABLE TENSOR │ │ │ │ events, dates, │ │ requires grad=True │ │ │ │ people, places │ │ 3. Define "Map Clarity Loss": │ │ │ │ 3. Run Relation │ │ - causally linked nodes should │ │ │ │ Extraction to │ │ be CLOSE │ │ │ │ find "A causes B" │ │ - unrelated nodes should be │ │ │ │ 4. Output: list of │ │ FAR repulsion │ │ │ │ event, date, │ │ - nothing should overlap │ │ │ │ relation, event │ │ 4. Run backprop on the │ │ │ │ triples │ │ COORDINATES not the model's │ │ │ │ │ │ weights using a hand-written │ │ │ │ │ │ Adam optimizer │ │ │ │ │ │ 5. When user drags a node, PIN │ │ │ │ │ │ that node's coords, re-run │ │ │ │ │ │ optimization on everyone else │ │ │ └────────────────────────┘ └───────────────────────────────────┘ │ │ │ │ │ │ ▼ ▼ │ │ ┌────────────────────────────────────────────────────────────────┐ │ │ │ PostgreSQL + pgvector stores triples + embeddings │ │ │ └────────────────────────────────────────────────────────────────┘ │ └─────────────────────────────────────────────────────────────────────┘ Walk through it end to end, because this sequence is the skeleton every later chapter hangs off of: /api/extract . CAUSES , ENABLED BY , PRECEDES , PART OF , or NONE . Assassination of Franz Ferdinand, CAUSES, Austria-Hungary ultimatum to Serbia . These get written to Postgres, and each event also gets a requires grad=True . This is the part that makes ChronoWeave unusual: the .backward , get gradients node id, x, y positions back to the frontend over the initial REST response or WebSocket, for the live re-layout case . requires grad=False for it, fixed at the drag target . The backend re-runs the optimizer on every That last step is the entire reason this project exists. A physics engine force-directed graph layout, e.g. D3's built-in forceSimulation can also do node repulsion and edge attraction — but it can't easily express arbitrary, learned, semantically-weighted objectives e.g., "cluster by decade AND by causal chain AND penalize edge crossings, with weights that adapt based on graph density" . Gradient descent on a custom loss can. That's the pitch. We'll come back to defending it rigorously in Chapter 3. | Layer | Choice | Why | |---|---|---| | ML core | PyTorch from raw tensors, then nn.Module later | You need autograd exposed at the tensor level to do coordinate optimization; PyTorch's dynamic graph makes this natural | | NER / Relation Extraction | Fine-tuned BERT-base, later maybe REBEL for joint extraction | BERT is small enough to fine-tune on a single GPU and well documented; REBEL is a strong upgrade path for joint entity+relation extraction | | Backend | FastAPI + WebSockets | Async-native, typed, and WebSocket support is first-class for the live re-layout streaming | | Database | PostgreSQL + pgvector | Relational structure for triples, vector column for semantic similarity search on event embeddings | | Frontend | React + D3.js | D3 gives you full control over force/position rendering; React manages component state and drag events | | Deployment | Docker Compose → cloud Render/Fly.io/AWS | Reproducibility first, then scale | Four phases, each roughly a month, building strictly bottom-up: 🧭 The Mentor Says: Don't let the ambition of the final product distract you from the ugliness of the early steps. In week 2 you'll be manually computing gradients for a two-layer network with a pencil-and-paper feel to it, and it will seem impossibly far from "beautiful interactive graph app." That gap is supposed to be there. Every person who deeply understands PyTorch went through exactly this tunnel. You don't skip it by using nn.Module early — you skip understanding by doing that. Weeks 1–4 · Prerequisite: comfort with Python, basic linear algebra — matrix multiply, dot products — and derivatives from a first calculus course A tensor is just a multi-dimensional array with two superpowers bolted on: it knows how to run on a GPU, and it can remember the sequence of operations that produced it, so it can later compute how a tiny nudge to its inputs would change its output. That second superpower is autograd , and it's the single idea Phase 1 exists to demystify. Think of a tensor with requires grad=True as a spreadsheet cell that doesn't just hold a number — it holds a number and a formula referencing other cells. Change an input cell, and every downstream cell knows exactly how much it would change, without you re-deriving the formula by hand. That "how much would it change" is the gradient. Set up your environment first: poetry init poetry add torch numpy matplotlib jupyter poetry shell Then in a notebook, don't build anything yet — just observe autograd: python import torch TODO: create a tensor x = 3.0 with requires grad=True x = ... TODO: create y = x 2 + 2 x + 1 y = ... TODO: call y.backward and print x.grad Predict on paper what dy/dx should be at x=3 BEFORE you run this ... x.grad should equal 2 x + 2 = 8.0 — exactly the calculus-class derivative, computed automatically. The click isn't "wow it did calculus," it's realizing PyTorch didn't look up a symbolic formula for x 2 + 2 x + 1 — it recorded the sequence of primitive operations pow , mul , add as you executed them, and walked that sequence backward, applying the chain rule at each step. This recorded sequence is called the computation graph , and it's rebuilt fresh every time you run forward — which is why PyTorch is called "define-by-run." Every gradient ChronoWeave ever computes — whether it's a loss with respect to a BERT weight, or a loss with respect to a node's x, y position on the canvas — goes through this exact mechanism. There is no different code path for "coordinates" versus "weights." This is the whole trick behind Phase 3, twelve weeks from now. 🕳️ The Rabbit Hole: Read about torch.autograd.grad vs .backward — the former lets you compute gradients without accumulating them into .grad , important later when you want gradients with respect to coordinates specifically , while leaving model weights untouched. ✅ Checkpoint 1.1: Explain, without looking anything up, why calling .backward twice on the same graph without retain graph=True throws an error. Answer: the graph is freed after the first backward pass to save memory. nn.Module nn.Module is a convenience wrapper. Underneath, a "layer" is nothing more than: take an input, multiply by a weight matrix, add a bias, pass through a nonlinearity. A "network" is several of those chained. "Training" is: compute a loss, get its gradient with respect to every weight, nudge each weight opposite to its gradient. Building this by hand once is the single highest-leverage exercise in this project. Fit y = sin x on x in -π, π with manual parameters: php import torch torch.manual seed 0 Architecture: 1 - 32 - 1, tanh activation W1 = ... TODO: 1,32 , requires grad=True, small random init b1 = ... TODO: 32, W2 = ... TODO: 32,1 b2 = ... TODO: 1, def forward x : TODO: z1 = x @ W1 + b1 ; a1 = tanh z1 ; z2 = a1 @ W2 + b2 ... return z2 def mse loss pred, target : TODO: mean squared error, raw tensor ops ... lr = 0.05 for step in range 2000 : x = torch.linspace -3.14, 3.14, 200 .unsqueeze 1 y true = torch.sin x pred = forward x loss = mse loss pred, y true TODO: zero old grads, loss.backward , manually update all 4 params wrap update in with torch.no grad : ... if step % 200 == 0: print step, loss.item Loss will not decrease if you forget to zero gradients — PyTorch accumulates them into .grad by default. Once burned by this, you'll never forget zero grad again — you'll understand why , not just cargo-cult it. Then plot forward x against sin x : a sine wave emerges from 4 matrices you initialized as noise. This loop — forward, loss, zero grad, backward, manual update — is structurally identical to the loop you write in Phase 3 for optimizing node coordinates. Only what the "parameters" represent changes. 😤 The Struggle: You'll likely hit RuntimeError: element 0 of tensors does not require grad and does not have a grad fn , usually from overwriting a leaf tensor without torch.no grad . Print .requires grad / .is leaf on every parameter right before the failing line. An optimizer answers: given a gradient, how exactly should I change the parameter? Plain SGD subtract lr grad works but is slow and oscillates on ravine-shaped losses. python class ManualSGD: def init self, params, lr=0.01, momentum=0.0 : self.params = list params self.lr = lr self.momentum = momentum self.velocities = ... TODO: zeros like buffer per param def step self : with torch.no grad : for p, v in zip self.params, self.velocities : if p.grad is None: continue TODO: v = momentum v - lr p.grad ; p += v ... def zero grad self : for p in self.params: if p.grad is not None: p.grad.zero class ManualAdam: def init self, params, lr=0.001, betas= 0.9, 0.999 , eps=1e-8 : self.params = list params self.lr, self.b1, self.b2, self.eps, self.t = lr, betas, eps, 0 self.m = ... TODO: zeros like per param first moment self.v = ... TODO: zeros like per param second moment def step self : self.t += 1 with torch.no grad : for i, p in enumerate self.params : if p.grad is None: continue g = p.grad TODO: m i = b1 m i + 1-b1 g TODO: v i = b2 v i + 1-b2 g 2 TODO: m hat = m i / 1-b1 t ; v hat = v i / 1-b2 t TODO: p -= lr m hat / sqrt v hat + eps ... def zero grad self : for p in self.params: if p.grad is not None: p.grad.zero Re-run the Week 2 sine-fit with ManualAdam , compare convergence speed to vanilla SGD. Adam converges in hundreds of steps where SGD needed thousands, and is far less sensitive to learning rate choice. This robustness is why you'll reach for it again in Phase 3, where the loss surface Map Clarity Loss over 2D positions is even messier. Your Phase 3 CoordinateAdam class will be a near copy of this one — same math, different thing being optimized. 💡 The Innovation early preview : "I wrote my own Adam optimizer for coordinate updates because I needed to freeze individual coordinates mid-optimization when a user grabs a node — the standard torch.optim API doesn't expose that cleanly." Keep this for Chapter 3. ✅ Checkpoint 1.2: Plot loss curves for SGD, SGD+momentum, RMSprop, Adam — same problem, same init, same step count. If Adam isn't fastest, check bias-correction first. A Transformer's core operation: for every token, compute a weighted average of every other token's representation , where weights "attention" are learned and reflect relevance. Everything else — multi-head, positional encoding, layer norm, feedforward — is scaffolding. Analogy: a room full of people tokens forming an opinion on a topic — you weight each person's input by relevance. Query = what you're looking for; Key = what each token offers; Value = what it contributes if attended to. python import torch, math import torch.nn.functional as F def scaled dot product attention Q, K, V, mask=None : d k = Q.shape -1 scores = ... TODO: Q @ K.T / sqrt d k if mask is not None: scores = scores.masked fill mask == 0, float '-inf' weights = ... TODO: softmax scores, dim=-1 output = ... TODO: weights @ V return output, weights Sanity check: verify weights.sum dim=-1 is all 1.0 python class TinyMultiHeadAttention torch.nn.Module : def init self, d model, n heads : super . init assert d model % n heads == 0 self.d k, self.n heads = d model // n heads, n heads ... TODO: four Linear layers W q, W k, W v, W o def forward self, x, mask=None : ... TODO: project, reshape to heads, attend per head, concat, W o class TinyEncoderLayer torch.nn.Module : def init self, d model, n heads, d ff : super . init self.attn = TinyMultiHeadAttention d model, n heads self.norm1 = torch.nn.LayerNorm d model self.ff = torch.nn.Sequential torch.nn.Linear d model, d ff , torch.nn.ReLU , torch.nn.Linear d ff, d model self.norm2 = torch.nn.LayerNorm d model def forward self, x, mask=None : ... TODO: x = norm1 x + attn x ; x = norm2 x + ff x Train on next-character prediction over a small corpus a few historical-text paragraphs fits the theme . Visualize attention weights for one sentence as a heatmap — rows should light up on semantically relevant tokens e.g. a pronoun attending to the noun it refers to . Attention stops being a diagram and becomes something you watch your model do. In Chapter 2 you'll swap this toy encoder for a pretrained BERT, understanding exactly what happens inside every layer because you built the smallest version yourself. 🕳️ The Rabbit Hole: Attention has no inherent sense of token order. Read the sinusoidal positional encoding and ask why sine/cosine rather than a learned embedding. ✅ Checkpoint 1.3 — End of Phase 1: From memory, sketch a Transformer encoder layer's forward pass, and explain what breaks if you remove residual connections, layer norm, or the sqrt d k scaling. By end of Week 4, from raw tensor operations, you have: 1 intuitive autograd understanding, 2 a hand-built 2-layer network with manual gradient updates, 3 three hand-built optimizers, 4 a working tiny Transformer with visualized attention. Your Week 3 optimizers become the Phase 3 coordinate optimizer almost verbatim; your Week 4 Transformer understanding is what lets you debug BERT rather than treat it as a black box. Weeks 5–8 · Prerequisite: Phase 1 complete, comfort reading model architecture diagrams Your tiny Transformer from Week 4 learned character prediction from scratch on almost no data. BERT is the same architecture, scaled up 12 layers, 12 heads, 768-dim , pretrained on billions of words. Fine-tuning means: keep the pretrained weights as a strong starting point, attach a small task-specific head on top here, a linear layer mapping each token's final hidden state to a label like B-EVENT , I-EVENT , B-DATE , O , and train the whole thing or just the head on your labeled data for a few epochs. The reason this works with so little labeled data compared to training from scratch: BERT already learned general-purpose language structure from pretraining grammar, word relationships, some world knowledge . Fine-tuning just teaches it to route that existing knowledge toward your specific labeling scheme. python from transformers import BertTokenizerFast, BertForTokenClassification import torch TODO: load "bert-base-cased" tokenizer and BertForTokenClassification with num labels = len your label list e.g. label list = "O","B-EVENT","I-EVENT","B-DATE","I-DATE","B-PERSON","I-PERSON" tokenizer = ... model = ... def tokenize and align labels text, word labels : """ text: list of words, e.g. "The", "assassination", "of", "Franz", "Ferdinand" word labels: list of label ids, one per word Returns tokenized input + labels aligned to WordPiece subtokens subtokens after the first get label -100 so they're ignored in the loss """ TODO: tokenizer text, is split into words=True, ... then use .word ids to map each subtoken back to its source word ... TODO: set up a small labeled dataset start with ~50-100 hand-labeled sentences from historical text — yes, you label it yourself first TODO: standard fine-tuning loop: forward, CrossEntropyLoss ignore index=-100 , backward, optimizer.step -- use torch.optim.AdamW this time, not your manual one, since you've already proven you understand what it's doing Run inference on a sentence the model never saw during fine-tuning and watch it correctly tag "the outbreak of war" as an EVENT span and "1914" as a DATE span, purely from ~100 examples. That's the pretraining transfer working — this would be essentially impossible to get right training from random initialization on 100 sentences. This NER output — spans tagged as EVENT, DATE, PERSON, PLACE — is exactly what Week 6's relation extraction will pair up into causal triples. 😤 The Struggle: Label alignment between words and WordPiece subtokens is the single most common source of silent bugs in NER fine-tuning — a misaligned label doesn't crash, it just quietly trains the model on garbage. Always spot-check tokenizer.convert ids to tokens against your aligned label array for a handful of examples before you trust any training run. NER tells you what the entities are. Relation extraction tells you how they relate . The simplest approach: for every pair of entities that co-occur in the same sentence or a short window of sentences , feed their contextualized representations plus the sentence itself into a classifier that predicts one of a fixed set of relation labels: CAUSES , ENABLED BY , PRECEDES , PART OF , or NONE . A more powerful approach — REBEL — does entity and relation extraction jointly, generating the full set of triples as a structured text sequence in one pass, rather than requiring you to first extract entities, then classify every pair. It's a strong upgrade path once your simpler pairwise classifier is working and you understand why the joint approach is more efficient it doesn't blow up combinatorially with the number of entities in a sentence . Start with the pairwise classifier — it's the better learning exercise, even though REBEL is more production-capable: For each entity pair e1, e2 in the same sentence: 1. Mark their spans in the input with special tokens, e.g. " E1 The assassination /E1 of Franz Ferdinand triggered E2 Austria-Hungary's ultimatum /E2 to Serbia." 2. Run through BERT 3. TODO: take the hidden states at the E1 and E2 marker positions, concatenate them, pass through a small classifier head - softmax over {CAUSES, ENABLED BY, PRECEDES, PART OF, NONE} The output for that Franz Ferdinand sentence should be CAUSES with high confidence. Then feed it a sentence with two entities that are merely mentioned near each other with no causal link "The war began in 1914. Assassinations were common in the region." and watch it correctly predict NONE . That contrast — the model discriminating relevance , not just proximity — is the whole point of relation extraction over naive "entities near each other are related" heuristics. Every entity 1, relation, entity 2 triple this produces becomes an edge in the causal graph that Phase 3's spatial engine will lay out. 🕳️ The Rabbit Hole: Read about "distant supervision" for relation extraction — a technique for auto-generating noisy training labels by aligning a knowledge base like Wikidata against text, instead of hand-labeling everything. Useful if your ~100 hand-labeled examples aren't enough once you scale up. A pipeline is only as trustworthy as its weakest stage boundary. This week isn't about new ML — it's about wiring Weeks 5 and 6 together into something that reliably takes raw pasted text and emits a clean, deduplicated, database-ready set of triples, with sane error handling for the inevitable garbage input empty text, non-English text, text with no extractable events . python class ExtractionPipeline: def init self, ner model, relation model, tokenizer : ... def extract self, raw text: str - list dict : """ Returns: {"subject": "...", "relation": "CAUSES", "object": "...", "subject date": "1914-06-28" or None, ...}, ... """ TODO Step 1: sentence-split raw text spaCy or nltk sentence tokenizer TODO Step 2: run NER on each sentence - entity spans TODO Step 3: for entity pairs within a sentence AND across adjacent sentences causal claims often span 2 sentences , run the relation classifier TODO Step 4: filter relation predictions below a confidence threshold start at 0.6, tune empirically TODO Step 5: deduplicate triples same subject/object/relation appearing from overlapping sentence windows TODO Step 6: return structured triples ... Set up PostgreSQL with pgvector: CREATE EXTENSION IF NOT EXISTS vector; CREATE TABLE events id UUID PRIMARY KEY DEFAULT gen random uuid , text TEXT NOT NULL, date text TEXT, embedding VECTOR 768 , -- BERT's hidden size created at TIMESTAMPTZ DEFAULT now ; CREATE TABLE relations id UUID PRIMARY KEY DEFAULT gen random uuid , source event id UUID REFERENCES events id , target event id UUID REFERENCES events id , relation type TEXT NOT NULL, confidence FLOAT ; Paste in three unrelated paragraphs of history say, one on WWI causes, one on the French Revolution, one on the fall of Rome and watch the pipeline correctly keep the triples from each topic separate, with no spurious cross-topic relations — because relation extraction is confidence-thresholded and entity embeddings from unrelated topics don't get spuriously matched. This pipeline is the entire backend of the /api/extract endpoint from Chapter 0's data flow diagram. Everything downstream Phase 3, Phase 4 consumes its output. ✅ Checkpoint 2.1: Run your pipeline on a genuinely messy input a Wikipedia paragraph with footnote markers, inconsistent date formats, nested clauses and verify it degrades gracefully — no crash, just fewer/lower-confidence triples — rather than throwing an unhandled exception. An extraction pipeline that "looks right" on your three favorite test paragraphs is not the same as one that's reliable. This week is about building a small held-out evaluation set hand-labeled, separate from training data and computing real metrics: precision, recall, and F1 for both entity spans and relation classification. python def evaluate ner model, eval sentences, eval labels : TODO: run inference, compute span-level not token-level precision/recall/F1 Span-level means a predicted "B-EVENT I-EVENT" span only counts as correct if it matches the gold span's exact boundaries, not just individual token labels — this is a stricter and more meaningful metric. ... def evaluate relations model, eval pairs, eval labels : TODO: per-class precision/recall/F1, plus a confusion matrix CAUSES vs ENABLED BY vs PRECEDES are semantically close and commonly confused — expect this ... Your confusion matrix will likely show CAUSES and ENABLED BY bleeding into each other — this is expected and informative, not a bug. It tells you those two categories are genuinely hard to distinguish from surface text alone a human annotator would disagree with themselves on some of these too , which should inform how confidently the frontend displays that distinction later e.g., maybe you merge them into one edge style with a subtler visual difference rather than two starkly different arrow types . Phase 2 milestone: a real, measured, imperfect-but-quantified extraction system, feeding real triples into a real database. You now have a fine-tuned BERT NER model, a relation extraction classifier, a hardened pipeline connecting them, a Postgres+pgvector store, and honest evaluation metrics on held-out data. This is a legitimate, demoable NLP system on its own — worth pausing to appreciate before diving into Phase 3, which is where the project becomes genuinely novel. Weeks 9–12 · Prerequisite: Phase 1 and 2 complete. This is the conceptual core of ChronoWeave — take this chapter slowly. Everything before this point — the Transformer, the NER model, the relation extractor — is, architecturally, "standard" deep learning applied to a specific domain. Impressive to build from scratch, but conceptually well-trodden. This chapter is where ChronoWeave does something genuinely uncommon: using gradient descent as a general-purpose layout algorithm , with coordinates as first-class trainable parameters. Take this slowly. It's the part of the project you'll actually be excited to explain in an interview. Every optimization problem in ML has the same shape: parameters, a loss function measuring how bad the current parameters are, and an optimizer that nudges parameters to reduce that loss. So far, "parameters" has meant neural network weights. Nothing in that shape requires the parameters to be weights. A parameter is just a tensor with requires grad=True that appears somewhere in a differentiable computation whose output is your loss. So: what if the parameters are the x, y positions of graph nodes on a canvas, and the loss is a hand-designed function that scores how "good" a layout is? This reframes graph layout — traditionally solved with physics simulations force-directed layout, spring-embedder algorithms — as an optimization problem solvable with the exact same machinery you built in Phase 1. Set up the skeleton, no loss function yet — just prove coordinates can be optimized at all: python import torch num nodes = 10 TODO: initialize positions as a num nodes, 2 tensor, random in some reasonable range e.g. -5 to 5 , requires grad=True positions = ... Toy goal: pull every node toward the origin 0,0 — trivial loss, just to prove the plumbing works before you write anything smarter def toy loss positions : TODO: sum of squared distances from origin return positions 2 .sum optimizer = torch.optim.Adam positions , lr=0.1 for step in range 100 : optimizer.zero grad loss = toy loss positions loss.backward optimizer.step if step % 20 == 0: print step, loss.item TODO: plot positions before and after — every node should have collapsed toward 0,0 Watching ten random points converge toward the origin using torch.optim.Adam — the exact same optimizer class you'd use to train a neural network — is the moment this project's central idea stops being abstract. You just used backpropagation to solve a geometry problem. No physics, no forces, no simulation of springs — just calculus. toy loss is a stand-in. Next week you replace it with the real Map Clarity Loss — but the training loop around it doesn't change at all. 🧭 The Mentor Says: Resist the urge to jump straight to the full Map Clarity Loss this week. Build up loss terms one at a time, verify each one does what you expect in isolation attraction only, then repulsion only, then combined , and only then combine everything. Debugging a five-term loss function that's never converged correctly at any point is miserable. Debugging a five-term loss function where you've verified each term separately is straightforward. A good layout satisfies several competing objectives simultaneously: Each of these becomes a differentiable term, and the total loss is a weighted sum: L total = w1 L attraction + w2 L repulsion + w3 L overlap + w4 L temporal The weights w1..w4 are hyperparameters you'll tune empirically — this is exactly analogous to loss weighting in multi-task neural network training, another well-known hard problem, so don't be surprised if getting a visually pleasing balance takes real iteration. python def attraction loss positions, edges : """ edges: list of i, j node index pairs that are causally linked Pulls linked nodes together — squared distance, like a spring """ i idx = torch.tensor e 0 for e in edges j idx = torch.tensor e 1 for e in edges TODO: diff = positions i idx - positions j idx TODO: return diff 2 .sum dim=1 .mean ... def repulsion loss positions, min distance=2.0 : """ Pushes ALL pairs of nodes apart if they're closer than min distance. This is O n^2 -- fine for the node counts ChronoWeave targets tens to low hundreds of events per graph , but know that this term is the one that would need approximating e.g. Barnes-Hut at scale. """ n = positions.shape 0 TODO: compute pairwise distance matrix hint: torch.cdist positions, positions does this in one call dist matrix = ... TODO: for pairs closer than min distance, penalize min distance - dist ^2 TODO: mask out the diagonal distance of a node to itself = 0, don't penalize that ... def overlap loss positions, box size=1.0 : TODO: similar to repulsion but specifically penalizes overlap of fixed-size boxes -- can start as a simplified version of repulsion loss with min distance = box size, and refine later once you see real overlaps ... def temporal loss positions, dates normalized : """ dates normalized: tensor of shape n, , each event's date mapped to 0, 1 earliest event=0, latest=1 Soft-encourages x-coordinate to correlate with date """ TODO: target x = dates normalized canvas width TODO: return positions :, 0 - target x 2 .mean ... def map clarity loss positions, edges, dates normalized, weights : w1, w2, w3, w4 = weights return w1 attraction loss positions, edges + w2 repulsion loss positions + w3 overlap loss positions + w4 temporal loss positions, dates normalized Run the full optimization on a real extracted graph from your Phase 2 pipeline with all four terms weighted, and watch causally-connected clusters visually separate from unrelated clusters, while individual nodes within a cluster stay legibly spaced apart. This is the actual "map" in ChronoWeave — and unlike a force-directed layout from a library, you can explain precisely why every node ended up where it did, because you wrote every term of the objective yourself. This is the function whose gradient — with respect to positions , not any neural network weight — drives literally everything the user sees on the canvas, and everything that happens when they drag a node in Chapter 4. 😤 The Struggle: The most common failure mode here is loss term imbalance — e.g. repulsion dominating so hard that attraction can't pull anything together, producing a uniform grid-like scatter with no visible clustering. When this happens, don't guess-and-check weights blindly. Log each individual loss term's value not just the weighted sum every N steps, and look at their relative magnitudes . If repulsion loss is naturally 50x the scale of attraction loss just from how you defined it e.g. squared distances over many more pairs , your weights need to correct for that scale difference before they can express your actual priorities between the terms. torch.cdist docs: ✅ Checkpoint 3.1: Take a graph with two clearly separate causal clusters e.g. WWI causes and, unrelated, causes of the 2008 financial crisis, extracted from two different pasted texts and verify optimization produces two visually distinct, non-overlapping clusters — not because you told it to, but because attraction pulls each cluster's internal nodes together while repulsion pushes the two clusters apart as aggregate masses. Two refinements this week. First: gradient hooks . PyTorch lets you register a function that runs every time a gradient is computed for a specific tensor, which is how you'll implement per-node learning rate decay, gradient clipping for numerically unstable nodes, or debugging instrumentation logging exactly which nodes have the largest gradients at each step — useful for diagnosing which part of the layout is "fighting" hardest . Second: formalize your Phase 1 ManualAdam into a purpose-built CoordinateAdam that supports freezing individual coordinates — critical for Chapter 4's drag-and-reoptimize feature, where the dragged node must stay exactly where the user put it while every other node reoptimizes around it. python Gradient hook example def make logging hook node names : def hook grad : grad shape: num nodes, 2 TODO: find and print the node with the largest gradient norm this step -- useful for seeing which node is "hardest to place" ... return grad must return grad unchanged or modified -- don't return None return hook positions.register hook make logging hook node names class CoordinateAdam: """ Like your Phase 1 ManualAdam, but supports freezing a subset of coordinates e.g. a node the user just dragged . """ def init self, positions: torch.Tensor, lr=0.05, betas= 0.9, 0.999 , eps=1e-8 : self.positions = positions self.lr, self.b1, self.b2, self.eps, self.t = lr, betas, eps, 0 self.m = torch.zeros like positions self.v = torch.zeros like positions TODO: a boolean mask num nodes, -- True = frozen, don't update self.frozen mask = torch.zeros positions.shape 0 , dtype=torch.bool def freeze self, node idx : self.frozen mask node idx = True def unfreeze self, node idx : self.frozen mask node idx = False def step self : self.t += 1 with torch.no grad : g = self.positions.grad TODO: zero out gradient rows for frozen nodes BEFORE the Adam update, so their moment buffers don't drift either g = g.clone g self.frozen mask = 0.0 self.m = self.b1 self.m + 1 - self.b1 g self.v = self.b2 self.v + 1 - self.b2 g 2 m hat = self.m / 1 - self.b1 self.t v hat = self.v / 1 - self.b2 self.t update = self.lr m hat / v hat.sqrt + self.eps TODO: apply update only to non-frozen rows update self.frozen mask = 0.0 self.positions -= update def zero grad self : if self.positions.grad is not None: self.positions.grad.zero Freeze one node mid-optimization simulate a user drag by just calling .freeze node idx and manually setting that row of positions to a fixed target , re-run .step for another 100 iterations, and watch every other node smoothly re-flow into a new stable configuration around the pinned node — while the pinned node itself doesn't move a single pixel, even though it still has a nonzero gradient every step. This freeze/unfreeze mechanism is exactly the backend logic Chapter 4's WebSocket drag handler calls into. You are not building new logic in Phase 4 — you're wiring this class up to a live event stream. 💡 The Innovation: This is the strongest, most concrete "why is this novel" answer for the whole project: "Standard force-directed layout libraries treat a dragged node as an external constraint bolted onto a physics simulation — usually by literally fixing its position and letting the simulation tick forward. My approach treats the drag as freezing one parameter's gradient within the exact same optimization loop used to generate the original layout, which means the re-layout objective is provably the same objective function, just re-solved under an added constraint — not a different, ad hoc mechanism." That's a real distinction, not a marketing line, and you can defend it because you wrote both halves. register hook API: Gradient-based layout fails in ways that are different from — and often more confusing than — a standard training loop failing, because there's no "accuracy" metric to sanity check against, only a loss value and a visual result. This week is a deliberate practice week: you'll intentionally break your Week 10–11 system in several ways and learn to recognize the symptoms. lr by 10x before assuming it's a logic bug torch.cdist produces a zero distance for a node compared to itself; if your repulsion loss divides by distance anywhere instead of just using squared distance, you'll get a divide-by-zero on the diagonal. Always double-check you've masked the diagonal. positions and print per-node gradient norms; the runaway node almost always has an anomalously large gradient from a bug specific to its edges, e.g. it participates in zero attraction edges so repulsion has nothing to balance against After deliberately inducing and then fixing all four failure modes above, run your full pipeline end-to-end — real text in, real triples extracted, real layout optimized — and watch it just... work, cleanly, without you needing to touch a single hyperparameter mid-run. That reliability is earned specifically by having debugged each failure mode once on purpose, rather than encountering them for the first time under deadline pressure in Phase 4. Phase 3 milestone: a working, debugged spatial intelligence engine that takes a causal graph and produces a legible 2D layout via gradient descent, with the ability to freeze nodes and re-optimize live. 🧭 The Mentor Says: Keep a running "failure log" as a markdown file next to your code: every bug you hit, what the symptom looked like, and what actually fixed it. Six months from now when this happens again in a different project, that log is worth more than any Stack Overflow search, because it's your debugging vocabulary for this specific class of problem gradient-based systems with non-standard parameters , which almost nobody else has written down anywhere. ✅ Checkpoint 3.2 — End of Phase 3: Explain to a rubber duck or a friend, or a mirror why gradient descent on coordinates is a valid approach to graph layout at all — i.e., why the loss function's gradient with respect to position tells you a meaningful direction to move a node, given that "position" isn't something with an obvious ground truth the way a classification label is. The core answer: there's no ground truth position, but there's a well-defined relative preference — closer is better for linked nodes, farther is better for unlinked ones — and gradient descent is a general algorithm for finding local optima of any differentiable preference function, not just supervised losses with ground truth targets. Weeks 13–16 · Prerequisite: Phase 3 complete Two distinct interaction patterns need two different transport mechanisms. The initial "paste text, click Generate" flow is a classic request/response — REST fits fine. But the drag-and-reoptimize flow needs the server to push a stream of intermediate frames to the client as optimization runs so the user sees nodes flow smoothly into place rather than jumping instantly , which REST can't do — that's what WebSockets are for. python from fastapi import FastAPI, WebSocket from fastapi.responses import JSONResponse app = FastAPI @app.post "/api/extract" async def extract payload: dict : text = payload "text" TODO: run ExtractionPipeline Chapter 2 - triples TODO: run spatial optimization Chapter 3 to convergence TODO: return {"nodes": ... , "edges": ... } ... @app.websocket "/ws/relayout/{graph id}" async def relayout websocket: WebSocket, graph id: str : await websocket.accept TODO: receive {"dragged node id": ..., "new x": ..., "new y": ...} drag event = await websocket.receive json TODO: load current positions for graph id, freeze the dragged node at new x, new y using CoordinateAdam.freeze from Chapter 3 TODO: run optimization step-by-step not to convergence in one go and after every K steps, send the current positions: for step in range 200 : optimizer.step ... if step % 5 == 0: TODO: await websocket.send json {"positions": ... } ... await websocket.close Open your browser's network tab, trigger a drag, and literally watch a stream of JSON frames arrive over the WebSocket connection every few milliseconds — the exact same optimization loop from Chapter 3, just now visible as it happens, one HTTP-adjacent message at a time. This is the wiring that turns Chapter 3's offline optimization script into a live, interactive feature. D3 excels at binding data to SVG elements and handling enter/update/exit transitions smoothly — exactly what you need when node positions update every few WebSocket frames. React manages component state and lifecycle; D3 manages the actual SVG manipulation and smooth transitions between positions. The common pattern: let React own the DOM structure which nodes/edges exist , let D3 own the transitions how a node's x/y animates from old position to new . // TODO: a GraphCanvas component that: // 1. Receives nodes and edges as props from the /api/extract response // 2. Renders SVG circles for nodes, lines for edges, using D3 scales // to map data coordinates to screen coordinates // 3. Uses d3.transition to animate position changes smoothly whenever // the nodes prop updates i.e., whenever a new WebSocket frame arrives // 4. Attaches d3.drag behavior to each node, which on 'end' fires a // callback e.g. onNodeDragged nodeId, newX, newY that the parent // component uses to open the /ws/relayout WebSocket and stream in // the live re-layout frames from Week 13 function GraphCanvas { nodes, edges, onNodeDragged } { // TODO: useRef for the svg element, useEffect that runs D3 rendering // logic whenever nodes or edges change } Drag a node and watch not just that node move, but every connected node smoothly animate to its new position over the following second or two — not teleporting, not choppy, an actual fluid re-settling. This is the "magic moment" the entire project overview promised in Chapter 0, now real in a browser. Nothing further — this is the product. Everything from Chapter 1 onward has been building toward this specific ten seconds of interaction. 😤 The Struggle: React re-rendering and D3 both wanting to own the DOM is a classic source of fights — D3 mutates elements directly, React expects to control them via its virtual DOM diff. The cleanest fix most people converge on: let React render the SVG container and static structure, but hand D3 a ref to manipulate node/edge elements directly inside a useEffect , and never let React's render also try to set those same attributes. Pick one owner per DOM property. Docker Compose ties your FastAPI backend, Postgres+pgvector database, and optionally a separate model-serving container into one reproducible unit you can run locally exactly as it'll run in the cloud — closing the "works on my machine" gap before it costs you a demo day. docker-compose.yml — TODO fill in the blanks services: db: image: pgvector/pgvector:pg16 environment: POSTGRES PASSWORD: ... volumes: - pgdata:/var/lib/postgresql/data backend: build: ./backend depends on: - db environment: DATABASE URL: postgresql://...@db:5432/... ports: - "8000:8000" TODO: mount your fine-tuned model weights as a volume rather than baking them into the image -- keeps image builds fast during dev frontend: build: ./frontend ports: - "3000:3000" volumes: pgdata: docker-compose up on a completely fresh machine or a cloud VM and the whole system — model inference, database, backend, frontend — comes up correctly with zero manual steps beyond that one command. That's the difference between "a project on my laptop" and "a project." Deploy to Render, Fly.io, or a similar platform for a public demo URL — essential for the hackathon-ready, portfolio-ready version of this project. The last week is deliberately not about new features. It's about the unglamorous 20% that makes a demo feel finished: loading states while extraction/optimization runs, graceful handling of bad input, a couple of pre-baked example texts a judge or interviewer can one-click try instead of needing to paste their own, and a README that explains the project in 30 seconds. /api/extract is in flight this can take several seconds — a live optimization loop is not instant, and users need to know it's working, not frozen Show the finished product to someone who has never seen it, say nothing, and watch them paste in their own text and drag a node without prompting. If they intuitively understand what happened without you explaining it, the product design succeeded. A live, deployed, full-stack application: paste historical text, get an extracted causal graph laid out via gradient descent, drag any node and watch the rest of the graph re-optimize in real time — all running on infrastructure you understand end to end, built from raw tensor operations up. Lead with the live demo, not the tech stack list. The first ten seconds should be someone pasting text and dragging a node — let the magic moment sell itself before you explain any internals. The "gradient descent as general-purpose graph layout" idea, generalized beyond history specifically, touches on active research areas: differentiable graph drawing, learned layout objectives, and combining symbolic graph constraints with continuous optimization. A write-up comparing your Map Clarity Loss approach against classical force-directed layout on layout-quality metrics edge crossing count, node overlap, cluster separation would be a legitimate small research contribution, or at minimum a strong technical blog post. Current design targets tens to low-hundreds of nodes per graph the O n² repulsion term is the binding constraint . Scaling further means either approximating repulsion Barnes-Hut / quadtree methods, the same trick classical force-directed layout libraries use at scale or moving to a hierarchical layout where clusters are optimized independently and then composed. 1. Install Python 3.11+, then: curl -sSL https://install.python-poetry.org | python3 - poetry --version verify install 2. Project scaffold mkdir chronoweave && cd chronoweave poetry init --name chronoweave-ml -n poetry add torch numpy matplotlib jupyter transformers 3. Verify PyTorch sees your GPU if you have one -- CPU is fine for Phase 1 poetry run python -c "import torch; print torch.cuda.is available " 4. Launch a notebook and run the Week 1 autograd exercise before doing anything else -- confirm your environment works before building on it poetry run jupyter notebook If you don't have a GPU: everything in Phase 1 and most of Phase 3 runs fine on CPU small networks, small graphs . Phase 2's BERT fine-tuning is the one place a GPU meaningfully helps — Google Colab's free tier is sufficient for the scale of fine-tuning this project needs. | Symptom | Likely Cause | Fix | |---|---|---| | Loss doesn't decrease at all | Forgot zero grad before .backward | Gradients are accumulating across steps — add zero grad at the top of the loop | RuntimeError: ...does not require grad and does not have a grad fn | A leaf tensor got overwritten outside torch.no grad | Check every in-place parameter update is inside with torch.no grad : | Loss becomes NaN after N steps | Divide-by-zero, usually in a pairwise-distance loss term where the diagonal self-distance = 0 isn't masked | Mask the diagonal before dividing by any distance term | | Everything collapses to a point during layout optimization | Repulsion term isn't actually contributing masking bug or weight of 0 | Log each loss term separately, verify repulsion is nonzero and comparable in scale to attraction | NER model predicts O for everything | Label misalignment between words and WordPiece subtokens | Spot-check tokenizer.convert ids to tokens against your label array on several examples | | WebSocket disconnects mid-reoptimization | Sending too many frames too fast, or an unhandled exception mid-loop killing the coroutine | Throttle to sending every 5th step, wrap the optimization loop in try/except with a graceful websocket.close on error | | D3 nodes flicker or don't animate smoothly | React re-render fighting with D3's direct DOM manipulation | Ensure only one of React/D3 sets a given DOM attribute; do D3 updates inside useEffect , not render | | Docker Compose backend can't reach the database | Using localhost instead of the service name in DATABASE URL | Compose networking resolves service names db , not localhost , between containers | Week: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Phase 1 ██ ██ ██ ██ Autograd ██ Manual NN ██ Optimizers ██ Tiny Xformer ██ Phase 2 ██ ██ ██ ██ NER fine-tune ██ Relation extract ██ Data pipeline ██ Evaluation ██ Phase 3 ██ ██ ██ ██ Coords as params ██ Map Clarity Loss ██ Grad hooks + optimizer ██ Debugging practice ██ Phase 4 ██ ██ ██ ██ FastAPI + WS ██ React + D3 ██ Deployment ██ Polish + demo ██ Phase 1 correctness of understanding, not the app : torch.optim.Adam on the same toy problem Phase 2 quantitative, on your held-out eval set : CAUSES specifically the class you care most about ; lower is acceptable for rarer/harder classes like ENABLED BY Phase 3 behavioral, verified visually + numerically : Phase 4 product-level : The 30-second version: "I built a system that reads historical text, extracts cause-and-effect relationships between events using a fine-tuned Transformer, and lays those events out on a 2D map using gradient descent instead of a physics engine — meaning the layout is generated by backpropagating a custom loss function with respect to node coordinates, the same way you'd train a neural network, except the 'parameters' are positions on a canvas instead of weights." Anticipated follow-up questions and how to answer them: "Why not just use a force-directed graph library?" — Because I needed the layout objective to express more than physical forces can naturally capture: e.g., a soft temporal-ordering constraint alongside causal clustering, with tunable relative weights between competing goals, and the ability to freeze arbitrary subsets of nodes mid-optimization while re-solving the same objective for everyone else. Gradient descent on a custom differentiable loss generalizes to all of that; a spring simulation would need a bespoke mechanism bolted on for each one. "How did you handle relation extraction being noisy?" — I measured it rather than assumed it: held-out precision/recall/F1 per relation class, a confidence threshold before a triple gets displayed, and I designed the frontend to reflect uncertainty where the model itself is uncertain e.g., visually softer distinction between relation types the model confuses most, per the confusion matrix . "What was the hardest bug?" — Talk about one specific failure mode from Chapter 3 Week 12 in detail e.g., the NaN-from-unmasked-diagonal bug — specificity here is what separates "I used ML" from "I understand ML." "Why build the optimizer from scratch instead of using torch.optim.Adam directly?" — Because I needed per-node freezing during live re-layout, which meant controlling exactly how the moment buffers and update step interacted with a frozen mask — not something the standard API exposes, and writing it myself meant I fully understood what I was modifying. "What would you do differently at scale?" — The repulsion loss term is O n² ; past a few hundred nodes I'd move to an approximate method Barnes-Hut quadtree, the same technique classical force-directed layouts use , or a hierarchical approach that optimizes clusters independently before composing them. End of guide. This document covers the full architecture, all four learning phases, and the operational appendices for building ChronoWeave. Treat each chapter's code skeletons as a starting point to fill in and argue with — the goal was never for you to run code someone else wrote, but to be the one who could have written PyTorch's autograd yourself, at least once, in miniature.