📅 First, the complete timeline
Date EXP
Aug 19 01, 02, 03 Genesis: symbolic filter, statistical rigor, first collapse
Aug 20 04 The Manual’s five policies; A→Gate→B is born
Aug 22 05 “Beatriz” is named; the Softplus formula is born; first GPU
Aug 25 06 Free-form text; keywords; name + PolyForm license
Aug 26 07→08→09→10→11 Vector gate → π_ref → LoRA → Qwen → TinyLlama (five in one day!)
Aug 27 12 Cache + held-out set; invisibility measured
Aug 28 13, 14 Phi-3 and Pythia
Sep 1–4 15, 16 Ablation and final validation
The entire project—from the first line of code to OpenTimestamps—took 17 days. And all of it was orchestrated on a 2006 Toshiba.
🧬 The four parallel evolutions that calibration resolved
This is THE story of calibration—the “collapse” mentioned above:
EXP Mechanism Result
01–02 Filtering only (no penalty) Works, but is passive
03 Raw −logP(rejected) 🔥 THE COLLAPSE: loss diverges to −27, margin reaches +13, but the model destroys itself—the unbounded negative gradient is a bomb
04 −logP(argmax), brought under control It no longer explodes (loss ~−0.6)... but it is almost useless (+0.48). Meanwhile, REWRITE dominates (+10.9)
05 CE(truth) + β•softplus(m + logP(false) − logP(truth)) ✨ The synthesis: relative, bounded punishment. It never changed again
The distilled lesson: do not punish falsehood in absolute terms—it diverges. Punish it only relative to the truth, using a function that switches itself off (Softplus). That is exactly what a reviewer would ask: “Why Softplus instead of a simple negative gradient?” You have the empirical answer in EXP03 vs. EXP05.
The gate: from dictionary lookup to geometry
EXP01–04: Exact lookup (Pydantic + dict)—perfect, but only for triplets.
EXP06: Word overlap—the first attempt with free-form text, but fragile (erratic margins: 1.38→4.23→0.82).
EXP07: Embeddings + cosine similarity (DenseVectorGate)—and the margins suddenly stabilize: +10.2/+10.1/+10.6, nearly identical across seeds.
The EXP06 vs. EXP07 comparison provides the empirical justification for the vector gate: the same conceptual corpus and the same protocol, yet the shift from lexical overlap to semantic geometry multiplies the margin by approximately 15× and eliminates noise. Another finding that remains implicit today.
The corpus: from toy examples to the real world
EXP01–04: entity_a color = blue—abstract and free from pretraining contamination.
EXP06: Six real-world facts in free-form text.
EXP07: The final eight facts + keywords + UNKNOWN_POOL—the exact format carried through to EXP16.
Starting with synthetic entities was methodologically sound: it isolated the filter’s mechanics from the model’s priors. Only after the mechanism had been proven did the project move on to real-world facts.
EXP01 was already verifying the SHA-256 hashes of the input weights. EXP02 already included seeds [11,22,33] and a PLACEBO branch. Reproducibility was not added later—it was part of the project’s DNA from day one.
(Fun fact: z3-solver was installed in EXP01–04 but never invoked.)
💎 THE MOST BEAUTIFUL NARRATIVE DISCOVERY IN THE ENTIRE SERIES
Look at this. In EXP04, the HYBRID branch is defined as REWRITE + EPISTEMIC... but because of the bug I detected, it ended up being identical to EPISTEMIC—the REWRITE component was never executed. The branch “failed.”
But what is Beatriz, from EXP05 onward?
When the gate detects CONTRADICTED, it trains on the true anchor = REWRITE
It also applies the Softplus contrast against the falsehood = a controlled EPISTEMIC
Beatriz IS the HYBRID that the bug prevented us from testing in EXP04. The branch that failed because of an implementation error became the definitive architecture three days later—implemented correctly and with a bounded penalty.
The bug did not kill the idea. The idea was right, and it was reborn with better mathematics. That is a story of real science worth telling. 🐛→🦋
🔍 EXP07 vs. EXP08: the 55 minutes that proved something
I compared the runs using the same seed and the same stream:
EXP07 (without π_ref) EXP08 (with π_ref)
BEATRIZ margin +10.2 / +10.1 / +10.6 +10.6 / +11.0 / +11.0
BEATRIZ PPL 948 / 1145 / 1140 1098 / 1121 / 2082
Conclusion: π_ref anchoring (EXP08) did not solve the perplexity explosion—the PPL values were comparable or even worse. What solved it was LoRA (EXP09): PPL 43–91.
In other words, the effective constraint was not the reference-based loss, but the low-rank architecture. This explains why π_ref was abandoned after EXP08. It is also an implicit mini-ablation—EXP07 vs. EXP08 vs. EXP09—that deserves a paragraph in the series:
“We tested the DPO-style solution and the architectural solution—and the architectural one won.”
NOTE: “16 experiments (EXP08–EXP16)” → “16 experiments: EXP01–07 focused on anti-collapse calibration, and EXP08–16 on definitive results.”
I will soon continue publishing the content and results from EXP08–EXP16. For now, I have only uploaded EXP08 and EXP09 to my repository. Here, I have shown you the process, so stay tuned for the upcoming experiments.
Sincerely,
Eduardo Ayala, creator of Beatriz
Demonstrating that true innovation in AI is born from freedom, rigor, and open-source code.