I built an epistemic gate to stop LLM data poisoning during fine-tuning. Tested across 5 architectures, orchestrated on a 2006 Toshiba laptop for $0.

An independent developer has open-sourced Beatriz Epistemic Gate, a lightweight defensive proxy that detects data poisoning during LLM fine-tuning by verifying generated text against an immutable anchor corpus via embedding cosine similarity. Validated across five architectures including GPT-2, Qwen-2.5-0.5B, TinyLlama-1.1B, Pythia-1.4B, and Phi-3-mini-4k-instruct, the gate reported 0.93 precision and 0.80 recall with a +4.19 truth margin on a 30-fact held-out benchmark, at roughly 0.1ms decision latency. The work was orchestrated on a 2006 Toshiba Satellite laptop with 2GB RAM using free Kaggle T4 GPUs.

Hi everyone, For a long time, the AI industry has pushed the narrative that advanced safety research, data poisoning auditing, and model alignment require massive clusters and millions of dollars. I wanted to test if that's true. Over the last few months, using a 2006 Toshiba Satellite 2GB RAM for orchestration and free Kaggle T4 GPUs $0 cost , I built and ran a 16-experiment series EXP08–EXP16 exploring epistemic collapse and data poisoning in LLMs. Today, I'm open-sourcing the first phase of this work: Beatriz Epistemic Gate. What problem does it solve? Malicious data poisoning during fine-tuning is often surgical and invisible to aggregate metrics like standard perplexity . An attacker can destroy specific facts down to an exact indifference point while global metrics actually improve because of generic fine-tuning on fluent prose. This makes silent poisoning a stealthy threat for small teams and startups fine-tuning local models. How does Beatriz work? Instead of invading or rewriting complex training loops, Beatriz acts as a lightweight, non-invasive defensive proxy ~0.1ms decision latency that sits between a generative source and a training student model. It verifies generated text against an immutable anchor corpus using cosine similarity in embedding spaces, applying a composite loss function: • LceLce: Preserves linguistic fluency. • LcontrastivaLcontrastiva Softplus : Keeps active pressure on the model to anchor to truth even after cross-entropy saturates. Key Results EXP08–EXP16 : • Multi-architecture validation: Tested across GPT-2 124M , Qwen-2.5-0.5B, TinyLlama-1.1B, Pythia-1.4B, and Phi-3-mini-4k-instruct 3.8B . • Ablation EXP15 : Pure filtering GATE ONLY provides 65% of the defensive benefit out of the box. The contrastive term adds the remaining 35%. • Scaled Held-Out EXP16 : Validated on a rigorous 30 multi-domain unattacked facts benchmark, consolidating a robust truth margin of +4.19 ± 0.08 with 0.93 Precision and 0.80 Recall on the gate. What's in the repo right now? To avoid overwhelming the community, I'm releasing the project in phases. Today you'll find: