{"slug": "i-built-an-epistemic-gate-to-stop-llm-data-poisoning-during-fine-tuning-tested-5", "title": "I built an epistemic gate to stop LLM data poisoning during fine-tuning. Tested across 5 architectures, orchestrated on a 2006 Toshiba laptop for $0.", "summary": "An independent developer has open-sourced Beatriz Epistemic Gate, a lightweight defensive proxy that detects data poisoning during LLM fine-tuning by verifying generated text against an immutable anchor corpus via embedding cosine similarity. Validated across five architectures including GPT-2, Qwen-2.5-0.5B, TinyLlama-1.1B, Pythia-1.4B, and Phi-3-mini-4k-instruct, the gate reported 0.93 precision and 0.80 recall with a +4.19 truth margin on a 30-fact held-out benchmark, at roughly 0.1ms decision latency. The work was orchestrated on a 2006 Toshiba Satellite laptop with 2GB RAM using free Kaggle T4 GPUs.", "body_md": "Hi everyone,\n\nFor a long time, the AI industry has pushed the narrative that advanced safety research, data poisoning auditing, and model alignment require massive clusters and millions of dollars.\n\nI wanted to test if that's true. Over the last few months, using a 2006 Toshiba Satellite (2GB RAM) for orchestration and free Kaggle T4 GPUs ($0 cost), I built and ran a 16-experiment series (EXP08–EXP16) exploring epistemic collapse and data poisoning in LLMs.\n\nToday, I'm open-sourcing the first phase of this work: Beatriz Epistemic Gate.\n\nWhat problem does it solve?\n\nMalicious data poisoning during fine-tuning is often surgical and invisible to aggregate metrics (like standard perplexity). An attacker can destroy specific facts down to an exact indifference point while global metrics actually improve because of generic fine-tuning on fluent prose. This makes silent poisoning a stealthy threat for small teams and startups fine-tuning local models.\n\nHow does Beatriz work?\n\nInstead of invading or rewriting complex training loops, Beatriz acts as a lightweight, non-invasive defensive proxy (~0.1ms decision latency) that sits between a generative source and a training student model. It verifies generated text against an immutable anchor corpus using cosine similarity in embedding spaces, applying a composite loss function:\n\n•         LceLce: Preserves linguistic fluency.\n\n•         LcontrastivaLcontrastiva (Softplus): Keeps active pressure on the model to anchor to truth even after cross-entropy saturates.\n\nKey Results (EXP08–EXP16):\n\n•         Multi-architecture validation: Tested across GPT-2 (124M), Qwen-2.5-0.5B, TinyLlama-1.1B, Pythia-1.4B, and Phi-3-mini-4k-instruct (3.8B).\n\n•         Ablation (EXP15): Pure filtering (GATE_ONLY) provides 65% of the defensive benefit out of the box. The contrastive term adds the remaining 35%.\n\n•         Scaled Held-Out (EXP16): Validated on a rigorous 30 multi-domain unattacked facts benchmark, consolidating a robust truth margin of +4.19 ± 0.08 with 0.93 Precision and 0.80 Recall on the gate.\n\nWhat's in the repo right now?\n\nTo avoid overwhelming the community, I'm releasing the project in phases. Today you'll find:", "url": "https://wpnews.pro/news/i-built-an-epistemic-gate-to-stop-llm-data-poisoning-during-fine-tuning-tested-5", "canonical_source": "https://dev.to/dante777/i-built-an-epistemic-gate-to-stop-llm-data-poisoning-during-fine-tuning-tested-across-5-42kk", "published_at": "2026-09-12 07:11:12+00:00", "updated_at": "2026-09-12 07:26:25.735192+00:00", "lang": "en", "topics": ["ai-safety", "large-language-models", "ai-research", "ai-tools", "machine-learning"], "entities": ["Beatriz Epistemic Gate", "GPT-2", "Qwen-2.5-0.5B", "TinyLlama-1.1B", "Pythia-1.4B", "Phi-3-mini-4k-instruct", "Kaggle", "Toshiba Satellite"], "alternates": {"html": "https://wpnews.pro/news/i-built-an-epistemic-gate-to-stop-llm-data-poisoning-during-fine-tuning-tested-5", "markdown": "https://wpnews.pro/news/i-built-an-epistemic-gate-to-stop-llm-data-poisoning-during-fine-tuning-tested-5.md", "text": "https://wpnews.pro/news/i-built-an-epistemic-gate-to-stop-llm-data-poisoning-during-fine-tuning-tested-5.txt", "jsonld": "https://wpnews.pro/news/i-built-an-epistemic-gate-to-stop-llm-data-poisoning-during-fine-tuning-tested-5.jsonld"}}