{"slug": "epistemic-engine-verify-ai-generated-code-and-forecast-what-will-break", "title": "Epistemic Engine – verify AI-generated code and forecast what will break", "summary": "Epistemic Engine, a new open-source tool that reads codebases as systems of justified beliefs, has been released to verify AI-generated code and forecast which parts of a codebase will break. The tool, which runs offline with no LLM or network dependency, includes a pre-commit/CI verifier that blocks unsafe code and a prediction tool that forecasts belief collapse with causal chains, probability, and ROI. Field-tested on 1,500+ production files, it achieved precision/recall/false-positive rates of 1.0/1.0/0.0 with a 95% CI lower bound of 0.92 on a 114-sample corpus.", "body_md": "**Computational epistemology for software.** It reads a codebase as a system of\n*justified beliefs* — claims the code makes about itself (\"this function\nauthenticates the caller\", \"this value is never null\", \"this uses approved\ncrypto\") — reconstructs how well-justified each one is from git history, and\ntracks how that confidence changes over time.\n\nOn top of that foundation it ships two tools you can use today:\n\n— a pre-commit / CI verifier that`ee guard`\n\n**blocks unsafe or unjustified code before it lands**(great for reviewing AI-generated changes).— forecasts`ee predict-chain`\n\n**which beliefs are about to collapse**, why, roughly when, and what it would cost to fix now vs. later — and it** tells you when it can't trust its own forecast.**\n\nEverything is **deterministic** (no LLM, no network, no wall-clock in any\ncomputed value), runs **offline**, and is backed by **167 tests**.\n\n```\npip install epistemic-engine\n```\n\nRequires Python 3.10+. Apache-2.0 licensed.\n\n```\nee ingest  ./my-repo        # read git history into a local graph\nee analyze ./my-repo        # extract beliefs, justifications, trajectories\n\nee guard   ./my-repo --all          # find unsafe / unjustified code\nee predict-chain ./my-repo          # forecast which beliefs will collapse\nee dashboard ./my-repo              # explore all of it in your browser\n```\n\n`ee guard`\n\nextracts the beliefs a change makes about itself and **blocks** on\nrisky ones, then layers on direct OWASP-class scanners and known-CVE matches.\nEach finding carries a severity, the reason, a concrete fix, and a stable\nfingerprint.\n\nIt catches, among others:\n\n| Class | Examples |\n|---|---|\n| Weak crypto | `md5` /`sha1` /`mt_rand` used as a security primitive |\n| Injection | SQL built by string concatenation, `os.system` /`popen` on user input |\n| Unsafe sinks | `eval` , `innerHTML` , `unserialize` , `pickle.loads` on untrusted data |\n| Secrets | hardcoded API keys / tokens (the value is never printed or stored) |\n| Misconfig | TLS verification off, `Access-Control-Allow-Origin: *` , debug enabled |\n\n```\nee guard ./repo --staged          # verify staged changes (exit 1 = would block the commit)\nee guard ./repo --all             # verify the whole tree\nee guard ./repo --format sarif    # SARIF 2.1.0 for GitHub code scanning\nee guard ./repo --emit-workflow   # print a ready .github/workflows/ee-guard.yml\nee hook install ./repo            # run it automatically on every `git commit`\n```\n\n**Adoption features** so it fits a real team: inline `# ee-ignore[rule]`\n\nsuppression, a `disabled_rules`\n\nconfig, and a **baseline** mode\n(`--update-baseline`\n\n/ `--baseline FILE`\n\n) that fails only on *new* findings so\nyou can adopt on an existing repo without fixing everything first.\n\nField-tested on 1,500+ real production files: precision / recall / false-positive\nrate of **1.0 / 1.0 / 0.0** (95% CI lower bound 0.92 on a 114-sample corpus).\n\nThe engine already knows how each belief's confidence is eroding. `predict-chain`\n\nprojects that forward and, for each at-risk belief, gives you:\n\n- a\n**causal chain**— the ranked factors driving the predicted collapse, each tagged** measured**(from your repo's history) or** assumption**(your cost model); - a\n**collapse probability** within a horizon, from a seeded Monte Carlo; - a\n**calendar ETA** from your repo's own commit cadence; - a\n**fix + ROI**, computed under*your*cost assumptions and labelled as such.\n\n```\nee predict-chain ./repo --horizon 30    # forecasts with causes, ETA, ROI\nee calibrate     ./repo                 # is the forecast trustworthy on THIS repo?\n```\n\n**It refuses to lie.** `ee calibrate`\n\nruns a leakage-free back-test against your\nrepo's own history and reports a Brier **skill score** vs. a base-rate baseline.\nIf the model doesn't beat guessing on your repo, the report says so — and\n`predict-chain`\n\nprints that verdict above every forecast. It never fabricates\nprobabilities for undisclosed future CVEs, and never presents a dollar figure as\nfact.\n\n```\nee dashboard ./repo      # loopback-only web UI; Ctrl-C to stop\n```\n\nA local, offline, self-contained dashboard with three tabs:\n\n**Beliefs**— every claim the code makes, by domain, coloured by confidence.**🔮 Predictions**— the collapse forecasts, with the calibration verdict on top.**🛡 Guard**— the`ee guard`\n\nfindings, grouped by severity, with a BLOCK/PASS banner. Scans the whole repo by default and caches the result.\n\n**Deterministic.** Identical inputs produce byte-identical output. No LLM, no sampling, no wall-clock in any computed value — so results are reproducible and diffable in CI.**Honest about uncertainty.** Gates are evaluated on the*lower bound*of a Wilson confidence interval, not a point estimate. Predictions ship with a calibration verdict. Cost/ROI figures are labelled assumptions.**Offline-first.** No network access except an explicit`ee sync`\n\n.**Bounded formalism.** Beliefs outside the closed PO-1 predicate ontology are recorded as`unformalised`\n\nand excluded from reasoning — never silently coerced.\n\n```\npip install epistemic-engine                 # core (zero heavy dependencies)\npip install \"epistemic-engine[parsing]\"      # + tree-sitter for entity-level beliefs\n```\n\nWithout the `parsing`\n\nextra the engine degrades gracefully to file-granularity\nanalysis. Python 3.10+.\n\n| Command | What it does |\n|---|---|\n`ee ingest <repo>` |\nRead git history into the local epistemic graph |\n`ee analyze <repo>` |\nExtract beliefs, justifications, change events, trajectories |\n`ee guard <repo>` |\nVerify changed/all code; block unsafe or unjustified beliefs |\n`ee hook install <repo>` |\nInstall a git pre-commit hook running `ee guard --staged` |\n`ee predict-chain <repo>` |\nForecast belief collapse with causes, ETA, and ROI |\n`ee calibrate <repo>` |\nBack-test the forecast model against the repo's own history |\n`ee dashboard <repo>` |\nServe the local web dashboard |\n`ee report <repo>` |\nEpistemic health report (text / json / markdown / html) |\n`ee beliefs / falsifiers / debt / timeline` |\nInspect specific slices |\n`ee doctor` |\nValidate environment, ontology, and configuration |\n\nRun `ee <command> --help`\n\nfor options, or `ee man`\n\nfor a full man page.\n\n```\ngit clone <your-repo> && cd epistemic-engine\npip install -e \".[parsing,dev]\"\npytest                       # 167 tests\n```\n\nBoth are portability choices for a single-developer, offline-first, cross-platform (incl. Windows) build; neither changes observable semantics.\n\n**Graph store** is abstracted behind`storage.GraphStore`\n\nwith a**SQLite** default backend (zero-install, deterministic); RocksDB is optional.**Git access** goes through the**git CLI** via subprocess, abstracted behind`ingestion.GitExtractor`\n\n.", "url": "https://wpnews.pro/news/epistemic-engine-verify-ai-generated-code-and-forecast-what-will-break", "canonical_source": "https://github.com/aswinsasi/epistemic_engine", "published_at": "2026-07-25 11:36:02+00:00", "updated_at": "2026-07-25 11:52:18.550728+00:00", "lang": "en", "topics": ["developer-tools", "ai-safety", "ai-agents"], "entities": ["Epistemic Engine", "OWASP", "GitHub", "Apache-2.0", "Python"], "alternates": {"html": "https://wpnews.pro/news/epistemic-engine-verify-ai-generated-code-and-forecast-what-will-break", "markdown": "https://wpnews.pro/news/epistemic-engine-verify-ai-generated-code-and-forecast-what-will-break.md", "text": "https://wpnews.pro/news/epistemic-engine-verify-ai-generated-code-and-forecast-what-will-break.txt", "jsonld": "https://wpnews.pro/news/epistemic-engine-verify-ai-generated-code-and-forecast-what-will-break.jsonld"}}