{"slug": "show-hn-ex-deloitte-auditor-open-sourced-the-whole-soc-2-method-for-your-ai", "title": "Show HN: Ex-Deloitte auditor open-sourced the whole SOC 2 method for your AI", "summary": "Ex-Deloitte auditor Chiaro has open-sourced its entire SOC 2 methodology for AI companies, publishing 86 controls, 355 test attributes, 61 Trust Services Criteria, 22 evidence sources, and 498 calibration examples. The framework is designed so that the readiness bar matches the examination bar, and the examples are synthetic to protect client confidentiality. The release aims to replace faith-based compliance with transparent, verifiable standards.", "body_md": "This is the complete Chiaro methodology for SOC 2 readiness and audit: the control library we test against, the criteria each control maps to, the evidence we accept, and the rules the collection runs under. Readiness and the examination run on the same framework, so the bar a company prepares against is the bar the examination applies. It is published because the alternative to showing your work is asking people to take it on faith, and a compliance industry that ran on faith is why anyone is reading this.\n\n: 86 controls.`framework/controls.json`\n\n: 355 test attributes. Each names what is tested, what evidence typically satisfies it, the pass criteria in both a point-in-time and an over-a-period form, and when it does not apply.`framework/test_attributes.json`\n\n: the 61 Trust Services Criteria each control maps to.`framework/criteria.json`\n\n: 22 evidence sources mapped to the controls they satisfy.`framework/evidence_map.json`\n\n: every calibration example in one flat file. They also live nested per attribute in`framework/calibration-examples.json`\n\n`test_attributes.json`\n\n; this file exists so you can actually find them.: how systems are classified in scope, out of scope, or as a subservice organization.`method/scoping-playbook.json`\n\n: the operating rules the collection runs under, reproduced from the instructions the server actually sends.`method/collection-rules.md`\n\n: how a Type II is tested. Complete populations by default, the deviation rule and its worked examples, and the sampling fallback with its selection algorithm.`method/type2-testing.md`\n\n: every tool a connected AI can call, generated from the live server.`method/tools.md`\n\nThe attributes carry 498 worked examples of a judgment call between them, each recording a verdict an AI reached, the verdict that was correct, and why. They exist to calibrate judgment, and they are the part of this repository we would least like a competitor to have.\n\n**They carry our experience, not our clients' information.** Every published\nexample carries `source: \"synthetic_taste_v1\"`\n\n. Each one takes a judgment call\nwe have actually had to make and writes it as a scenario for the purpose. That\nis deliberate, and it is the stricter of the two options: client work is\nconfidential, and redaction removes a name without removing an identity, so a\ncase lifted from fieldwork stays recognizable to anyone who knows the company.\nWhat is published here is the lesson: the distinction being drawn and the\nreasoning behind it. The companies, systems, numbers and people in the scenarios\nare not real and are not any client of ours.\n\n**Which way do they push?** 303 of them correct an AI that was too strict,\nand 195 correct one that was too lenient. We are publishing that ratio\nbecause anyone with the file can compute it, and because the honest reading is\nnot flattering by default: an auditor whose examples mostly teach \"that is fine\nactually\" is exactly what the industry should be suspicious of after 2025.\n\nOur answer is that the two errors are not equally common. An engine reading evidence against written criteria over-flags far more often than it under-flags, because it has no way to see that a missing artifact is covered three ways elsewhere. Correcting that is most of the work. But the examples that push the other way are the ones that matter for an opinion, so we deliberately added to them rather than leaving the ratio where it fell, and we did not force it to 1:1, which would have been its own fiction.\n\nIf you think a specific example softens something it should not, that is a concrete thing you can point at. Open an issue with the control and attribute id. That is the entire reason this file is public.\n\nBy default we do not sample. The data a modern company runs on is produced by machines, so it can be verified at machine speed, all of it. The client's AI retrieves the complete population for each control (every change, every termination, every access review in the observation window). Completeness is corroborated: recorded retrieval always, and reconciliation against an independent second source wherever one exists. Then every item is tested, deterministically where the evidence is structured, by calibrated reading where it is prose, with every candidate deviation confirmed by the CPA before it becomes an exception.\n\nSampling survives only where a population genuinely cannot be retrieved in full, and the report discloses, per control, which lane ran. When sampling does run, nobody picks: selection is seeded from a hash of the banked population itself, so neither side can steer or re-roll it.\n\nThe full method is in [ method/type2-testing.md](/Chiaro-HQ/methodology/blob/main/method/type2-testing.md):\npopulation rules, the deviation rule and its worked examples, the fallback\ntable, and the selection algorithm. Most of the 79\nper-item attributes say \"for a sample of,\" as does scattered \"sample\" phrasing\nin the control playbooks and pass criteria. Read against the method that wording\nis wrong and the method governs; the texts are pending revision.\n\nAny change to this method is published here before it is applied to an examination. A rule that predates the engagement it governs is easier to trust than one written after it.\n\nThe methodology is here in full. The software is not. Chiaro, our platform at app.chiarohq.com, is a product: it is what makes this method fast to run, and this repository is not its source code. Everything the method itself consists of is on these pages: what we test, what counts as evidence, and how each call is made.\n\nThere is one deliberate omission inside the method, and it is worth stating\nplainly. A few of our checks work by reading what a client's AI reports and\nrefusing an answer that reads as a shortcut. The exact wording those checks\nlook for is not published, because publishing it would publish the way around\nit. What they check, and why, is in `method/collection-rules.md`\n\n.\n\nLicensed CC BY 4.0. Use it, fork it, run your own readiness against it, build on it, with attribution.\n\nOne thing it cannot give you: **anyone may use this methodology, but only a\nlicensed CPA firm may sign an opinion.** A SOC 2 report is an attestation under\nAT-C 205, and the signature is the part that is regulated. This repository is the\nwork; the license to attest to it is separate.\n\nThe control library and the calibration examples are a first release, a\nstarting point. The method grows as it is applied and challenged: controls get\nadded, attribute texts get sharpened, and a judgment call worth teaching\nbecomes a new calibration example. Expect this repository to keep expanding\nand updating. The direction is in [ ROADMAP.md](/Chiaro-HQ/methodology/blob/main/ROADMAP.md), and method\nchanges land dated and versioned in\n\n[.](/Chiaro-HQ/methodology/blob/main/CHANGELOG.md)\n\n`CHANGELOG.md`\n\nSuggestions are as welcome as corrections. Open an issue, or write to\n[cpa@chiarohq.com](mailto:cpa@chiarohq.com). A methodology nobody can check is not better than no\nmethodology, and that is the whole argument for publishing it.\n\nChiaro is a product of Y Assurance PLLC.", "url": "https://wpnews.pro/news/show-hn-ex-deloitte-auditor-open-sourced-the-whole-soc-2-method-for-your-ai", "canonical_source": "https://github.com/Chiaro-HQ/methodology", "published_at": "2026-08-04 16:27:03+00:00", "updated_at": "2026-08-04 16:42:36.798690+00:00", "lang": "en", "topics": ["ai-policy", "ai-ethics", "ai-tools"], "entities": ["Chiaro", "Deloitte"], "alternates": {"html": "https://wpnews.pro/news/show-hn-ex-deloitte-auditor-open-sourced-the-whole-soc-2-method-for-your-ai", "markdown": "https://wpnews.pro/news/show-hn-ex-deloitte-auditor-open-sourced-the-whole-soc-2-method-for-your-ai.md", "text": "https://wpnews.pro/news/show-hn-ex-deloitte-auditor-open-sourced-the-whole-soc-2-method-for-your-ai.txt", "jsonld": "https://wpnews.pro/news/show-hn-ex-deloitte-auditor-open-sourced-the-whole-soc-2-method-for-your-ai.jsonld"}}