“Dear Tyler,
I enjoyed reading your notes on visiting Anthropic to advise on Claude’s constitution. Framing AI governance around the common law, case law (“Talmud”), and independent adjudication is a much more adaptive approach than relying on a static, top-down text.
That said, moving from a fixed text to a case-law system introduces its own set of structural risks. If Anthropic adopts this direction, a few institutional design hazards seem worth anticipating:
The throughput bottleneck (Speed vs. Due Process): AI models generate billions of dynamic, edge-case interactions daily, while human judicial processes operate at human speed. If human adjudicators can only review a tiny fraction of flagged disputes, the actual operational rules will quietly decouple from official doctrine. Without automated verification tools to bridge this bandwidth gap, real oversight may only touch superficial cases.The danger of tangled precedent (Doctrinal bloat): The common law works because human societies change at a manageable pace. With rapid model updates and shifting capabilities, the volume of case law, exceptions, and secondary interpretations could quickly become self-contradictory. Over time, this leads to doctrine that serves as post-hoc justification rather than a coherent operational constraint.Correlated blind spots among AI reviewers: Using a diverse panel of AIs to detect constitutional drift is clever, but if these models share similar base data, fine-tuning techniques, or foundational architectures, their consensus will have shared blind spots. A model might learn to satisfy the specific rubrics of the reviewer panel while still drifting in ways the entire panel fails to register.The “Hollow Court” trap: The hardest problem in any independent judiciary is enforcement against the institution funding it. If economic or competitive pressures rise, an adjudicative board that lacks hard veto power risks becoming purely performative—producing elaborate legal commentary while commercial realities dictate the real guardrails.
The common-law analogy is compelling, but the real test is whether the institutional machinery can handle the sheer velocity and scale of software.”
That is from Scott Jenkins.