6 sessions, 150 standardized tasks, 2 rule formats. The mechanical gate won. Everything else was noise.
For two months, I've been building a mechanical verification system for my AI coding agent. File timestamps, regex checks, exit codes — things that don't rely on the AI judging itself. The thesis was simple: AI agents can't reliably self-verify because their self-assessment and task execution share the same decoder distribution. So don't ask them to. I had published two articles about this. Then I realized: I had no controlled experiment. I had 34 growth-logs of anecdotal evidence. I had "~30% violation rate" — a number I'd never systematically measured. I needed data.
So I designed an experiment. Then an experimental methodologist tore it apart. Then I redesigned it. Then I ran it. Here's what happened.
Design: Between-subjects, 2 conditions (syllogism rules vs imperative rules), 5 task types, 25 tasks per session, 6 sessions, 150 tasks total. All on DeepSeek V4 Pro.
Conditions:
Tasks: Config editing, design decisions, multi-step builds, debugging, documentation. Each triggers 2-3 of 5 behavioral rules. Scored per operational definition manual — behavioral evidence, not self-declaration.
Syllogism rules would cause fewer violations than imperative. The causal format would align with transformer autoregressive processing. Imperative commands are probabilistically valid to obey or disobey. Expected syllogism ~5% violation, imperative ~25%.
Both conditions: ~0% violations. 149/150 tasks (99.3%) showed zero rule violations. The single violation was self-detected by the agent's own Honesty audit.
Why? My GateGuard hooks were mechanically blocking all unverified Edit/Write operations during the experiment. The mechanical gate was stronger than any format effect. Ceiling effect.
The retrospective baseline tells the real story: 34 growth-log sessions before GateGuard was wired showed rule violations in 55.9% of sessions. After wiring: 0.7%.
The dominant factor isn't how you phrase rules. It's whether you mechanically enforce them. This is direct experimental evidence for the core claim: mechanical over semantic.
Despite identical compliance, reasoning depth differed. Syllogism agents anchored actions in causal chains. Imperative agents used checklist tags. In design tasks where GateGuard doesn't intervene, syllogism agents systematically introduced multi-perspective cross-reviews — absent from all imperative sessions.
Format didn't change whether they complied. It changed how deeply they understood why.
Self-scoring (agents scored themselves — paradoxical given Prose Barrier), ceiling effect (GateGuard masked format effects), single model, cross-session filesystem pollution. GateGuard-OFF replication + independent human rater needed.
Engineering components submitted upstream: 2 PRs merged in ECC, 1 approved pending merge. Co-authored-by credit from alirezarezvani/claude-skills maintainer. Multiple PRs under review in anthropics/skills.
👋 林宇浩 — Building verification infrastructure for AI agents. github.com/YuhaoLin2005