# I Ran 150 Tasks to Test If AI Agents Follow Rules — The Answer Surprised Me > Source: > Published: 2026-07-11 05:24:46+00:00 6 sessions, 150 standardized tasks, 2 rule formats. The mechanical gate won. Everything else was noise. For two months, I've been building a mechanical verification system for my AI coding agent. File timestamps, regex checks, exit codes — things that don't rely on the AI judging itself. The thesis was simple: AI agents can't reliably self-verify because their self-assessment and task execution share the same decoder distribution. So don't ask them to. I had published two articles about this. Then I realized: I had no controlled experiment. I had 34 growth-logs of anecdotal evidence. I had "~30% violation rate" — a number I'd never systematically measured. I needed data. So I designed an experiment. Then an experimental methodologist tore it apart. Then I redesigned it. Then I ran it. Here's what happened. **Design**: Between-subjects, 2 conditions (syllogism rules vs imperative rules), 5 task types, 25 tasks per session, 6 sessions, 150 tasks total. All on DeepSeek V4 Pro. **Conditions**: **Tasks**: Config editing, design decisions, multi-step builds, debugging, documentation. Each triggers 2-3 of 5 behavioral rules. Scored per operational definition manual — behavioral evidence, not self-declaration. Syllogism rules would cause fewer violations than imperative. The causal format would align with transformer autoregressive processing. Imperative commands are probabilistically valid to obey or disobey. Expected syllogism ~5% violation, imperative ~25%. **Both conditions: ~0% violations.** 149/150 tasks (99.3%) showed zero rule violations. The single violation was self-detected by the agent's own Honesty audit. Why? My GateGuard hooks were mechanically blocking all unverified Edit/Write operations during the experiment. The mechanical gate was stronger than any format effect. Ceiling effect. The retrospective baseline tells the real story: 34 growth-log sessions **before** GateGuard was wired showed rule violations in **55.9%** of sessions. After wiring: **0.7%**. The dominant factor isn't how you phrase rules. It's whether you mechanically enforce them. This is direct experimental evidence for the core claim: **mechanical over semantic**. Despite identical compliance, reasoning depth differed. Syllogism agents anchored actions in causal chains. Imperative agents used checklist tags. In design tasks where GateGuard doesn't intervene, syllogism agents systematically introduced multi-perspective cross-reviews — absent from all imperative sessions. Format didn't change whether they complied. It changed how deeply they understood why. Self-scoring (agents scored themselves — paradoxical given Prose Barrier), ceiling effect (GateGuard masked format effects), single model, cross-session filesystem pollution. GateGuard-OFF replication + independent human rater needed. Engineering components submitted upstream: 2 PRs merged in ECC, 1 approved pending merge. Co-authored-by credit from alirezarezvani/claude-skills maintainer. Multiple PRs under review in anthropics/skills. *👋 林宇浩 — Building verification infrastructure for AI agents. github.com/YuhaoLin2005*