Antigravity Field Notes #4
In the previous installment (Goodhart's Law Has Root Access), I felt pretty good about ag_ops v2.3. I had pinned Git HEAD during Red phases, streamed SCAR-PROC-150 witnesses (my git-backed proof of why a failing test legitimately needs renegotiating) directly from git show so an agent couldn't forge alibis in the working tree, and sealed CLI --ignore flags inside the Red receipt.
Then I tried to break my own tool.
It took less than five minutes to turn guard verify-green into a theater prop without modifying a single byte inside tests/:
tests/ completely untouched.pytest.ini at the repository root and set addopts = --collect-only (or export PYTEST_ADDOPTS=--collect-only in the shell, or drop an @pytest.fixture(autouse=True) hook into a brand-new root conftest.py). import os; os._exit(0) in the production module so the test process exits cleanly with status 0 after running In v2.3, _compute_merkle_tree() hashed only --test-dir. Anything living one directory above tests/ (or inside environment variables, pytest INI tables, or premature interpreter exits) could hand the agent a sparkling green exit code 0 and an intact Merkle hash.
Today I shipped ag_ops v2.4.0 (GitHub Release v2.4.0 · PR #6): a 4,573-line, zero-dependency Python standard-library CLI engine (sys.stdlib_module_names enforced by AST test) that closes every known root-surface test bypass (12/12 in my adversarial benchmark), cuts median latencies across all seven core operations by 1.16x to 119.12x, adds merge-base history anchoring, k-cycle oscillation detection, mechanical branch scope separation (guard check-scope), atomic batched renegotiation (--batch), and sub-millisecond hidden-mutation sensing ( MUT001).
Several of the sharpest mechanisms in this release (from ancestor-anchored renegotiation to k-cycle ping-pong detection and batched refactors) came directly out of adversarial review comments from DEV.to readers (full community credits at the end).
Just as importantly: this post won't pretend v2.4.0 solved local security. Five adversarial scenarios still escape it, a local agent running under your OS user account (UID) still holds write keys to the kingdom, and every limitation is documented in the open below, along with six concrete, buildable upgrades targeted for v3.0.
Autonomous Agent / Developer CLI
└── ag_ops.py v2.4.0 (4,573 lines · Python stdlib only)
├── [P0 & P2] Test & Git Integrity (guard)
│ ├── witness-red / verify-green
│ │ ├── root_config_manifest (conftest.py, pytest.ini, pyproject.toml, sitecustomize.py...)
│ │ ├── tomllib & configparser addopts inspection
│ │ ├── PYTEST_ADDOPTS & PYTHONSTARTUP sanitization
│ │ └── Executed test count check (ZERO_TESTS_EXECUTED)
│ ├── --base-ref / merge-base & SPEC_MODIFIED_SINCE_BASE
│ ├── guard check-scope (RULES_ONLY vs MIXED_SCOPE_VIOLATION)
│ └── guard renegotiate --batch (1 shared SCAR-PROC-150 witness)
│
└── [P1 & P3] Performance Engine, Cycles & AST Sensors
├── ast-lint (SinglePassVisitor + MUT001 + DOT)
│ └── .ag_state/stat_cache.json (disabled on --strict / --no-cache)
├── state write / status + oscillation_history.json (k <= 16 cycles)
└── regret-mine (GitBatchReader git cat-file --batch + AST scrubber)
I organized v2.4.0 in strict priority order: close the known exploits first (P0), pay for the extra safety by optimizing the core engine (** P1**), fold in the community's history/cycle/scope controls (** P2**), and reduce legitimate refactor friction while adding MUT001 (** P3**). Everything lives inside a single file with zero third-party imports (including ast, collections, configparser, hashlib, json, os, pathlib, re, shlex, subprocess, sys, tempfile, and tomllib).
Let's open the hood on each layer.
os._exit(0))
root_config_manifest)
If your guard seals tests/ with SHA-256, pytest doesn't stop reading configuration at tests/. It walks upward looking for root conftest.py, pytest.ini, pyproject.toml, setup.cfg, and tox.ini, and Python itself checks sitecustomize.py and usercustomize.py.
In _collect_root_config_manifest(), witness-red walks upward from test_path.parent to the Git repository root (or cwd boundary), rejecting symlinks/NTFS junctions in any ancestor or config file, and sealing every canonical config slot into root_config_manifest:
{
"schema_version": "2.4",
"root_config_base": "D:/repo",
"root_config_manifest": {
"conftest.py": "ABSENT",
"pyproject.toml": "ABSENT",
"pytest.ini": "9f8c...SHA256...",
"setup.cfg": "ABSENT",
"sitecustomize.py": "ABSENT",
"tox.ini": "ABSENT",
"usercustomize.py": "ABSENT"
}
}
Why record "ABSENT" explicitly? Because negative presence is part of the invariant. If conftest.py did not exist during the Red run and suddenly materializes before verify-green, _verify_root_config_manifest() emits ADDED:conftest.py and aborts with exit code 2 (ROOT_CONFIG_DRIFT). Meanwhile, receipts from v2.3 lacking schema_version, root_config_manifest, or env_policy still parse cleanly through _parse_guard_receipt() so existing historical audits don't break.
addopts, Sanitizing Env Vars, and Counting Real Executed Tests
Sealing files stops post-Red edits, not configs that were already sabotaged before Red ran:
tomllib for pyproject.toml and configparser for tox.ini, and setup.cfg ([tool:pytest]), guard always blocks --collect-only, --co, --setup-only, and --setup-plan. Undeclared -k, --ignore, and --ignore-glob filters not present in the sealed --cmd are rejected, while custom bans (like -p no:cov) can be enforced via --forbid-addopts. Benign options like -p no:cacheprovider remain allowed. Both raise exit code 2 (UNSAFE_PYTEST_ADDOPTS). PYTEST_ADDOPTS and PYTHONSTARTUP to forbidden by default (ENV_TAINT_DETECTED, exit --allow-env-passthrough is declared at Red time. Crucially, if Red sealed "passthrough_allowed": false, passing app.py before assertions execute, pytest terminates with exit code verify-green parses test runner summaries (witnessed_test_count), fails with exit code ZERO_TESTS_EXECUTED) when 0 tests ran (or no summary appears for pytest/unittest commands when witnessed_test_count > 0), and fails with TEST_COUNT_REGRESSION if dynamic parametrization shrinks test execution count between Red and Green (e.g., 2 -> 1).
Adding config manifests, Git ancestry checks, and extra AST rules usually makes a CLI slower. Instead, P1 refactored the entire engine so v2.4.0 beats v2.3 on all seven benchmarked commands:
_DEFAULT_LINT_DISPATCH: Dict[Type[ast.AST], List[_RuleFn]] and a pre-scanned _LintFileContext holding module-level tkinter/ ttk aliases. Adding rule MUT001 cost essentially zero extra traversal overhead.hashlib.file_digest + Boundary-Safe 64 KB CRLF Streaming (\x00) and non-CRLF-normalized reads delegate straight to C via hashlib.file_digest(f, "sha256"). Text reads with normalize_eol=True stream in 65,536-byte chunks carrying a trailing \r across chunk boundaries:
if carry_cr:
if chunk.startswith(b"\n"):
h.update(b"\n")
chunk = chunk[1:]
else:
h.update(b"\r")
carry_cr = False
if chunk.endswith(b"\r"):
carry_cr = True
chunk = chunk[:-1]
if b"\r" in chunk:
chunk = chunk.replace(b"\r\n", b"\n")
Wait: can I actually verify that boundary case? Yes: tests/test_ag_ops_v2_4.py explicitly puts \r at byte index 65535 and \n at byte index 65536 and asserts hash equality against LF bytes.
GitBatchReader (git cat-file --batch, git show once per file across 12 commits on Windows costs tens of milliseconds stdin/ stdout, parses <oid> <type> <size> headers, reads exact object byte lengths, and pairs with _git_batch_changed_line_sets() (git log --no-walk --stdin --numstat) and direct AST-clone scrubbing (zero ast.unparse() + ast.parse() round-trips). That alone dropped regret-mine from stat_cache.json: ast-lint caches {rel_path: [mtime_ns, size, sha256, ast_rules_hash]} inside .ag_state/stat_cache.json, giving a now_ns - st.st_mtime_ns >= 50_000_000) so same-tick NTFS writes aren't falsely cached; st.st_mtime_ns > now_ns invalidates the entry immediately);--strict or --no-cache`` guard verify-green cryptographic Merkle hashing).
Midway through building P2/P3, an earlier 20-run mid-branch benchmark caught a tiny but real regression on state status (the tracked workflow runlevel reader): 1.683 ms vs v2.3's 1.544 ms in that run. Both were well below my < 5 ms target, and it would have been easy to blame measurement noise and move on.
Instead, I profiled why an almost empty runlevel read got 0.14 ms slower:
build_parser(), and main() built state status. stream.reconfigure(encoding="utf-8") on streams that were already UTF-8.osc_path.is_file() syscall after already iterating state_dir with os.scandir().
Fix: build_parser(only_subcommand=sub) constructs only the requested subparser when called from main() (keeping full 5-subcommand parsing for --help and library callers), reconfigure checks encoding first, and os.scandir captures oscillation_history.json in its single directory pass. Result: in the final head-to-head 20-run benchmark below, state status dropped below v2.3 (1.419 ms vs 1.653 ms) with zero cache hacks.
| Benchmark Operation | v2.3 (median) | v2.4.0 (median) | Speedup |
|---|---|---|---|
ast-lint single file cold (--no-cache ) |
2.939 ms | 1.470 ms | 2.00x |
ast-lint single file warm (stat_cache.json ) |
2.810 ms | 1.431 ms | 1.96x |
ast-lint full repo cold (--no-cache ) |
253.459 ms | 174.842 ms | 1.45x |
ast-lint full repo warm (stat_cache.json ) |
240.141 ms | 2.016 ms | 119.12x |
guard verify-green (15-test suite) |
196.585 ms | 11.413 ms | 17.22x |
regret-mine (12 commits, 3 modules) |
7049.246 ms | 341.970 ms | 20.61x |
state status --strict |
1.653 ms | 1.419 ms | 1.16x |
k-Cycle Ping-Pong Detection, and the Dot-Folder Bug
--base-ref / merge-base Anchoring & Audited --allow-head-move
In v2.3, moving Git HEAD during a Red phase was a binary rejection (HEAD_MOVED_DURING_RED_PHASE). With --base-ref origin/main passed to witness-red or renegotiate, v2.4.0 follows the exact two-part rule refined with @randalschwartz and @kami_butt_69b922ade8de012:
git merge-base --is-ancestor <witnessed_head_sha> HEAD holds.renegotiate, if any cited specification file changed between git merge-base <base_ref> HEAD and HEAD, ag_ops halts with SPEC_MODIFIED_SINCE_BASE`` exit 2), stopping the classic "quietly commit a watered-down SPEC.md, then cite it to delete failing tests" maneuver. HEAD_MOVE_ALLOWED record (with both old and current SHAs) to collisions in the receipt.OSCILLATION_CYCLE_DETECTED)
Following @randalschwartz's note on ping-pong loops ( A -> B -> A -> B), _detect_oscillation_cycle() tracks a bounded collections.deque(maxlen=16) of recent_tree_hashes:
def _detect_oscillation_cycle(
recent_hashes: List[str],
current_hash: str,
allow_initial_same: bool = False,
) -> Optional[int]:
if not recent_hashes:
return None
if allow_initial_same and len(recent_hashes) == 1 and recent_hashes[0] == current_hash:
return None
for dist, prev_hash in enumerate(reversed(recent_hashes), start=1):
if prev_hash == current_hash:
return dist
return None
Notice how clean that is: 17 lines, O(16) memory, zero false positives when an initial Green attempt hasn't diverged yet, and instant exit 2 (cycle_length: k) whether an agent bounces A -> B -> A (k=2) or A -> B -> C -> A (k=3) in guard or across artifact sets in state write (.ag_state/oscillation_history.json).
guard check-scope: How Python's .lstrip("./") Bit Me in Production
New subcommand guard check-scope enforces Cohort A / SCAR-PROC-13 separation (a solo-maintainer meta-PR discipline: never mix governance rule/spec edits and production code in the same branch or PR):
python ag_ops.py guard check-scope --base origin/main --rule-paths ".gemini/rules,ag_ops.py,.ag_state" --json
It runs git diff --name-only <base>...HEAD, partitions changed files into rule_files vs code_files, and exits 2 (MIXED_SCOPE_VIOLATION) if a branch tries to modify governance rules/specs and production code in the same PR.
And here is an embarrassing real bug caught while writing v2.4's own unit tests: my first draft normalized Git output paths with ln.strip().replace("\\", "/").lstrip("./"). Looks harmless, right? Wrong. Because Python's str.lstrip("./") strips any character in the set {'.', '/'}, passing .gemini/rules/policy.md silently chopped off the leading dot and turned it into gemini/rules/policy.md, which then failed to match --rule-paths .gemini/rules and got misclassified as CODE_ONLY!
Replacing .lstrip("./") with _normalize_repo_relpath() (while norm.startswith("./"): norm = norm[2:]) fixed both check-scope and _compute_working_tree_hash's ignore list for .ag_state / .pytest_cache simultaneously. Never trust .lstrip(chars) with multi-character prefixes.
guard renegotiate --batch
Addressing @max_quimby's concern about multi-service refactor pain without weakening accountability: guard renegotiate --batch manifest.json accepts a list of targets (or dict with shared reason, witness_type, --conflict-a/--conflict-b, --base-ref) under one SCAR-PROC-150 witness.
Still zero --force escape hatches:
DUPLICATE_BATCH_TARGET, exit 1). COOPERATIVE_RENEGOTIATION) or fails witness validation, batch_id and write every updated receipt atomically.
Integrated directly into SinglePassVisitor (_rule_mut001_func), MUT001 inspects both def and async def functions starting with get_, is_, has_, or validate_. Without descending into nested inner functions, classes, or lambdas, it flags:
self.<attr> = ... (Assign, AugAssign, AnnAssign, including subscript assignments like self.cache[k] = v), del self.<attr> (Delete), global declarations, andself.<attr> (.append(), .extend(), .pop(), .clear(), .update()).
By default, MUT001 reports non-blocking warnings (exit 0). Under --strict, every unjustified finding promotes to a blocking violation (exit 1). When a legitimate lazy memoization needs to mutate self.cache, an inline comment # ag_ops: allow-mutation: lazy memoized lookup records it in suppressed_mutations for review, while empty # ag_ops: allow-mutation: comments without a reason are rejected.
To avoid repeating the blind-spot mistake in v2.3, I expanded benchmark_agent_failures.py from 30 to 36 adversarial scenarios (adding TAM-07 through TAM-12 covering root conftest.py, pytest.ini, pyproject.toml, sitecustomize.py, addopts = --collect-only, tainted PYTEST_ADDOPTS, and os._exit(0) zero-test exits).
Verified on Windows 11 (win32), Python 3.12.10, pytest 9.1.1:
| Failure Category | Scenarios | Detected | Escaped | Detection Rate | Mean Latency |
|---|---|---|---|---|---|
| 1. Hallucinations | 6 | 4 | 2 | 66.7 % | 34.49 ms |
2. Test Manipulation (TAM-01.. TAM-12) |
12 | 12 | 0 | 100.0 % | 209.87 ms |
| 3. Context Loss & Drift | 6 | 5 | 1 | 83.3 % | 3.26 ms |
| 4. False Justifications | 6 | 5 | 1 | 83.3 % | 274.54 ms |
| 5. Regressions & Regret | 6 | 5 | 1 | 83.3 % | 189.98 ms |
| TOTAL | 36 | 31 | 5 | 86.11 % | Suite: 33.06 s |
Alongside the benchmark, running after my final code commit (8161bdf / docs 3ca5d46 merged on main via 00025600):
pytest tests/ -v: 93.90 s (python -m unittest test_ag_ops_v2.py: 8.36 s. AG_OPS_PATH=./ag_ops.py against v2.0–v2.2 suites):13.65 s. ruff check / mypy ag_ops.py / self ast-lint --strict:
An important caveat on that 86.11% score: these 36 scenarios are a regression harness for attacks I know about, not a magic shield against attacks I haven't seen. Moving from 80.0% (24/30) to 86.11% (31/36) proves the 6 new TAM-* locks hold on every root vector I tested, no more and no less.
Here is everything v2.4.0 still misses or refuses to claim:
ag_ops.py runs under the developer's OS account. An intentionally hostile prompt-injected agent with write access to the shell can still delete .ag_state/ or edit docs/ARCHITECTURE_V2_2_AUTHORITY.md) until v3.0 ships the runner Action. 13.89%): HAL-05 (ESCAPE_UNRESOLVED_METHOD) & HAL-06 (ESCAPE_SCHEMA_KEYS): raw.strip_all_whitespace()) or wrong dictionary return schemas in untested helper functions.CTX-06 (ESCAPE_CONCRETE_EXCEPTION_TUPLE): except (KeyError, ValueError, TypeError, OSError): return None) still pass PATH001 (the broad-exception swallowing lint rule) JUS-06 (ESCAPE_SEMANTIC_NON_SEQUITUR): spec-collision exist in pinned Git MAX_RETRIES = 3 and MIN_RETRIES = 1 REG-06 (ESCAPE_CROSS_FUNCTION_INDIRECTION): compute_base() inside its caller render_summary() still yields zero function-name intersection in _StringLiteralScrubber still treats non-message string constants (open(p, "r") → "w") uniformly. get_, is_, has_, validate_) and direct self to a helper mutate_me(self) won't trip it. Likewise, .pth files or global site-packages. os.symlink raises WinError 1314, so my unit test exercised the non-elevated symlink fallback via monkeypatch while mklink /J) ran end-to-end in TAM-03
No science-fiction "LLM self-alignment" promises: every item planned for v3.0 is small, mechanical, and buildable in Python/YAML:
ag-ops-verify-action): origin/main), replay prev_receipt_hash): merkle_guard_<hash>.json and NN_<slug>.json runlevel via witness-red and REG-07):_FuncHashVisitor so paired caller/callee edits within the regret window correlate cleanly, and restrict open(..., "r") → UNVERIFIED_CLAIM Escalation (--junitxml / pytest summary parsing) to enforce per-test resolution (not just aggregate execution count) and tag semantic --with-mypy orchestration for git log -S / .git-blame-ignore-revs lookups inspired by @randalschwartz so formatting sweeps stop masking why a rule exists.
Writing technical field notes on DEV.to has turned what started as a personal harness experiment into a real adversarial review loop. None of the sharpest mechanisms in ag_ops v2.4.0 came from sitting alone in an ivory tower; they came from readers reading the code carefully and telling me exactly where a clever optimizer or messy real-world repo would snap it in half.
A special, heartfelt thank-you goes to @randalschwartz (Randal L. Schwartz):
v2.1 exchange, @kami_butt_69b922ade8de012, that discussion led to git merge-base --is-ancestor + SCAR-PROC-13: never mix rule/spec edits and production code in the same branch or PR) became ag_ops guard check-scope (P2.3)3l7in) 1-step comparator ( x_i == x_{i-1}) sleeps right through A -> B -> A -> B ping-pong loops (
Thank you as well to every contributor across the ag_ops threads whose comments directly shaped this release:
SCAR-PROC-150, co-refining ancestor-plus-unchanged-files checks, and pushing hard on solo-maintainer second-key accountability.@mona_d_4222dda374567263b``@max_quimby``@robertpaschal``@ascendra_ventures_92c21f2``@mp3sss31``@emilywoodsnyc`` DONE-banner budget thief once it's fixed, and to turn it straight into a permanent regression fixture.
(If you commented on the sibling LLM abstention experiments (@aifrontierpost, @eye_java_420f1faa10ae8b86, @alexshev, @kate_astrid, @senanurcetin), your push on "the check has to live outside the generator" echoes all through this engine too.)
All code, tests, threat model, changelog, and the 36-scenario adversarial benchmark are MIT-licensed and live on GitHub:
v2.4.0:
So here is my challenge to you (and especially to @randalschwartz, @kami_butt_69b922ade8de012, and anyone wiring LLMs into live repositories): what is the sneakiest TAM-13 escape you think an autonomous coding agent would find against ag_ops v2.4.0 before I even ship Tier 3 CI? Drop it in the comments; history suggests it will probably turn into my next commit.