I have a script called git_commit.py
in one of my repos. It shells out to claude -p
with the staged diff, gets back a Conventional Commit message, and prints it. It's wired into a prepare-commit-msg
git hook so every commit gets a pre-filled message for free. Small, dumb, useful.
The first version had one instruction in the system prompt: "Output ONLY the commit message — no explanation, no markdown, no quotes." That's it. It worked fine for a while, and then one day a commit landed with a trailing Co-Authored-By: Claude <noreply@anthropic.com>
line that I never asked for and definitely didn't want on a personal repo's history.
I did the obvious thing first: I made the prompt more specific.
SYSTEM = (
"You are a git commit message generator. "
"Output ONLY the commit message — one line, no explanation, no markdown, no quotes, "
"no co-author lines, no signatures, no AI references. "
"Follow Conventional Commits: type(scope): subject. "
"Types: feat, fix, docs, style, refactor, test, chore. "
"Subject: imperative, lowercase, max 72 chars."
)
This is the same move I see everywhere: the CLAUDE.md file in that same repo has a line that says, in bold, "NEVER add Co-Authored-By: or any Claude/AI reference to commit messages." I've seen the same pattern in a dozen other people's prompt files — a growing list of "never do X" instructions bolted onto a system prompt, each one added reactively after X happened once.
It helped. It did not solve it. A model call is a sample from a distribution, not a function with a guaranteed return type. Any single generation can still ignore an instruction — a longer diff, a different day, a subtly different phrasing of the request, and the same "never" line just doesn't fire. I don't actually know the mechanism on any given miss and I don't need to. The point is: a natural-language instruction is advisory. It shifts probability mass, it doesn't clamp it.
I ran into an article on dev.to making a point that reframed this for me: the alternative to an LLM "quality gate" isn't a better-worded LLM quality gate, it's routing the output through something deterministic afterward. Not "ask the model harder," but "stop trusting the model's output as final and put a real filter between the model and the thing it's about to touch."
For a commit message, that's about as low-stakes and easy to build as a deterministic gate gets. So instead of only tightening the prompt, I added a second, boring layer underneath it:
_STRIP = ("co-author", "co-authored", "generated by", "claude", "anthropic", "openai", "llm", "ai:")
raw = subprocess.check_output(
["claude", "-p", SYSTEM + "\n\n" + diff],
text=True,
).strip()
msg = "\n".join(
l for l in raw.splitlines()
if not any(s in l.lower() for s in _STRIP)
).strip()
print(msg)
That's the entire fix. Fourteen lines. It doesn't understand the commit message, doesn't call the model again, doesn't do anything clever. It just looks at each line the model produced and throws away any line containing one of a handful of substrings. No amount of the model deciding "actually I will add attribution this time" survives contact with this loop, because the check doesn't ask the model's permission — it runs after the model is done and out of the loop entirely.
I did the same thing in the MCP server that wraps this same generator as a tool (generate_commit_message
), because a tool call is just another code path that produces model output on its way out to something that matters:
_STRIP = ("co-author", "co-authored", "generated by", "claude", "anthropic", "openai", "llm", "ai:")
def _claude(prompt: str, system: str = None) -> str:
full = (system + "\n\n" + prompt) if system else prompt
raw = subprocess.check_output(["claude", "-p", full], text=True).strip()
return "\n".join(
l for l in raw.splitlines()
if not any(s in l.lower() for s in _STRIP)
).strip()
Same filter, same reasoning: whatever calls this function — a hook, an MCP client, a future script I haven't written yet — gets output that's already been through the gate. The guarantee lives in one place, outside the model, instead of being re-asserted in every prompt that happens to call it.
The generalizable version of this, for me, is: if there's a hard constraint ("never do X"), don't only encode it as an instruction to the thing that might do X. Encode it as a check on the output of the thing that might do X. The instruction is still worth keeping — it reduces how often the filter has to actually catch something, and for anything more nuanced than string matching it's the only lever you've got. But for the constraints you can actually verify mechanically — a banned substring, a required field, a value in range, a JSON shape — write the verifier. It's cheaper than it sounds, it doesn't drift when you change models or upgrade prompts, and unlike the model, it doesn't have an off day.
The tell that you need one: you've added the same "never do X" sentence to a prompt more than once, in more than one place, because X kept happening anyway. That repetition is the gate telling you it isn't a gate.