This is a submission for the Hacktoberfest Open-Source AI Challenge: Ghost in the Machine
On 8 August I committed a file called seed.sql. On 25 October I filled it in.
For 78 days it was zero bytes. Nothing ever told me. The file existed. The
import ran. The app worked. Only the data was missing — and an empty file
doesn't raise an error. It just sits there being nothing, very quietly, for two
and a half months.
I found out while deploying, a few days ago. So I built something to go looking
for whatever else was in there.
Ghosts in the Machine reads a public repository's git history and finds
failures that never produced an error — the ones nobody noticed, because there
was nothing to notice.
Point it at a repo. It walks the commits, finds what broke silently, and an
open-weight model groups it, ranks it, and writes up what happened.
a public repo
│
▼
┌─────────────────┐
│ walk history │ every commit, every diff
└────────┬────────┘
▼
┌─────────────────┐
│ rules │ no AI. zero-byte blobs,
│ (deterministic)│ credential-shaped lines
└────────┬────────┘
▼
┌─────────────────┐
│ oddness │ no AI. size collapses, resurrected
│ (deterministic)│ paths, diffs that don't match
└────────┬────────┘ their own message
▼ findings, with commit hashes
┌─────────────────┐
│ Gemma │ did this break? which of these
│ (open weights) │ matter? how do I explain it?
└────────┬────────┘
▼
┌─────────────────────────────────────┐
│ what the model made of it │
├─────────────────────────────────────┤
│ what git actually recorded │ ← both shown, so you can
└─────────────────────────────────────┘ check one against the other
When I ran it on my own repositories:
seed.sql — committed empty, filled 78 days laterrequirements.txt — committed empty in the initial commit, filled 79 days laterreadme.md — committed empty, by a commit whose message was manifest.json, empty, which I later deleted without ever noticing it held nothing
Every one of those is mine. None of them ever failed.
The organisers suggested a haunted terminal and an offline oral history tool. I
went elsewhere. I kept coming back to what a ghost actually is: something
present, that nobody can see, that everyone has stopped looking for.
That describes a file committed empty exactly. It's in the tree. It's in every
clone. It's in the history permanently. And because it never errors, nobody is
looking.
The repository is the haunted house. The commits are the record of who lived
there. I just read the record.
The case I kept thinking about isn't mine. It's inheriting a repo.
You join a team, or pick up something abandoned, and you want to know: what's
wrong with this that nobody is going to tell me? The README is aspirational.
The code is current. The history holds the real answers, and nobody reads 500
commits to find them.
Nobody hands over a project with a list of everything quietly broken in it.
Partly because it would be awkward. Mostly because they don't know either.
That's the whole point — these failures produced no error, so the person who
made them is as unaware as the person inheriting them.
The same goes for open-sourcing something, or handing a repo to a client.
What's in the history that shouldn't be? Secrets survive deletion, and people
usually find that out afterwards.
→ ghosts-in-the-machine.onrender.com
Two exhibits load on arrival, from my own repositories. Paste in any public
repo and it will read that one instead.
It's a free instance, so the first request after a quiet spell takes a moment
to wake up. Small and medium repos come back in under a minute. Large ones take
longer, because cloning dominates.
→ github.com/VidhiDixit2000/ghosts-in-the-machine
Two layers, deliberately separated.
The deterministic layer walks the history and emits findings. No AI. It
checks whether a blob is zero bytes when the file extension implies content,
and whether added lines look like credentials. Facts, with commit hashes
attached.
The model layer answers a different question: which of these matter, which
are the same story, and how do I explain them to a person? That's Gemma, via
OpenRouter — open weights, hosted inference, so the demo runs without a GPU.
The split isn't tidiness. Each layer does what the other is bad at.
A regular expression can't tell you an empty file mattered. It has no idea
whether 78 days is a long time. Ask a model to scan a whole history instead and
it invents findings, misses real ones, and burns tokens doing it.
Here's the split earning its keep. One repo produced three separate findings: a
connection string in database.py, a DATABASE_URL in docker-compose.yml,
and a POSTGRES_PASSWORD in the same file. Three true findings. One story —
the same password, three places, one commit. The rules found them. Only the
model saw they were one ghost.
On the page, both are shown. What the model made of it sits above what git actually recorded, so you read the narration and then check it against
That mattered more than I expected.
While I was tuning the narration prompt, Gemma 3 12B wrote this about the
credentials:
These values... were removed from the working tree before the commit.
However, they remain present in the repository's history.
That's impossible. Git history contains what was committed. If something was
removed before a commit, that commit never recorded it — it wouldn't be in the
history at all. The sentence contradicts itself.
This was google/gemma-3-12b-it at temperature 0.3.
What actually happened: committed 30 July, removed 25 October. Nearly three
months apart. The findings said so correctly. The model garbled them on the way
to prose.
I only noticed because the evidence was sitting right underneath. If the
narration had replaced the findings instead of sitting beside them, I'd have
shipped a well-written falsehood about my own repository.
Moving to Gemma 3 27B and lowering the temperature fixed that particular error.
But the architecture caught it, not the model choice.
If you're building on a model: test across versions and temperatures. A wrong
answer at 0.3 looks completely reasonable until you check it against something
that can't be wrong.
I started by writing rules for failures I'd personally hit. Empty files,
because of seed.sql. Leaked credentials, because of a .dockerignore I
forgot and a password I thought I'd deleted by commenting it out.
Then I saw what I'd built: a tool that finds failures someone has already met.
For a project about failures nobody noticed, that's close to self-defeating.
A rule per failure type, forever, and only ever the ones already on the list.
The way out isn't more rules. There are infinitely many. It's changing what the
rules are for.
Git's vocabulary is tiny. A file appeared, vanished, changed, or moved. Its
bytes grew or shrank. That's nearly all of it. So however many ways software
can break, a failure has to express itself in that vocabulary to be visible at
all — and it lands in one of a few shapes:
Truncated JSON and a gutted requirements.txt are different failures with the
same shape. A case-only rename and a resurrected file are different failures
with the same shape.
So I added a second deterministic layer that names nothing. Six measurements:
size collapse, a file too small for its extension, a resurrected path, two
paths differing only in case, a large diff under a short message, a change
that's almost all deletion. Each scores how unusual a commit is. The dozen
oddest go to the model in one call, and the model says what — if anything —
went wrong.
Like a metal detector. It doesn't know what a coin is. It knows there's metal
here. Someone else digs.
The signal set is bounded by what git can record. What the model can name isn't.
Here it is working. On a test repo, the oddness layer flagged a .json file
deleted inside a 14,000-line commit. Neither rule would ever have caught that —
it isn't empty and it isn't a credential. It surfaced because it was strange.
And then the model explained it as an attempt to hide evidence.
Both halves of that sentence are the point.
To be honest about what this is: those six measurements are still rules. I
wrote them by hand and picked every number in them. What changed is the ratio.
One rule in the first layer finds one failure type, forever. One measurement in
the second finds a truncated JSON, a gutted manifest, a half-written migration,
a botched merge — failures I've never thought of and don't need to.
The rules didn't disappear. They stopped growing with the problem.
This tool reads your repository. Every commit, every diff, every line that
ever held a credential. If it's going to do that, you should be able to read it
back: what it looks for, what it sends to a model, what it redacts before
sending. The code is public for the same reason the evidence sits beside the
narration. A tool making claims about your history should be checkable.
Open weights meant I could reason about failure. When Gemma produced that
impossible sentence, I could see which model, which version, what temperature,
what prompt. I changed two of them and watched the failure go away. That's a
debugging loop. Behind an opaque API it would have been a shrug.
Two named rules, six measurements. Most well-maintained repositories come
back empty, correctly. The page says so rather than pretending otherwise.
It doesn't read a whole repository. Commit depth is capped, and the page
tells you when a history was truncated.
Large repos are slow. Cloning dominates, not scanning.
The model reads intent into odd commits. On a repo called buggy-python,
with a commit message reading "write buggy code", it described a deleted JSON
file as an attempt to hide evidence. The measurement was right that something
unusual had happened. Everything after that was the model writing a story.
The narration isn't stable. Same repo, same prompt, same temperature: one
run called a deletion "possibly intentional experimentation", the next called
it "a deliberate attempt to hide evidence". No code changed in between.
It invents chronology inside a single commit. A commit is a snapshot —
the changes in it have no order. I told the model that in plain words. It
repeated it back correctly ("both changes were recorded in the same commit")
and then wrote, one sentence later, that one happened immediately after the
other. It isn't ignoring the rule. It states the rule and breaks it in the
same paragraph.
Secret detection is shallow next to dedicated tools. The quiet-failure
half is where the gap is.
Every one of those is visible on the page, because the evidence sits under the
narration. That isn't a coincidence. It's why the page is built that way.
Some named rules will always be worth keeping. A dependency manifest losing
most of its contents is worth catching directly rather than inferring. But they
shouldn't live in one growing file of conditionals.
The shape that scales is a small registry. Every rule implements the same
interface — does this apply to this change, and if so what does it find — and
every rule returns findings in one format. Adding a rule then costs nothing
structurally. Rules that don't apply to a file never run against it. Each
commit is parsed once and shared. Duplicate findings collapse by fingerprint.
One rule failing doesn't take the scan down.
A registry to make named rules cheap, and oddness scoring to make most of them
unnecessary. The first bounds the maintenance cost. The second bounds how many
you need at all.
I'm the kind of person who commits functionality in pieces. Somewhere in all
that pushing, something ends up in a file that shouldn't be there. There are
mistakes in every repo I've shipped that I'd only see if I stepped away, came
back, looked properly, and went wait, what?
So I built something to look. It spent most of its time looking at mine.
The one that got me wasn't the empty files. It was the password. I'd found it
weeks earlier — a connection string in database.py — and I commented it out,
felt responsible, moved on. My own tool later showed me that same line, in a
commit from July, with the # still on the front. The comment character meant
nothing. Git had kept it the whole time, and so had every clone anyone had ever
made.
Then the same password twice more, in a file I hadn't thought about. Three
copies of one mistake, and I'd only ever found one.
Somewhere in there it stopped feeling like I was finding ghosts and started
feeling like I'd been making them.
The 78-day empty file is the one I tell people about, but it isn't really a
mistake. Nobody did anything wrong. A file was committed before it had content,
and there was no moment at which anything would have said so. It sat there, in
every clone, for two and a half months, being nothing.
That's what I find unsettling about git. Your repository remembers everything
you did and tells you none of it unless you go and ask. Every project I've
shipped has things in it I don't know about yet.
Which means the next person to open one of my repos wouldn't know either. That
part I hadn't thought about until I was looking at the output.
Building a ghost detector and finding five of your own is a strange way to
spend a couple of days. I commit differently now.