This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine
Every data team has a graveyard. It holds the nightly job nobody remembers writing, the .sql file in legacy/, and the cron log that just stops one night in March. Someone always asks what that job did, and the person who knew left two reorgs ago.
Ghostlog holds a séance for that job. You point it at the remains, either an old cron or Airflow log or a deprecated SQL file, and it:
raw.orders -> stg.orders -> mart.daily_revenue);
It fits Día de los Muertos: the job gets remembered properly, and the next person who inherits the warehouse gets the context it took with it.
Summoning a nightly ETL job that died on 2025-03-14 when an upstream column disappeared. This is real Gemma output (gemma3:4b on a CPU-only Linux machine, about 8 seconds):
The tombstone it wrote:
A 60-second narrated walkthrough of a real run, summoning the job with Gemma and then opening its tombstone:
Live tombstone: nightmoose.github.io/ghostlog/sample/tombstone.html. All the samples are at nightmoose.github.io/ghostlog.
The raw transcripts and generated pages are committed in docs/sample/, including one run against a deprecated SQL file with a CTE and one run with --offline.
A local séance for dead data pipelines.
Point Ghostlog at the remains of a retired job (an old cron or Airflow log, or a
deprecated .sql file) and it summons the job's ghost. Ghostlog reads the
remains, rebuilds the job's table lineage with sqlglot
and asks Gemma, running locally through
Ollama, to let the job speak for itself in the first person
Then it carves an ofrenda-style tombstone page you can open in any browser.
Everything runs on your own machine. Your logs and SQL stay on your laptop, and it keeps working with the Wi-Fi off.
Built for the DEV Hacktoberfest Open-Source AI Challenge, Week 4: Ghost in the Machine.
git clone https://github.com/nightmoose/ghostlog
cd ghostlog
python -m venv .venv && . .venv/bin/activate
pip install -e .
ollama pull gemma3:4b
ghostlog summon examples/legacy/nightly_orders_etl.log
… MIT licensed. It has one runtime dependency (sqlglot, MIT), 36 offline tests, and CI on Python 3.10, 3.12, and 3.13.
pip install -e .
ollama pull gemma3:4b
ghostlog summon examples/legacy/nightly_orders_etl.log
Ghostlog is Python, split into four small modules.
logparse`` column "cust_id" does not exist becomes lineage`` INSERT, CREATE TABLE AS, MERGE, or UPDATE to sqlglot and records a source-to-target edge for each table it reads. CTE names are skipped, so WITH active AS (...) doesn't show up as a phantom table. SQL that fails to parse is reported rather than crashing the séance. The tombstone shows the longest source-to-sink chain.medium`` format forces a reply with exactly three lines of testimony and three lines of epitaph.tombstone
What I learned from the model. My first runs used gemma3:1b. It had plenty of atmosphere but a loose grip on facts: it once said the job ran "for fourteen days each night." So I split the work. Code computes the facts, and the model only supplies the voice. On top of that, a grounding check rejects any reply where the ghost doesn't name its own job or says a number that isn't in the facts. When a reply fails the check, Ghostlog says why and falls back to a deterministic template. gemma3:4b passed the check on the sample log, so it became the default. The fallback also covers the case where Ollama isn't installed at all, so the séance always finishes.
Tests. The suite runs fully offline. A fixture fails any test that tries a real network call, and the model is exercised through fakes for valid, malformed, ungrounded, and unreachable replies. CI also smoke-tests the CLI on a runner with no model installed.
Pipeline logs and SQL are some of the most sensitive text a company has. They hold table names, customer columns, and the shape of the business. With an open-weight model running locally, nothing leaves the laptop: no API key, no upload, no usage bill. It works on a plane with the Wi-Fi off.
Open weights also let me swap models with one flag (--model gemma3:1b, --model gemma3:4b, or anything else Ollama serves). That's how I found the right size for the job: I tried both locally and kept the one that stayed grounded. sqlglot, also open source, does the precise work of reading SQL across dialects, so the model never has to guess at lineage.
And because the whole stack is open, a tool for remembering abandoned software can't itself be abandoned by a vendor. If Ghostlog dies one day, anyone can dig it up and run it again.
The first time the 4b model answered "My existence ceased on 2025-03-14 due to the loss of cust_id," the job I'd made up felt like a real colleague. Every warehouse has a few jobs like that, and they deserve a marigold.
gemma3:4b, with gemma3:1b compared) runs locally through Ollama and gives each dead job its voice, constrained by a JSON schema and a grounding check.