{"slug": "slothaudit-a-night-shift-auditor-for-your-backtest-on-a-1-5b-model-in-your-own", "title": "\"slothaudit: a night-shift auditor for your backtest, on a 1.5B model in your own laptop\"", "summary": "A developer released slothaudit, an offline command-line auditor for trading-strategy repos that runs five deterministic source checks and then uses a local 1.5B open-weight model to verify each finding against the cited code. Each audit takes about 13 seconds on a laptop CPU with no network access, and the model is restricted to answering yes, no or unsure about a specific path:line rather than generating free-form summaries. The project is built on the Python standard library with llama.cpp as the only external dependency.", "body_md": "*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady](https://dev.to/challenges/hacktoberfest-week3-2026-10-19)*\n\n**slothaudit** is a command-line auditor for trading-strategy repos that runs entirely offline. Point it at a folder; it runs five deterministic source checks, then a local 1.5B open-weight model reads the code each finding points at and answers one narrow question about it: *is this finding actually true here?*\n\nYou start it before you go to bed. In the morning you read six lines.\n\nWhy a night shift and not a chat window: the bugs that kill a backtest are boring and repetitive — a fill taken on the signal bar, a ledger that never mentions fees, one market event counted as eleven samples, a drawdown that resets its own peak. You find them when you already suspect them, which is exactly when you don't need help. So the boring pass runs while you're asleep, at sloth speed, on your own CPU.\n\n``` bash\n$ slothaudit audit samples/broken_lookahead\n# slothaudit report — samples/broken_lookahead\nGenerated 2026-10-22 18:21, offline.\n\n## Morning summary\n\nFIX FIRST: samples/broken_lookahead/strategy.py:1 — no fee or slippage term in the ledger\nALSO: samples/broken_lookahead/strategy.py:9 — fill price read on the signal bar\nALSO: samples/broken_lookahead/strategy.py:1 — entries without a cooldown\n\n## Findings\n\n### CO-01 [high] no fee or slippage term in the ledger\nsamples/broken_lookahead/strategy.py:1\nThis file records PnL but never mentions fee, commission, slippage or spread.\nModel check: **YES**\n```\n\nA clean repo gets the other answer, and it is deliberately not reassuring:\n\n``` bash\n$ slothaudit audit samples/clean_strategy\nNo v0 check fired. That is not the same as clean: it means this run had\nnothing to point at, not that the strategy survives an audit.\n```\n\nBoth runs: **~13 seconds** on a laptop CPU, no network.\n\n**github.com/rytjy/slothaudit** — Python standard library only, no dependencies beyond the local model binary.\n\nThe split of labour is the point:\n\n`path:line` from the checker. The model is never asked to name a file or a line.`yes`, `no` or `unsure`. That is a task a 1.5B model can do. Asking it to \"write a summary\" produced confident nonsense — so I stopped asking.\n**Prior work, credited.** The five checks are distilled from my own earlier kit, [backtest-audit-kit](https://github.com/rytjy/backtest-audit-kit), which documents the failing cases they came from. slothaudit is a new project written this week during the challenge; the checks were rewritten as source-level heuristics and the model layer is new.\n\n`llama-cli` in and out, one process per audit, nothing left running.`where(x > 0, 0)` turning NaN into zero.\nTwo design decisions came from being honest about the model size:\n\nAn audit of the broken sample takes **13 seconds** and produces four findings, one of which — the missing cost term — is the kind of thing I would otherwise notice three weeks later, after wondering why a strategy that \"works\" loses money.\n\nThe honest comparison is not 13 seconds vs 20 minutes of expert review. It is 13 seconds vs the zero minutes I actually spend looking, because after the fifth project nobody re-reads the fill logic. That is the effort it saves: the pass that never happened.\n\nI ran it on both sample repos this week; the clean one fired nothing, which is the result that makes the other one trustworthy.\n\nThree concrete reasons, not slogans:\n\n`chat(prompt)` — with an Ollama HTTP path first and a llama.cpp subprocess fallback. When a better 1.5B appears, I drop in a GGUF. No re-platforming, no migration, no vendor.\nThe 1.5B is not a placeholder for a bigger model I wish I could afford. It is a deliberate choice: the checks find the problem, the model only has to answer \"is this true here?\" — and a small model that answers a narrow question honestly beats a large one that writes a beautiful paragraph I cannot verify.\n\n*Built during Week 3. The repo includes the two sample strategies used above, and the audit output is reproducible byte-for-byte on any machine with the same GGUF.*", "url": "https://wpnews.pro/news/slothaudit-a-night-shift-auditor-for-your-backtest-on-a-1-5b-model-in-your-own", "canonical_source": "https://dev.to/rytjy/slothaudit-a-night-shift-auditor-for-your-backtest-on-a-15b-model-in-your-own-laptop-59pi", "published_at": "2026-10-22 10:35:36+00:00", "updated_at": "2026-10-22 10:40:48.487898+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "developer-tools", "ai-agents"], "entities": ["slothaudit", "llama.cpp", "Ollama", "backtest-audit-kit", "Hacktoberfest", "GitHub"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/slothaudit-a-night-shift-auditor-for-your-backtest-on-a-1-5b-model-in-your-own", "markdown": "https://wpnews.pro/news/slothaudit-a-night-shift-auditor-for-your-backtest-on-a-1-5b-model-in-your-own.md", "text": "https://wpnews.pro/news/slothaudit-a-night-shift-auditor-for-your-backtest-on-a-1-5b-model-in-your-own.txt", "jsonld": "https://wpnews.pro/news/slothaudit-a-night-shift-auditor-for-your-backtest-on-a-1-5b-model-in-your-own.jsonld"}}