{"slug": "the-dailyprog-puzzle-safety-net", "title": "The dailyprog puzzle safety net", "summary": "The dailyprog puzzle site now runs a nightly CI job that auto-generates a puzzle if tomorrow's is missing, using DeepSeek V4 Flash and validation gates, ensuring the site never goes dark. The workflow checks for an existing puzzle, generates one if absent, runs test suites, commits, and deploys, all with a two-hour lead time for human review.", "body_md": "[dailyprog](https://dailyprog.club) needs a new puzzle every day. I wrote the\nfirst few drafts by hand, then built\n[a generator](/posts/generating-coding-puzzles-with-llms/) that writes them with\nan LLM and validates the output against real sandbox execution. The generator\nworks, but it still needs someone to run it. Pick a pattern, kick off the\nscript, review the result, commit. Fifteen minutes on a good day. More if the\nmodel gets stuck in a repair loop and I need to nudge it.\n\nSome days I don’t have fifteen minutes. Some days I forget. Before last week, either of those meant the site went dark at midnight UTC and stayed dark until I woke up and fixed it. Now it doesn’t.\n\nAt 22:00 UTC every night, two hours before the puzzle day rolls over, a CI job runs on the Forgejo runner. It checks whether tomorrow’s puzzle exists in the catalogue. If it does, nothing happens. If it doesn’t, the job generates one, validates it, commits it, and deploys it. The site never goes dark.\n\nThe job is a single workflow file. The check step is four lines of bash:\n\n```\nTOMORROW=$(date -u -d '+1 day' +%Y-%m-%d)\nif [ -f \"workspaces/app/content/puzzles/$TOMORROW.json\" ]; then\n  echo \"puzzle for $TOMORROW is already authored — nothing to do\"\nfi\n```\n\nIf the file is missing, the rest of the job runs: install toolchain, install\ndependencies, call the same `gen:puzzle`\n\nscript I run manually. The model\ndefaults to DeepSeek v4 Flash. Medium difficulty, no pattern hint. The diversity\nconstraint (scan the last 7 puzzles’ technique tags, block repeats) keeps the\nmodel from producing three binary search puzzles in a row. It usually closes in\n2 attempts.\n\nThe generation is the easy part. The interesting thing is what comes after.\n\nThe generator script only writes files to disk when it passes every gate: schema validation, reference solution against all test cases, brute-force collapse proof, technique diversity. That’s the first safety net. But the CI job runs a second one anyway. After the generator finishes, it fires the full catalogue test suite:\n\n```\npnpm run check:ci\npnpm --filter @dailyprog/app exec vitest run lib/puzzles.test.ts lib/puzzle-quality.test.ts\n```\n\n`puzzles.test.ts`\n\nloads every `.solution.ts`\n\nfile in the catalogue and runs the\nreference solution against every visible and hidden test case. If the generator\nproduced a solution that doesn’t pass its own tests, this catches it.\n`puzzle-quality.test.ts`\n\nruns the guardrails: no duplicate technique in the\nrecent window, no back-to-back hard days, every language produces output within\nthe runtime budget. These are the same tests that run on every push to main. The\ngenerated puzzle goes through them twice.\n\nIf the tests pass, the job commits as puzzle-bot and pushes straight to main:\n\n```\nfeat: add puzzle 47 - staggered delivery\nAuto-generated by the nightly safety-net workflow.\nCo-Authored-By: DeepSeek V4 Flash <noreply@deepseek.com>\n```\n\nForgejo skips CI on pushes made with the run’s own token (standard loop\nprevention), so the commit doesn’t trigger another build. The puzzle is in git\nbut not on the site. One more step: the job POSTs to Forgejo’s API to dispatch\nthe deploy workflow. The deploy workflow SSHes into the VPS, pulls the new\ncommit, rebuilds the container image with `--no-cache`\n\n, and brings it up. The\nnew puzzle goes live about 10 minutes after generation started.\n\nThe two-hour lead time matters. I check the site most nights. If the model produced something weird (a puzzle where the English prompt is technically correct but the story doesn’t make sense, or one that passes every gate but just feels lazy), I have two hours to write a replacement. The safety net isn’t a substitute for human review. It’s a guarantee that the worst case is “a puzzle I didn’t author” rather than “no puzzle at all.”\n\nThe app doesn’t know any of this exists. It reads JSON files from\n`content/puzzles/`\n\n. The CI runner lives on the same VPS as the site, as an\nunprivileged user with no special access. The Deepseek API key is a repo secret\nscoped to the workflow. If the runner goes down, the site still serves puzzles\nfrom whatever is already in the catalogue.\n\nThe manual path and the automatic path converge on the same checks. I author a puzzle, run the tests, commit, deploy. CI generates one, runs the same tests, commits, deploys. The tests don’t care who wrote the JSON. If CI can’t pass them, the puzzle doesn’t ship. If it can, the puzzle is safe to serve.\n\nThere’s no handoff. No separate “auto-generated” code path with weaker validation. No flag that marks a puzzle as machine-authored and lets it skip a gate. The safety net is just the manual workflow running unattended.\n\nMost days I still write the puzzle myself. But the days I can’t, nobody notices :^)", "url": "https://wpnews.pro/news/the-dailyprog-puzzle-safety-net", "canonical_source": "https://blog.lvmbdv.dev/posts/the-puzzle-safety-net/", "published_at": "2026-07-16 02:21:03+00:00", "updated_at": "2026-07-16 02:56:22.545605+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence", "large-language-models"], "entities": ["dailyprog", "DeepSeek", "Forgejo", "DeepSeek V4 Flash"], "alternates": {"html": "https://wpnews.pro/news/the-dailyprog-puzzle-safety-net", "markdown": "https://wpnews.pro/news/the-dailyprog-puzzle-safety-net.md", "text": "https://wpnews.pro/news/the-dailyprog-puzzle-safety-net.txt", "jsonld": "https://wpnews.pro/news/the-dailyprog-puzzle-safety-net.jsonld"}}