Webhooks and cron jobs that fail safely in Next.js: signatures, retries and missed-run alarms

A developer behind AI Craftbook detailed how the small Next.js backend on Vercel verifies inbound Resend webhooks using Svix's HMAC-SHA256 signature scheme, checking the raw request body, a five-minute timestamp tolerance, and constant-time comparison to reject forged, tampered, and stale requests. The writeup also covers idempotency keys to prevent duplicate side effects from provider retries, bounded retries and request caps for upstream rate limits, and run logging with gap alarms to catch silently stopped cron jobs. Test output showed valid signatures accepted, tampered and stale payloads rejected, and duplicate deliveries both passing verification.

A webhook endpoint is a public URL that does real work when someone posts to it. A cron job is a function that only does its job if a scheduler remembers to call it. Both fail quietly: a forged request looks like a normal one, a retried delivery looks like a new one, and a cron that stopped running produces no errors at all. This is how we handle those failures in the small Next.js backend behind AI Craftbook, on Vercel. One inbound webhook forwards support email. One weekly cron job checks GitHub for changes to the projects we list. Code was written with Claude Code under our direction. Every output below comes from a test or a script we ran. Known gaps in our code are called out where they come up. | Failure | What it looks like | Defence | |---|---|---| | Forged request | Anyone who finds the URL posts a payload | Signature check on the raw body | | Replay | A captured, validly signed request is sent again later | Timestamp inside the signature, short tolerance | | Provider retry | The same event arrives twice; side effects run twice | Idempotency key per message | | Rate limit | Upstream API answers 403 or 429 mid-run | Bounded retries, conditional requests, a request cap | | Silent cron stop | Nothing runs, so nothing errors | Record each run; alarm on the gap | Our inbound email provider, Resend, signs webhooks with Svix. The Svix manual verification docs https://docs.svix.com/receiving/verifying-payloads/how-manual define the scheme: the signed content is ${svix id}.${svix timestamp}.${body} , the key is the base64 part of a whsec ... secret, the algorithm is HMAC-SHA256, and the svix-signature header holds a space-separated list of versioned signatures such as v1,<base64 . Here is our verifier, unchanged: js import { createHmac, timingSafeEqual } from "node:crypto"; const TOLERANCE SECONDS = 5 60; / Svix signature check Resend webhooks are signed by Svix . Secret is the "whsec ..." value. / export function verifyWebhook body: string, headers: Headers, secret: string, now = Date.now : boolean { const id = headers.get "svix-id" ; const timestamp = headers.get "svix-timestamp" ; const signatures = headers.get "svix-signature" ; if id || timestamp || signatures || secret.startsWith "whsec " return false; const ts = Number timestamp ; if Number.isFinite ts || Math.abs now / 1000 - ts TOLERANCE SECONDS return false; const key = Buffer.from secret.slice "whsec ".length , "base64" ; const want = Buffer.from createHmac "sha256", key .update ${id}.${timestamp}.${body} .digest "base64" ; return signatures.split " " .some entry = { const version, sig = entry.split "," ; const got = Buffer.from sig || "" ; return version === "v1" && got.length === want.length && timingSafeEqual got, want ; } ; } A few details matter more than they look: await request.text and verifies that string before JSON.parse . Resend's docs warn that parsing and re-serialising the JSON can break the signature. v1,bogus v1,<real to prove a bad first entry does not cause a rejection. timingSafeEqual . RangeError when the buffers differ in length we checked, output below . Comparing lengths first turns that into a clean false . We ran this exact function against a fabricated payload and a fake secret FAKE SIGNING KEY FOR DOCS , importing it by path with tsx : valid true tampered body false stale 10 min false edge 4m59s old true wrong secret false same delivery x2 true true The last line comes up again in section 2. The timestamp is part of the signed string, so an attacker cannot refresh it without the secret. We reject anything more than five minutes away from our clock in either direction, which matches the window Svix's own libraries use according to their docs https://docs.svix.com/receiving/verifying-payloads/why . That narrows a replay to five minutes, but it does not prevent one. The same delivery x2 true true line shows it: the same signed request is accepted twice inside the window. A signature proves who sent the request. It does not prove the request is new. Making duplicates harmless is a separate job, covered in section 4. Svix treats any 2xx as delivered and anything else including redirects, or no answer within 30 seconds as a failure. It then retries on a schedule documented in Svix's retry docs https://docs.svix.com/retries : immediately, then 5 seconds, 5 minutes, 30 minutes, 2 hours, 5 hours, 10 hours and 10 hours apart. Resend's own retry docs https://resend.com/docs/dashboard/webhooks/retries-and-replays list the same schedule. Our route uses that on purpose. Trimmed, with environment lookups removed: js export const runtime = "nodejs"; export const maxDuration = 60; export async function POST request: Request { // ... 503 if the signing secret is not configured const body = await request.text ; if verifyWebhook body, request.headers, secret return Response.json { error: "bad signature" }, { status: 401 } ; const event = JSON.parse body ; if isForSupport event return Response.json { ok: true, skipped: true } ; const result = await forwardReceived event.data.email id, { / ... / } ; // A non-2xx makes Resend retry the webhook. return Response.json result, { status: result.sent ? 200 : 502 } ; } Events we do not care about get a 200, so they are not retried. A failed forward gets a 502, so it is. Attachments are capped at 30 MB in total, which limits how much one email can make the handler download. There is a catch here. Our handler does its work inline: it fetches the email, downloads the attachments and sends the copy before it answers. The function may run for up to 60 seconds, but Svix stops waiting after 30. If a forward takes longer than that, or succeeds but the response is lost, the provider records a failure and sends the event again. The email is then forwarded twice. We checked: our inbound route does not de-duplicate on svix-id . A retried delivery runs the forward again. For support email, the cost is a duplicate message in an inbox. It is not a security hole, because every delivery still has to pass the signature check. It is still a gap, and if the webhook moved money or changed records it would be a serious one. The Svix docs say the message ID stays the same when a message is re-sent after a failure, so it works as an idempotency key. The usual pattern is to claim the ID in a table with a unique constraint before the side effect runs, and to release the claim if the side effect fails so the next retry can try again. Not yet in our code; illustrative. We ran this sketch against in-memory SQLite node:sqlite , Node 24 . It has not been run on a production database: js import { DatabaseSync } from "node:sqlite"; const db = new DatabaseSync ":memory:" ; db.exec "create table webhook deliveries id text primary key, received at text not null " ; const claim = db.prepare "insert into webhook deliveries id, received at values ?, ? on conflict id do nothing returning id" ; const release = db.prepare "delete from webhook deliveries where id = ?" ; async function handle msgId: string, send: = Promise<void : Promise<number { if claim.get msgId, new Date .toISOString return 200; // seen before: acknowledge, do nothing try { await send ; return 200; } catch { release.run msgId ; // let the provider's retry try again return 502; } } Output from our demo, which counts sends: first delivery 200 sent = 1 retry, same id 200 sent = 1 new id, fails 502 sent = 1 retry of msg 2 200 sent = 2 Postgres supports the same insert ... on conflict do nothing returning form we have not run it there . This is "at most once per ID", not "exactly once". If the process dies after the send but before it answers, the claim is kept and the retry is skipped. For email forwarding, that is the failure we would choose. There is a second trade-off, and it hits exactly the slow-handler case from section 3. If a retry arrives while the first attempt is still running, the claim already exists, so the retry gets a 200. If the first attempt then fails and releases its claim, the provider has already recorded success and will not try again, so the message is lost. We checked this with the same sketch: the overlapping retry got 200 and the first attempt got 502. One fix is to store a status with each claim and answer a non-2xx while a claim is still marked in progress, so the provider keeps retrying. A cron route is a public URL too. Vercel calls it with an HTTP GET to the production deployment, on a schedule in UTC Vercel cron docs https://vercel.com/docs/cron-jobs . Our vercel.json has a single cron entry shown trimmed, with the path shortened : { "crons": { "path": "/api/cron/...", "schedule": "0 13 1" } } That is Mondays at 13:00 UTC. Vercel's docs https://vercel.com/docs/cron-jobs/manage-cron-jobs say that if the project has an environment variable named CRON SECRET , Vercel sends it as Authorization: Bearer <value . They recommend a random value of at least 16 characters. Their sample compares the header with == . Ours is stricter. This is our check, with the environment lookup moved out so the secret is a parameter: js import { timingSafeEqual } from "node:crypto"; function authorized header: string | null, secret: string | undefined : boolean { if secret || secret.length < 32 return false; const got = Buffer.from header || "" ; const want = Buffer.from Bearer ${secret} ; return got.length === want.length && timingSafeEqual got, want ; } We require a shared secret of at least 32 characters. If the secret is missing or too short, the endpoint refuses every request instead of falling back to open. Output with a fake 36-character token: correct header true no header false wrong, same len false wrong length false secret unset false secret too short false timingSafeEqual on unequal lengths throws: RangeError Two more points from the same Vercel page shape everything below. Vercel does not retry a failed cron invocation. Delivery is best effort: a run can be missed, and occasionally the same run is invoked twice. So a cron job has to cope with missed runs and duplicate runs. The weekly job reads repository metadata from the GitHub REST API. This is the core of our client: async api<T pathname: string, etag?: string : Promise<ApiResult<T { for let attempt = 0; ; attempt++ { this.budget.take ; await sleep this.opts.delayMs ; const res = await fetch https://api.github.com${pathname} , { headers: this.headers etag , cache: "no-store" } ; if res.status === 304 return { status: 304, etag, notModified: true }; if res.status === 403 || res.status === 429 || res.status = 500 && attempt < this.opts.maxRetries { await sleep 2000 attempt + 1 ; continue; } const out: ApiResult<T = { status: res.status, etag: res.headers.get "etag" || undefined }; if res.ok out.data = await res.json as T; return out; } } We retry 403, 429 and 5xx, because GitHub documents https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api 403 or 429 for rate limits. We do not retry other 4xx answers: a 404 on the repository is treated as a deletion and reported as a change, not retried. GitHub's best-practices page notes that a 404 can also come from missing access to a private resource, so check credentials before trusting it. We ran the real class with fetch replaced by a local fake no network, retries set to 2, delay 0 : php 1 503 then 200 +0.0s fake GitHub - 503 +2.0s fake GitHub - 200 result: {"status":200,...,"data":{"ok":true}} | budget used: 2 2 429 three times retries exhausted +0.0s fake GitHub - 429 +2.0s fake GitHub - 429 +6.0s fake GitHub - 429 result: {"status":429,...} | budget used: 3 3 404 not retried +0.0s fake GitHub - 404 result: {"status":404,...} | budget used: 1 The output also shows where our client falls short of GitHub's guidance. The backoff is linear 2 s, then 4 s , not exponential. We do not read retry-after or x-ratelimit-reset , which GitHub's best-practices page https://docs.github.com/en/rest/using-the-rest-api/best-practices-for-using-the-rest-api says to wait for. It says to wait at least a minute when neither header is present. We also retry every 403, including permission errors that will never succeed. For a once-a-week job with a few retries, the cost is small. For a busier job it would matter. We store the etag from each response and send it back as If-None-Match next week. GitHub answers 304 Not Modified with no body if nothing changed. According to GitHub, a 304 does not count against the primary rate limit when the request was authorized with an Authorization header. Scenario 4 from the same demo: php 4 conditional request answered 304 +0.0s fake GitHub - 304 If-None-Match sent result: {"status":304,"etag":"W/\"fake-etag-1\"","notModified":true} | budget used: 1 The caller has to treat notModified as "same as last time", not as an error. Our source checker does that. The docs-page checker in the same job does the same with If-None-Match and If-Modified-Since , and a unit test asserts that the second request sends the stored ETag and reports no change. Each request goes through a counter that throws when a per-run cap is exceeded. The cap is set in a config file, not in code. Retries draw from the same budget, so a retry storm cannot exceed it. Scenario 5, with the cap set to 2: php 5 budget of 2 requests, 500 every time +0.0s fake GitHub - 500 +2.0s fake GitHub - 500 threw: github: request cap 2 reached. Raise it in pipeline.config.json if this is expected. In the GitHub source checker, each source has its own try / catch , so hitting the cap shows up as an error for each remaining source. Those errors go into the run's alert email, so the run never reports success when it skipped work. Because Vercel https://aicraftbook.com/mcp/vercel-mcp does not retry and does not promise delivery, the job writes a row at the start of every run. It then reads the start time of the previous run: / A scheduled run is weekly; a gap longer than this means at least one run did not happen. / export const MISSED RUN DAYS = 8; if gapDays == null && gapDays MISSED RUN DAYS lines.push The previous monitor run was ${Math.floor gapDays } days ago. At least one weekly run did not happen. ; Eight days gives a weekly job a day of slack before it counts as missed. The same alert lists errors from the run, so a failed GitHub call a 403, say is reported instead of passing as "no changes". The run also acts on what it finds. If a project we redistribute changes in a security-relevant way for example, its repository is deleted or files change in a folder whose text we show , the job withdraws that download from the site straight away, without waiting for an editor. The limit is that the gap check runs inside the job, so a missed run is only reported when the next run happens. If the schedule stops completely crons disabled, the entry removed in a deploy , nothing ever reports it. The fix is an external check, often called a dead man's switch: something outside the job that raises an alarm when a heartbeat stops arriving. We do not have one. TIM: confirm there is no external heartbeat on the Vercel cron, or describe it if there is. Most of the behaviour above has unit tests that need no network or secrets. Fakes stand in for fetch and for GitHub: inbound.test.ts : accepts a valid signature with a bogus first entry , rejects a tampered body, a 10-minute-old timestamp and missing headers. Forwarding sets Reply-To to the sender and does nothing without config. ops.test.ts : a deleted repository is a security change. A 403 is reported as an error, not a silent pass. A shared repository is fetched once per run. Alerts fire for security changes, errors and a gap of MISSED RUN DAYS + 1 , but not for a new release on its own. monitor-docs.test.ts : conditional headers are sent and a 304 counts as unchanged. mcp-monitor.test.ts : package-level changes new maintainers, removed packages, advisories are sorted by risk. On the day of writing, npx vitest run on these four files passed 35 of 35 tests. One gap: the retry and budget logic in the GitHub client has no unit test of its own. The ops tests use a fake API object that skips it. The demo in section 6 is currently its only check. This article was drafted with AI assistance Claude Code from our project's code and test records, and reviewed and edited by Tim S before publishing.