Show HN: Notifyd – notification queue on Postgres alone (Rust, MCP, no UI)

Agent-first notification service. One Rust binary, Postgres only. Email, SMS, WhatsApp, push, in-app inbox, MCP server built in.

SKIP LOCKED gives you, and what it does not Ramzi Laieb, 2026-09-06. Applies to notifyd 0.2.1. Parts of the code and of this text were drafted with an AI coding assistant and reviewed by the author. Every citation was fetched from its primary source on 2026-09-06; two that could not be fetched are marked as such.

SELECT … FOR UPDATE SKIP LOCKED has made PostgreSQL a credible job queue since 9.5 (2016) and a default one in several ecosystems (Rails 8’s Solid Queue, Oban, River, Graphile Worker, pg-boss). Notification delivery, however, is not a generic job: the bottleneck is not the database but a third-party provider with an account-wide quota, whose 429 answers must be interpreted, whose failures must be classified, and whose limits must be shared between a password reset and a 5 000-recipient campaign. This article surveys what the Postgres-queue literature and the existing queues provide, identifies six concerns that the claim primitive leaves open, describes how notifyd (a single Rust binary, Postgres as its only dependency) addresses them, and reports measurements on commodity hardware: an unoptimised implementation drained 516 jobs/s; batching the claim, the context lookups and the finalisation, and removing a per-job task, brought it to 3 537 jobs/s on the same 8-vCPU host with 23 MB of resident memory. We then list what these numbers do not show, and what remains open.

A product sends two kinds of notifications through the same providers: transactional messages (order shipped, password reset) whose value decays in seconds, and campaigns whose value is indifferent to a delay of an hour. Both go out through providers that meter by account: Resend’s default limit is 10 requests per second per team with a 100-message batch endpoint [R1, R2]; Amazon SES exposes a per-second sending rate that “you can exceed for short bursts, but not for sustained periods” [R3]; Postmark caps a batch at 500 messages [R4]. A provider answers excess with 429 Too Many Requests, which RFC 6585 allows to carry a Retry-After header [L3]. Since February 2024, bulk senders to Gmail and Yahoo must also keep the reported spam rate under 0.3 % and honour one-click unsubscribe within two days [L5, L6, L7].

The engineering question is therefore not “can Postgres hold a queue” (it can) but “what must sit between the queue and the provider so that a campaign never starves a password reset, a 429 never burns a retry, and an operator, human or agent, can see why something did not leave”.

SKIP LOCKED landed in PostgreSQL 9.5.0 on 2016-01-07, in the same release as INSERT … ON CONFLICT [P1]. The manual is explicit about its intended use and its cost: “Skipping locked rows provides an inconsistent view of the data, so this is not suitable for general purpose work, but can be used to avoid lock contention with multiple consumers accessing a queue-like table” [P2]. Two caveats in the same page matter for a queue: the table-level ROW SHARE lock is still taken, and with LIMIT, “locking stops once enough rows have been returned to satisfy the limit” [P2]. The alternative, advisory locks, is “faster, avoid[s] table bloat” but carries its own LIMIT hazard, which the manual illustrates with a query it labels -- danger! [P3].

The operational caveat is dead tuples. A queue row is inserted, updated to processing, then to sent or retry: three versions per job. Brandur Leach documented in 2015 how a long-running transaction elsewhere left “247311 dead row versions [that] cannot be removed yet” and pushed the lock query “from < 0.01 seconds to 0.1 s and above” [P5]. Crunchy Data’s 2021 write-up of the SKIP LOCKED + DELETE … RETURNING pattern ends on the same advice: monitor bloat, tune autovacuum, possibly rotate the table [P6]. Hatchet’s 2026 survival guide is blunter: “If autovacuum can’t keep up … you’ll get into a very unhealthy state, very quickly” [P7]. The manual itself notes that “some installations with extremely high update rates vacuum their busiest tables as often as once every few minutes” [P4]. PostgreSQL 17 made vacuum’s dead-tuple storage up to 20× smaller [P8]; PostgreSQL 18 added autovacuum_vacuum_max_threshold, a fixed dead-tuple trigger that no longer scales with table size, plus asynchronous I/O covering vacuum and B-tree skip scans [P9]. River’s 2026 “concurrent repack” describes the remaining gap: vacuum “marks space as reusable … but never fully reclaims it” [P10].

Table 1 summarises the queues whose source or documentation we read. All but two use SKIP LOCKED; most add LISTEN/NOTIFY to wake workers.

Table 1. Only Graphile Worker and River publish a jobs/s figure with the hardware. None of the surveyed queues ships a per-channel token bucket or a notion of “the provider said 429” inside the queue: rate limiting, where it exists, bounds the consumer’s own concurrency, not a third party’s quota.

Two figures from Table 1 frame our results. Graphile Worker’s ~183 000 jobs/s and River’s ~46 000 jobs/s are for no-op jobs; the authors say so, and River adds that “benchmarking is a highly imperfect science” [Q4]. A notification job is not a no-op: it resolves a sender identity, checks suppressions and preferences, renders a template, builds an RFC 5322 message with unsubscribe headers, and calls a provider. Section 5 measures that path with the provider call stubbed, which is the right comparison point for the engine, and states what it leaves out.

The notification platforms we could read handle the provider limit differently from what a queue reader might expect. Novu’s documentation states that a channel step “makes a single call to the provider. There is no automatic retry, no backoff, and no ceiling, because there is no second attempt”, and that “a 429 Too Many Requests … from a provider is handled the same way as a 400 Bad Request” [N1]; the worker source confirms that the send is wrapped in a try/catch that records PROVIDER_ERROR without inspecting the status code [N2]. Knock, Courier and SuprSend each document a “throttle” step, but all three define it as a per-recipient anti-flood control (“limit the number of times a workflow is executed for a recipient within a given window” [N3]; “how many … messages a user or group receives within a set timeframe” [N4]; “rate limit workflow executions per user” [N5]), not as pacing against the provider. Their API rate limits are documented on the inbound side [N3]. We found no public documentation of provider-side 429 pacing at any of the four.

The mechanisms we use are old and documented. Token buckets bound rate and burst [L1]; RFC 2697 formalises a two-rate variant [L2]. Exponential backoff with jitter is analysed by Brooker (2015), whose “full jitter” and “decorrelated jitter” formulas are the usual references [L4]. Retry-After is specified in RFC 9110 §10.2.3 and attached to 429 by RFC 6585 §4 [L3]. One-click unsubscribe is RFC 8058 [L5]; Gmail’s and Yahoo’s 2024 requirements make it mandatory above 5 000 messages a day and set the 0.3 % spam-rate ceiling [L6, L7]. Staging jobs in the same transaction as the business write, so that a crash between commit and enqueue cannot lose them, is Leach’s “transactionally staged job drain” [D2]; the broader “use Postgres, spend your innovation tokens elsewhere” argument is McKinley’s [D1] and Hunt’s [D3].

Reading Sections 2.1 to 2.3 together, six concerns are not addressed by the claim statement, and are only partly addressed by the queues built on it.

429. 422 unverified sender, a 503, a network timeout and a Err forces the caller to encode this, and most callers do not.SELECT count(*) … GROUP BY status at 3 a.m. The queues in Table 1 expose metrics; the platforms in 2.3 expose dashboards. Neither says what to do. notifyd is one Rust binary (axum, sqlx, tokio) with PostgreSQL as its only dependency. This section describes the parts of it that answer Section 3. All SQL below is quoted from src/worker.rs and src/api/send.rs at 0.2.1.

Jobs are rows. A worker claims a batch in a transaction, orders by priority then by schedule, skips channels that a provider has asked to , and marks the batch in a second statement:

SELECT … FROM jobs
WHERE status IN ('pending', 'retry')
  AND scheduled_at <= $1
  AND (next_retry_at IS NULL OR next_retry_at <= $1)
  AND NOT (channel = ANY($3))          -- channels d after a 429
ORDER BY priority ASC, scheduled_at ASC
LIMIT $2
FOR UPDATE SKIP LOCKED;

UPDATE jobs SET status = 'processing', attempts = attempts + 1, claimed_at = now()
WHERE id = ANY($1);

A partial index ``` ON jobs (priority, scheduled_at) WHERE status IN ('pending', 'retry')

 keeps the claim cheap as the table fills with `sent` rows. Priority
is a 0–100 integer; the API accepts `critical` (10), `high` (30), `normal`
(50), `low` (70), `bulk` (80) or a number, and `POST /v1/batch` defaults to
`bulk`, as does any send tagged `campaign`, `marketing` or `newsletter`.

A token bucket per channel (`EMAIL_RATE_PER_SEC`, `SMS_RATE_PER_SEC`) bounds
outbound calls per replica [L1]. When a provider answers `429`, the worker
(a) tries the fallback provider if one is configured, then (b) s the
*channel* for `Retry-After` when the provider sent one, or a configured
default otherwise, and (c) re-queues the job **without consuming an
attempt**:

UPDATE jobs SET status = 'retry', attempts = GREATEST(attempts - 1, 0), error = $2, next_retry_at = $3 WHERE id = $1;

Priorities do not bypass the . The provider’s limit is per account, so a
`critical` message sent during the  would be refused as well. What the
design guarantees is order on resume: the claim statement puts `critical`
ahead of `bulk`, so the password reset is in the first batch after
`Retry-After` elapses, whatever the campaign backlog. Other channels are not
affected by an email .

Connectors return a typed error: `RateLimited { retry_after }`, `Transient`,
`Permanent`, `Suppressed`. Only `Transient` and `RateLimited` trigger the
failover breaker; `Permanent` (4xx other than 429, SQLSTATE 23xxx on
in-app inserts) fails immediately; `Suppressed` records the outcome without
calling the provider. Transient failures follow a fixed schedule of 30 s,
2 min, 10 min, 30 min, 2 h with ±20 % jitter [L4], five attempts by default.
A reaper re-queues jobs left in `processing` for more than ten minutes, so a
worker that dies mid-batch loses at most that.

Each connector declares `batch_max()`: 100 for Resend (its API’s ceiling
[R2]), 1 for SMTP, Twilio and Telnyx. The worker chunks the claimed batch
accordingly. A batch refused with a 4xx is retried item by item, so one bad
address does not fail 99 good ones. Successful items are finalised in one
statement:

UPDATE jobs SET status = 'sent', sent_at = now(), error = NULL, provider = r.provider, provider_message_id = r.mid FROM unnest($1::uuid[], $2::text[], $3::text[]) AS r(id, provider, mid) WHERE jobs.id = r.id;

Before the change measured in Section 5, each job performed its own sender
lookup, suppression check, preference check and spawned a webhook task that
created an HTTP client and queried the project’s webhooks. After it, the
worker loads senders, suppressions, preferences and the set of projects that
have webhooks once per claimed batch; if that prefetch fails, it falls back
to the per-job query rather than skipping the check (a suppression must
never be bypassed because a cache failed). `POST /v1/batch` inserts its N
jobs with one `INSERT … SELECT FROM unnest(…)`, with the idempotency
conflict handled by a partial unique index on ```
(project_id, idempotency_key)
WHERE status NOT IN ('failed', 'cancelled')

.

Marketing email carries RFC 8058 headers pointing at an HMAC-signed unsubscribe URL [L5]; suppressions have a scope (all or marketing) so a customer who leaves the newsletter still receives their invoice. Send windows are evaluated in the recipient’s timezone. GET /v1/admin/digest ranks findings (d channel, bounce rate above 2 % or 5 %, oldest waiting job, failed jobs with their top cause, missing fallback provider) and attaches to each one the action an operator would take. The same operations are exposed as MCP tools with readOnlyHint / destructiveHint annotations, and a read-only key restricts an agent to the reporting subset. This is the answer to concern 6: not a dashboard, but a ranked list with actions that a human or an agent can execute.

| Host | 8 vCPU Arm Neoverse-V2, 30 GB RAM, Ubuntu, Docker | | PostgreSQL | 16, default postgresql.conf , one container on the same host | | notifyd | 0.2.x release build; WORKER_BATCH_SIZE=500 ,WORKER_POLL_INTERVAL_MS=100 ,DATABASE_MAX_CONNECTIONS=20 ,EMAIL_RATE_PER_SEC=0 (pacer disabled) | | Provider | EMAIL_PROVIDER=log , a no-op connector withbatch_max = 100 , i.e. the Resend code path minus the network | | Load | one project, unlimited inbound rate; 100 000 email jobs enqueued through POST /v1/batch in calls of 5 000, allscheduled_at in the future, released by oneUPDATE , drained by one worker | | Measurement | wall-clock from release to count(*) WHERE status IN ('pending','processing','retry') = 0 ; RSS sampled every 250 ms withps ;sent_at - scheduled_at percentiles from the table |

The commands are in docs/BENCHMARKS.md. Everything runs on one machine, which flatters latency and penalises nothing else; it is the setup a small company actually deploys.

Table 2. Same host, same Postgres, same load. “Before” is the per-job implementation; “after” is Section 4.4–4.5.

The 6.9× drain improvement did not come from SKIP LOCKED, which was already there; it came from removing per-job round trips (concern 3) and batching finalisation (concern 4). With a non-batching provider the same engine caps near 640 jobs/s: each message becomes its own provider call and its own finalisation UPDATE, which is the shape SMTP and SMS impose in production anyway, where the provider’s quota, not the engine, is the ceiling.

Against Table 1, 3 537 jobs/s is an order of magnitude below Graphile Worker and River, and it should be: a no-op job has none of the fixed costs listed in Section 3, and our poll interval (100 ms, no LISTEN/NOTIFY) bounds minimum latency. For the target workload the relevant comparison is the provider: at Resend’s 10 requests/s × 100 recipients, 100 000 emails leave in roughly 100 s if the provider allows the burst and around 8–9 minutes at a paced 2 requests/s; the engine spends most of that time waiting on the token bucket, at 23 MB of memory.

LISTEN/NOTIFY. NOTIFY, the worker does not. finalize_sent_batch commits results in a resend after the reaper fires; the provider’s idempotency, where it exists, is not used.autovacuum_vacuum_max_threshold [P9] is the right default for a queue table that sees 3 × N tuple versions per N jobs is an empirical question we have not answered.UPDATE … RETURNING) costs a round trip per batch and would be exact; River’s Pro roadmap names the same problem [Q4].(status, priority, scheduled_at) index serve both the claim and the operator queries; we have not measured it. Code: github.com/rmzlb/notifyd, tag v0.2.1, MIT. Benchmark protocol and commands: docs/BENCHMARKS.md. Unit tests: cargo test (65 tests, no database required). The numbers in Table 2 were produced on 2026-09-05 and 2026-09-06 on the host described in 5.1; we will link any independent measurement, on any hardware, that follows the protocol.

PostgreSQL

SELECT, “The Locking Clause”. https://www.postgresql.org/docs/current/sql-select.html Queues

lib/oban/engines/basic.ex. Notification platforms and providers

apps/worker/src/app/workflow/usecases/send-message/send-message-email.usecase.ts, branch next, read 2026-09-06. Mechanisms and requirements

Discourse