{"slug": "gate-your-app-on-a-postgres-migration-job-with-service-completed-successfully", "title": "Gate your app on a Postgres migration job with service_completed_successfully", "summary": "A developer documented a Docker Compose pattern that gates an API service on a one-shot Postgres migration job using `condition: service_completed_successfully`, after an initial deploy failed when the migration container could not resolve the database hostname. The writeup adds a wait loop and a TCP-based `pg_isready` healthcheck to the standard three-service setup, and reports a second rollout completing in 59 seconds.", "body_md": "*Originally published on the [Hubfly blog](https://blog.hubfly.space/docker-compose-migration-job-service-completed-successfully?utm_source=devto&utm_campaign=docker-compose-migration-job-service-completed-successfully).*\n\n**TL;DR**\n\n**What you get:** a 3-service Compose file (Postgres, a one-shot migration job, an API) where the API cannot start until the migration exits 0.\n**Who it's for:** anyone whose app has ever booted against a database with a missing table.\n**Time:** about 20 minutes. Our second rollout took 59 seconds from click to \"Stack deployment finished\".\n**The twist:** our first deploy failed. The migration job died in 2 seconds. The likely cause: a brand-new container couldn't resolve the database's hostname yet. The fix is a wait loop, and it's in the complete file at the end of Step 3.\n\n*By Octave Nkurunziza · Drafted with AI, tested on Hubfly on 16 October 2026*\n\nYou push a release. The API boots, runs its first query, and gets `relation \"todos\" does not exist`. The migration ran too, just 3 seconds later.\n\nMost Compose files only say `depends_on: [db]`. That waits for the database *container* to start, not for the schema.\n\nDocker's docs say so plainly: Compose waits until a dependency is running, not until it is ready ([Docker docs](https://docs.docker.com/compose/how-tos/startup-order/)).\n\nThe fix is well known: a healthcheck on Postgres, a one-shot migration service, and `condition: service_completed_successfully` on the app. What the popular guides skip is what happens when the gate itself misfires. We ran this on real infrastructure and it misfired on the first try. This post shows the file that survived.\n\n```\ndb (postgres:16-alpine)          stage 1\n  │  healthcheck: pg_isready over TCP\n  ▼  condition: service_healthy\nmigrate (one-shot job, exits 0)  stage 2\n  │  waits for DNS + port, then runs idempotent SQL\n  ▼  condition: service_completed_successfully\napi (PostgREST, serves /todos)   stage 3\n```\n\nThe API is [PostgREST](https://postgrest.org), which turns a Postgres schema into a REST API. It's a good test subject because it reads the schema at boot. If the migration hasn't run, there is nothing to serve.\n\nHubfly's composer turned these 3 services into 3 deploy stages. It marked `migrate` as a **Job** with restart policy **Never**, because of `restart: \"no\"`.\n\n```\ndb:\n  image: postgres:16-alpine\n  environment:\n    POSTGRES_DB: app\n    POSTGRES_USER: app\n    POSTGRES_PASSWORD: lab-only-change-me   # replace before you deploy\n  volumes:\n    - pgdata:/var/lib/postgresql/data\n  healthcheck:\n    test: [\"CMD-SHELL\", \"pg_isready -h 127.0.0.1 -U app -d app\"]\n    interval: 5s\n    timeout: 3s\n    retries: 10\n```\n\nNote the `-h 127.0.0.1`. Without it, `pg_isready` checks the Unix socket.\n\nOn first boot, the official image starts a temporary server to create your database. In our logs, that temporary server listened **only** on the Unix socket. The real server came up afterwards with `listening on IPv4 address \"0.0.0.0\", port 5432`.\n\nA socket check could go green during that setup window, so we switched to TCP as a precaution. A TCP check only passes once other containers can actually connect.\n\n```\nmigrate:\n  image: postgres:16-alpine\n  restart: \"no\"\n  environment:\n    PGHOST: \"{{service:db.host}}\"\n    PGUSER: app\n    PGPASSWORD: lab-only-change-me\n    PGDATABASE: app\n  command:\n    - sh\n    - -c\n    - |\n      timeout 60 sh -c 'until pg_isready -t 2; do echo \"waiting for db\"; sleep 2; done' || exit 1\n      psql -v ON_ERROR_STOP=1 \\\n        -c \"CREATE TABLE IF NOT EXISTS todos (id serial PRIMARY KEY, task text NOT NULL, done boolean NOT NULL DEFAULT false)\" \\\n        -c \"DO 'BEGIN IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = ''web_anon'') THEN CREATE ROLE web_anon NOLOGIN; END IF; END'\" \\\n        -c \"GRANT USAGE ON SCHEMA public TO web_anon\" \\\n        -c \"GRANT SELECT ON todos TO web_anon\" \\\n        -c \"INSERT INTO todos (task) SELECT 'ship the migration gate' WHERE NOT EXISTS (SELECT 1 FROM todos)\"\n  depends_on:\n    db:\n      condition: service_healthy\n```\n\nThree details matter here.\n\n`timeout 60` keeps a broken database from hanging the rollout forever; after a minute the job exits 1 and the gate stays closed.`ON_ERROR_STOP=1`.` psql` happily continues past a failed statement and exits 0. Your gate would open on a half-migrated schema.`NOTICE: relation \"todos\" already exists, skipping` and `INSERT 0 0`, and exited 0.\n`{{service:db.host}}` is Hubfly's service reference. At deploy time it became the database's private hostname, `db-1-7befe5b4.eucentral.hubfly.internal`.\n\nOn plain Docker Compose (we didn't test this part), replace `{{service:db.host}}` with the service name `db` in both places: `PGHOST: db` and `PGRST_DB_URI: postgres://app:…@db:5432/app`.\n\nThe `DO '...'` block uses ordinary quotes instead of `$$`. That's deliberate: Compose treats `$` as variable interpolation, and quoted SQL dodges the whole escaping question.\n\n```\n  api:\n    image: postgrest/postgrest:v12.2.3\n    environment:\n      PGRST_DB_URI: \"postgres://app:lab-only-change-me@{{service:db.host}}:5432/app\"\n      PGRST_DB_SCHEMAS: public\n      PGRST_DB_ANON_ROLE: web_anon\n      PGRST_SERVER_PORT: \"3000\"\n    ports:\n      - \"3000:3000\"\n    depends_on:\n      migrate:\n        condition: service_completed_successfully\n\nvolumes:\n  pgdata:\n```\n\n`service_completed_successfully` means exit code 0, nothing else. If the job exits 1, the API is never created. That's the whole point: a bad migration stops the release instead of shipping it.\n\nThe anonymous role gets `SELECT` only. We checked: a `POST /todos` from the browser returned **401** with `permission denied for table todos`.\n\nFor production, have PostgREST log in as a dedicated low-privilege authenticator role instead of the `app` owner. We kept one user here to keep the file short.\n\nHere's the whole file, exactly as we deployed it:\n\n```\nservices:\n  db:\n    image: postgres:16-alpine\n    environment:\n      POSTGRES_DB: app\n      POSTGRES_USER: app\n      POSTGRES_PASSWORD: lab-only-change-me\n    volumes:\n      - pgdata:/var/lib/postgresql/data\n    healthcheck:\n      test: [\"CMD-SHELL\", \"pg_isready -h 127.0.0.1 -U app -d app\"]\n      interval: 5s\n      timeout: 3s\n      retries: 10\n\n  migrate:\n    image: postgres:16-alpine\n    restart: \"no\"\n    environment:\n      PGHOST: \"{{service:db.host}}\"\n      PGUSER: app\n      PGPASSWORD: lab-only-change-me\n      PGDATABASE: app\n    command:\n      - sh\n      - -c\n      - |\n        timeout 60 sh -c 'until pg_isready -t 2; do echo \"waiting for db\"; sleep 2; done' || exit 1\n        psql -v ON_ERROR_STOP=1 \\\n          -c \"CREATE TABLE IF NOT EXISTS todos (id serial PRIMARY KEY, task text NOT NULL, done boolean NOT NULL DEFAULT false)\" \\\n          -c \"DO 'BEGIN IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = ''web_anon'') THEN CREATE ROLE web_anon NOLOGIN; END IF; END'\" \\\n          -c \"GRANT USAGE ON SCHEMA public TO web_anon\" \\\n          -c \"GRANT SELECT ON todos TO web_anon\" \\\n          -c \"INSERT INTO todos (task) SELECT 'ship the migration gate' WHERE NOT EXISTS (SELECT 1 FROM todos)\"\n    depends_on:\n      db:\n        condition: service_healthy\n\n  api:\n    image: postgrest/postgrest:v12.2.3\n    environment:\n      PGRST_DB_URI: \"postgres://app:lab-only-change-me@{{service:db.host}}:5432/app\"\n      PGRST_DB_SCHEMAS: public\n      PGRST_DB_ANON_ROLE: web_anon\n      PGRST_SERVER_PORT: \"3000\"\n    ports:\n      - \"3000:3000\"\n    depends_on:\n      migrate:\n        condition: service_completed_successfully\n\nvolumes:\n  pgdata:\n```\n\n`pgdata` volume, and 2 `DEPENDS` arrows. The composer's monthly estimate for this stack on the shared tier was `api` container, open `https://<your-subdomain>.eu1.hubfly.app/todos`.\nWe got this back:\n\n```\n[{\"id\":1,\"task\":\"ship the migration gate\",\"done\":false}]\n```\n\nA second `GET /todos` from the browser took 271 ms end to end.\n\nOne surprise: `ports: \"3000:3000\"` also created a public TCP endpoint on `tcp.eu1.hubfly.app` by itself. That's fine for an API. Never add `ports:` to `db`.\n\n| Time (CAT) | Event | \n|---|---|\n| 06:32:16 | Deploy starts, access validated | \n| 06:32:20 | Volume `pgdata` reused | \n| 06:32:36 | `db` deployed (stage 1) | \n| 06:32:38 | `db` healthcheck satisfied | \n| 06:32:53 | `migrate` deployed (stage 2) | \n| 06:32:55 | `migrate` exited 0, gate opens | \n| 06:33:13 | `api` deployed (stage 3) | \n| 06:33:14 | Stack deployment finished | \n\nHubfly marked `db` healthy about 2 seconds after it deployed, in both runs. So don't rely on the healthcheck alone. The wait loop is what made the second run work.\n\nOur first file didn't have the wait loop or the TCP healthcheck. Here's what happened.\n\n`db` went healthy at 06:28:24. `migrate` deployed at 06:28:36 and failed 2 seconds later. The API was never created, so the gate did its job. But why did a migration fail against a healthy database?\n\nThe job's container was rolled back, so its logs were gone. We can't prove the cause, but 2 clues from the second run point to it.\n\nThe migration job's own first line, with the wait loop in place:\n\n```\ndb-1-7befe5b4.eucentral.hubfly.internal:5432 - no response\nwaiting for db\ndb-1-7befe5b4.eucentral.hubfly.internal:5432 - accepting connections\nCREATE TABLE\n...\n```\n\nAnd PostgREST's first 2 seconds of life (container logs are in UTC; the deploy table is CAT, UTC+2):\n\n```\n04:33:10 Failed to query the PostgreSQL version ... could not translate host name\n         \"db-1-7befe5b4.eucentral.hubfly.internal\" to address: Name does not resolve\n04:33:12 Successfully connected to PostgreSQL 16.15\n04:33:14 Schema cache loaded 1 Relations\n```\n\nIn the second run, a freshly started container could not resolve the database's private hostname for about 2 seconds. PostgREST shrugs that off because it retries. A one-shot `psql` doesn't retry. It fails, exits non-zero, and takes your deploy down with it.\n\nThat's not a Hubfly quirk you can ignore elsewhere. On any orchestrator, \"the dependency is healthy\" and \"my container can reach it right now\" are two different facts. A job that runs once must check the second one itself.\n\n**1. The job fails in 2 seconds and you can't see why.**\n\nA failed job may be rolled back with its logs. Add the wait loop from Step 2, and make the job print what it's doing (`echo \"waiting for db\"`). If it still fails, the next run's logs will show whether it's DNS, the port or the SQL.\n\n**2. \"Healthy\" doesn't mean \"reachable\".**\n\n`pg_isready` without `-h` checks the Unix socket, and the first-boot setup server listens only there. Use `pg_isready -h 127.0.0.1 -U app -d app`, and still keep the wait loop in the job: in our runs, `db` was marked healthy about 2 seconds after deploy either way.\n\n**3. A failed rollout leaves an old database behind.**\n\nAfter our failed first deploy, the rollback of `db` itself failed (`Rollback failed for 'db'`). The old `db` was left behind, asleep and still attached to the volume. Our second deploy then created `db-1` next to it, on the same `pgdata` volume.\n\nTwo Postgres servers on one data directory is how you lose data. Before you redeploy after a failure, open the project, and stop or remove any leftover database container.\n\n**4. The migration isn't idempotent.**\n\nThe job runs on every deploy. `CREATE TABLE` without `IF NOT EXISTS` will fail on deploy number 2 and block every release after it. Guard every statement, or use a migration tool that records what it applied (Prisma's `migrate deploy`, Flyway, golang-migrate, Alembic).\n\n`service_completed_successfully` is a good gate. It only works if the job behind it is honest: it waits for the database itself, stops on the first error, and is safe to run again. Put those 3 properties in your migration job and a broken schema stops the release instead of reaching users.\n\nPaste the Compose file into a new Hubfly project (**Create → Compose Stack → YAML**) and click **Deploy Stack**. The shared tier needs no card. Change `lab-only-change-me` first.\n\nThe composer estimated $0.01/month for this stack on the shared tier. That matches a 1 GB volume: 1 GB × $0.00001536 per GB-hour × 730 h = $0.011 (the free tier includes 1 GB of storage).\n\nShared containers auto-sleep, and their compute is free within the free tier's 360 hours a month. Two containers kept awake all month would use 1,460 hours.\n\nIf you keep db and api awake using a dedicated container (0.5 vCPU, 256 MB):\n\nCPU: 0.5 = $1.46\n\nRAM: 256 MB = $0.36\n\nPer container: $1.94/month; for db + api: $3.88/month, plus $0.11 of storage\n\nThe migration job runs for a couple of seconds per deploy, which rounds to $0.00.\n\n**Read next:**\n\n`{{service:...}}` references", "url": "https://wpnews.pro/news/gate-your-app-on-a-postgres-migration-job-with-service-completed-successfully", "canonical_source": "https://dev.to/octave_nkurunziza_afb0512/gate-your-app-on-a-postgres-migration-job-with-servicecompletedsuccessfully-and-the-dns-race-1pee", "published_at": "2026-10-16 06:28:55+00:00", "updated_at": "2026-10-16 06:35:29.753226+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure"], "entities": ["Docker", "Postgres", "PostgREST", "Hubfly", "Octave Nkurunziza"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/gate-your-app-on-a-postgres-migration-job-with-service-completed-successfully", "markdown": "https://wpnews.pro/news/gate-your-app-on-a-postgres-migration-job-with-service-completed-successfully.md", "text": "https://wpnews.pro/news/gate-your-app-on-a-postgres-migration-job-with-service-completed-successfully.txt", "jsonld": "https://wpnews.pro/news/gate-your-app-on-a-postgres-migration-job-with-service-completed-successfully.jsonld"}}