{"slug": "maintaining-wordpress-sites-behind-http-basic-auth-playwright-urllib-and", "title": "Maintaining WordPress sites behind HTTP Basic auth — Playwright, urllib, and encrypted credentials", "summary": "A developer built a utility module to handle HTTP Basic auth credentials for WordPress maintenance tools using Playwright and urllib. The module centralizes credential extraction and encryption, preventing format mismatches and silent failures in rollback decisions. It uses Fernet encryption for passwords at rest and ensures existing sites without auth keys continue to work unchanged.", "body_md": "It's pretty common to throw a layer of HTTP Basic auth on a WordPress site: a staging environment before launch, an internal test instance only employees should see, or any environment that wants an extra gate before the WordPress login screen itself.\n\nFrom a maintenance-tool point of view, this setup creates a peculiar **\"half-working, half-broken\"** asymmetry. The SSH/WP-CLI side runs fine. But everything HTTP-based — visual checks, thumbnail generation, browser-based fallback updates — hits 401 and dies. This post walks through how we resolved that asymmetry.\n\nA maintenance tool actually touches a Basic-auth-protected site through two distinct paths:\n\n`browser.new_context()`\n\n→ navigation → screenshotWith no credentials, both paths see a **401 Unauthorized** from the protected site.\n\nThe Playwright symptom is the obvious one: the screenshot you save is the browser's \"authentication required\" dialog. The thumbnail grid fills with dark auth-prompt images, and you start wondering whether anything actually works.\n\nThe urllib symptom is much worse — **it silently breaks rollback decisions**. A 401 baseline followed by another 401 after the update looks like \"nothing changed = healthy.\" Real failures can hide behind that match, and the rollback that should have fired never does.\n\nWhen the same credentials need to flow through multiple code paths, picking them out of the site dict separately at each call site invites format-mismatch and missed-update bugs. So the first thing we did was build a small `core/basic_auth_utils.py`\n\nmodule that **owns every form of credential extraction**.\n\n``` python\n# core/basic_auth_utils.py\ndef get_basic_auth_tuple(site):\n    \"\"\"Return (user, password), or None if not configured.\"\"\"\n    if not isinstance(site, dict):\n        return None\n    user = (site.get('basic_auth_user') or '').strip()\n    pw = site.get('basic_auth_password') or ''\n    if not user:\n        return None  # No user → treat as \"no auth\"\n    return (user, pw)\n\ndef get_playwright_http_credentials(site):\n    \"\"\"Returns dict for Playwright's new_context(http_credentials=...).\"\"\"\n    auth = get_basic_auth_tuple(site)\n    if auth is None:\n        return None\n    return {'username': auth[0], 'password': auth[1]}\n\ndef get_basic_auth_header(site):\n    \"\"\"Returns {'Authorization': 'Basic <base64>'} for urllib.\"\"\"\n    auth = get_basic_auth_tuple(site)\n    if auth is None:\n        return {}\n    raw = f\"{auth[0]}:{auth[1]}\".encode('utf-8')\n    encoded = base64.b64encode(raw).decode('ascii')\n    return {'Authorization': f'Basic {encoded}'}\n```\n\nThe key idea: **both the Playwright and urllib forms derive from the same get_basic_auth_tuple() root**. Format drift between the two callers becomes structurally impossible.\n\nA small but important detail: **the existing site-configuration JSON has no basic_auth_user / basic_auth_password keys**. Naively writing\n\n`site['basic_auth_user']`\n\nwould crash with `KeyError`\n\nthe moment someone opens an existing site.We went with the `site.get('basic_auth_user') or ''`\n\nempty-string-fallback pattern. Missing key, empty string, or `None`\n\nall collapse to \"no auth,\" so existing sites behave exactly as before. Only the sites that actually set Basic auth flip into the authenticated path.\n\nOn top of that, the Basic auth password gets the same treatment as the WordPress admin password: **Fernet-encrypted at rest**. Adding `'basic_auth_password'`\n\nto the `ENCRYPTED_SITE_KEYS`\n\nconstant is all it takes — encryption and decryption happen automatically on save and load.\n\nWith the helpers in place, the call sites get rewired.\n\n**Playwright path**: one shared helper `_new_context_with_auth(browser, site)`\n\nreplaces three `new_context()`\n\ncall sites (visual check, thumbnail capture, browser residual update) at once.\n\n``` python\ndef _new_context_with_auth(browser, site):\n    http_credentials = get_playwright_http_credentials(site)\n    if http_credentials:\n        return browser.new_context(http_credentials=http_credentials)\n    return browser.new_context()\n```\n\n**urllib path**: `_http_status_check(url, basic_auth=None)`\n\ngains a `basic_auth`\n\nparameter and sends the Authorization header internally. The \"baseline 401 → post-update 401\" false negative disappears — the rollback decision now sees the real status code after authentication (200 / 5xx / etc.) and fires correctly.\n\nInside the maintenance main loop `run_ssh_maintenance`\n\n, we **extract _basic_auth from the site dict exactly once** and pass it into all five\n\n`_http_status_check_stable()`\n\ncalls. Pulling it out from the dict at every call site invites \"this one place forgot,\" so a local variable is the safer move.From a user perspective, Basic-auth-protected sites are the minority. Putting two always-visible input fields in the site-add modal would clutter the UI for the 99% of sites that don't need them.\n\nSo we put the credentials behind a `<details>`\n\nelement — **a collapsed-by-default \"🔐 Basic Auth (optional)\" section** at the end of the WordPress info group in the site-add/edit modal.\n\n```\n<details>\n  <summary>🔐 Basic Auth (optional)</summary>\n  <input name=\"basic_auth_user\" placeholder=\"Auth username\">\n  <input type=\"password\" name=\"basic_auth_password\"\n         placeholder=\"Auth password\">\n</details>\n```\n\nUsers with Basic-auth-protected sites expand and fill it in; the rest see no change. Saved passwords go through Fernet with the `ENC:`\n\nprefix.\n\nThree principles worth keeping from this round:\n\n`(user, password)`\n\nroot function. That keeps format drift from happening`site.get(key) or ''`\n\ntreats missing keys as \"feature off,\" letting you add functionality without touching existing data. No migration needed`<details>`\n\ncollapse with a small icon expresses \"optional feature\" cleanly, and gets out of the way for everyone elseStaging environments behind Basic auth aren't rare in the WordPress world. If your automation tool is \"half working\" against those sites, teams quickly fall back to manual maintenance just for that subset. Designing a clean credential path through both the Playwright and urllib sides — once — is worth the up-front investment.", "url": "https://wpnews.pro/news/maintaining-wordpress-sites-behind-http-basic-auth-playwright-urllib-and", "canonical_source": "https://dev.to/susumun/maintaining-wordpress-sites-behind-http-basic-auth-playwright-urllib-and-encrypted-credentials-5f2f", "published_at": "2026-07-01 00:56:50+00:00", "updated_at": "2026-07-01 01:18:45.179099+00:00", "lang": "en", "topics": ["developer-tools", "large-language-models"], "entities": ["Playwright", "urllib", "WordPress", "Fernet"], "alternates": {"html": "https://wpnews.pro/news/maintaining-wordpress-sites-behind-http-basic-auth-playwright-urllib-and", "markdown": "https://wpnews.pro/news/maintaining-wordpress-sites-behind-http-basic-auth-playwright-urllib-and.md", "text": "https://wpnews.pro/news/maintaining-wordpress-sites-behind-http-basic-auth-playwright-urllib-and.txt", "jsonld": "https://wpnews.pro/news/maintaining-wordpress-sites-behind-http-basic-auth-playwright-urllib-and.jsonld"}}