# Phase 8 — Making It Trustworthy: Hardening a FastAPI App with an Audit, a Test Net, and a Logging Bug I Typed Twice > Source: > Published: 2026-08-16 13:58:28+00:00 Phase 7 gave the app a brain. Phase 8 was about making it *trustworthy* — the unglamorous gate between "works on my machine" and "safe to put a URL in front of." No new features. Instead: a security audit that probed my own API for leaks, a global exception handler so unexpected errors stop leaking internals, a five-test pytest suite that turns every manual check into a permanent one, and a git-history secrets sweep. Plus the war story — a logging placeholder I fat-fingered the same way *twice*, a test that silently never ran, and a login that failed because of one capital letter. Spoiler: the app had zero leaks. Almost every bug this phase was in the *terminal*, not the code. `/health` endpointPhase 7b ended with the seam holding: a category column, a rules engine, and a Gemini-backed `LLMCategorizer` living behind the same `Categorizer` contract — validated, cached, and with a fallback that was finally loud about failing. I closed that post nagging at three things: the cache dies on restart, the model name is a drifting alias, and my free tier can't afford to be the default. All real. All *enhancements*. But there was a bigger question I'd been walking past: **is this thing actually safe to deploy?** I had auth, I had owner-scoped queries, I *believed* users were isolated — but belief isn't proof. Phase 8 is where I stop believing and start verifying. The temptation was to jump straight to "add security stuff." I didn't. You can't harden what you haven't observed, so I ordered the work from *lowest risk and highest evidence* outward: | Step | What | Why this order | |---|---|---| | 1 | A `/health` endpoint | Purely additive warm-up; also the thing my host will ping in Phase 9 | | 2 | Audit the real API for leaks | Get evidence before writing a single fix — don't guess what leaks | | 3 | Global exception handler | The one gap the audit couldn't test: the unexpected 500 | | 4–5 | A pytest suite | Turn every manual proof into a permanent, re-runnable one | | 6 | Pagination | Put a ceiling on an unbounded list before it becomes a problem | | 7 | Secrets / git-history sweep | The one item with real downside if skipped before going public | The principle from Phase 7 still drives it: **observe first, then change one thing at a time.** When something breaks, I want to know it broke in the thing I just touched. `/health` endpoint Every real route in my app is auth-gated behind `get_current_user` . That's correct for expenses — but a monitoring system *can't log in*. So I need exactly one deliberately public, side-effect-free route that answers "are you alive?" with a fast, boring `200` . ``` python @app.get("/health") def health_check(): return {"status": "ok"} ``` | Idea | Why it matters | |---|---| No `Depends` , no auth | The absence is the point — a monitor hits it without a token | | No DB call | It answers "is the process up," not "is every subsystem healthy." A deeper check is a conscious later step | Returns a plain `dict` | FastAPI serializes it to JSON automatically — no `response_model` needed for something this trivial | Additive, zero risk, and it's the exact URL Phase 9's host will hit to confirm the deploy is up before I even try to log in. Good warm-up. This is the part I'd skipped in every previous project: actually attacking my own endpoints and *recording what they return*. Not "I think it 404s" — curl it and read the status line. I grabbed a token once, stashed it in a shell variable, and fired four deliberately-bad requests. Here's the evidence table that came out: | # | Probe | Expected | Actual | Verdict | |---|---|---|---|---| | 1 | POST expense, negative amount | 422 | 422 | Input validation holds | | 2 | GET `/expenses` , garbage token | 401 | 401 | Auth boundary holds | | 3 | GET a non-existent expense id | 404 | 404 | Not-found path is clean | | 4 | GET another user's expense id | 404 | 404 | Cross-user isolation holds | Two things I want to call out, because they're *design* wins, not luck: **Probe #1 dies at the schema, not the database.** My `ExpenseCreate` declares `amount` with `gt=0` , so Pydantic rejects a negative amount before my route body ever runs. Nothing gets written. Validation at the edge means the dangerous code never executes. **Probe #4 returns 404, not 403 — on purpose.** My query filters by *both* `id` and `user_id == current_user.id` , so a row I don't own comes back as `None` → 404. A `403 Forbidden` would be the wrong choice here: it would confirm "this expense exists, you're just not allowed to see it," which leaks the record's existence. `404` says "nothing here for you" and reveals nothing. The candid bit: the "raw stack trace" fear I'd been carrying is mostly unfounded on FastAPI. It doesn't put tracebacks in the HTTP response by default — it returns a generic 500 and logs the trace server-side. Tracebacks only leak into the body if you run with debug on. So the real production job isn't "hide the trace," it's "make sure debug stays off, and add a net for the unexpected." Which is Step 3. The audit found zero leaks — but it also *couldn't* test one thing, because nothing in my app currently throws an unhandled error. Production code eventually will. When it does, I want a deliberate, consistent response — not whatever Starlette's default is. FastAPI already handles two error families: my intentional `HTTPException` s (401/404/409) and validation errors (422). What it doesn't handle is the *unexpected* — a bug, a `None` where I assumed a value. A catch-all lets me log the full traceback **server-side** with a correlation id, and return a clean generic body to the client. ``` python @app.exception_handler(Exception) async def unhandled_exception_handler(request: Request, exc: Exception): error_id = uuid.uuid4().hex[:8] logger.exception( "Unhandled error [%s] on %s %s", error_id, request.method, request.url.path ) return JSONResponse( status_code=500, content={"detail": "Internal server error", "error_id": error_id}, ) ``` | Concept | What it does | |---|---| `@app.exception_handler(Exception)` | Catches only what a more specific handler didn't — my `HTTPException` s and 422s keep their exact status codes | `logger.exception(...)` | Logs at ERROR level with the full traceback — the server-side record | `uuid4().hex[:8]` | A short correlation id, printed in both the log and the response | Generic `{"detail", "error_id"}` body | The client sees a reference code to quote to support — never internals | The critical nuance is that first row. A handler on the broad `Exception` type does *not* swallow my deliberate `HTTPException` s — if it did, every 404 would turn into a 500. So the design is a floor under the *unexpected*, not a replacement for my existing errors. And I proved it — not by faith, by a throwaway `/debug/boom` route that raised on demand, watched the handler catch it, then deleted the route. The 404 regression check afterward confirmed the catch-all left my intentional errors alone. (Getting that log line to actually format is its own war story below.) Everything I proved in Step 2 with curl — the 422, the 401, the 404, the isolation — evaporates the moment I change code. A test turns a one-time proof into a permanent guarantee, and it's the single most credible signal to a stranger reading the repo that the app actually works. FastAPI's `TestClient` calls the app **in-process** — no uvicorn, no network, fast and isolated. I proved the harness with one trivial `/health` test first (prove the tooling before you trust the result), then added the real ones. The interesting piece is testing *authenticated* paths without a live login, using `dependency_overrides` — and the trap that comes with it: ``` python @pytest.fixture def as_authenticated_user(): def _fake_user(): return User(id=1, name="Test User", email="test@example.com", hashed_password="x") app.dependency_overrides[get_current_user] = _fake_user yield app.dependency_overrides.clear() ``` | Concept | What it does | |---|---| `dependency_overrides` | Swaps a real dependency for a fake — auth is satisfied, the route runs | A fixture, not a global | The trap: a global override poisons every test, silently breaking the 401 test | `yield` split | Everything before is setup, everything after is teardown — the `.clear()` is what keeps tests isolated | The proof that the teardown works: after adding the fixture, my `test_expenses_requires_auth` test *still* returned 401. If the override had leaked, it would've gotten the fake user and failed. Isolation intact. The highest-value test is the one thing I'd only proven by hand: cross-user isolation. But it *writes* to the database, so it can't touch my real `expenses.db` . That means a proper test-database fixture — and the trickiest concept of the whole phase. ``` python @pytest.fixture def test_db(): engine = create_engine( "sqlite://", connect_args={"check_same_thread": False}, poolclass=StaticPool, ) Base.metadata.create_all(bind=engine) TestingSessionLocal = sessionmaker(bind=engine, autoflush=False, autocommit=False) def _override_get_db(): db = TestingSessionLocal() try: yield db finally: db.close() app.dependency_overrides[get_db] = _override_get_db seed = TestingSessionLocal() try: yield seed finally: seed.close() app.dependency_overrides.clear() Base.metadata.drop_all(bind=engine) ``` The `StaticPool` line is the one to internalize. `TestClient` runs requests in a thread pool, and a normal in-memory SQLite database is *per-connection* — a second connection sees an *empty* database, so my seeded rows would vanish. `poolclass=StaticPool` forces SQLAlchemy to reuse **one single connection** for everything, so the schema and seeded data persist across request threads. Without it, this test fails in a baffling way. The test itself seeds two users and one expense owned by user2, then makes the *same* request as each identity: ``` python def test_cross_user_isolation(test_db): # ... seed user1, user2, and an expense owned by user2 ... app.dependency_overrides[get_current_user] = lambda: User(id=u2_id) owner = client.get(f"/expenses/{expense_id}") assert owner.status_code == 200 # the control app.dependency_overrides[get_current_user] = lambda: User(id=u1_id) other = client.get(f"/expenses/{expense_id}") assert other.status_code == 404 # the guarantee ``` The `200` assertion is the *control*, and it's what stops this from being a false pass. It proves the row genuinely exists and is readable — so the following `404` can *only* mean my owner-scoped query hid it from a different user. Same id, two identities, two outcomes. That's the isolation guarantee, now permanent. (This test almost didn't run at all — see the war story.) My `list_expenses` was returning *every* row for the user with no ceiling. Fine at 12 rows, a latent problem at 10,000. Two query params fix it — with validation at the edge: ``` python @app.get("/expenses", response_model=List[ExpenseRead]) def list_expenses( db: Session = Depends(get_db), current_user: User = Depends(get_current_user), limit: int = Query(default=50, ge=1, le=100), offset: int = Query(default=0, ge=0), ): expenses = db.scalars( select(Expense) .where(Expense.user_id == current_user.id) .order_by(Expense.spent_on.desc(), Expense.id.desc()) .limit(limit) .offset(offset) ).all() return expenses ``` The non-obvious part: **offset/limit is meaningless without a deterministic ORDER BY.** SQLite makes no promise about row order unless you ask, so "skip 20, take 20" could overlap or drop rows between requests if the order shifts. Pagination and ordering are a package deal — hence the `order_by` .The actual hardening is `le=100` : a caller asking for `limit=999` gets a clean 422 instead of defeating the whole point. And I named the behavior change out loud — the list now returns *max 50 by default* where it used to return everything — so future-me isn't surprised when the frontend eventually needs a "load more." The one mental model to never forget: **git history is a security camera that never stops recording, and deleting a file does not delete its past.** If a secret appeared in *any* commit — even one I later "undid" — it's still recoverable and will go public the moment I push. So a secrets audit is really three questions, and people who get burned only ask the first: | Question | Command | What it proves | |---|---|---| Tracked now? | `git ls-files \ | grep -iE "env\ | | File ever committed? | {% raw %}`git log --all --oneline -- backend/.env` | Past tense, by filename | Value ever committed, any file? | `git log --all --oneline -S "actual-value"` | Past tense, by content | All three came back clean. But question 1 taught a bonus lesson: it matched `backend/alembic/env.py` — Alembic's migration script, which just has the letters "env" in its name. A total false positive. **A filter matches strings, not meaning** — so you always eyeball *what* matched instead of trusting the filter. The through-line from question 3: a secret leaks by *content*, not filename. Someone hardcodes `SECRET_KEY = "..."` into a `.py` "just to test," commits it, then later moves it to `.env` . The `.env` history is spotless — but the value is in that old commit forever. Only the pickaxe (`git log -S` ) searching for the actual value catches that. (Honest note I wrote down: a clean pickaxe only proves the *current* values were never committed. If a secret has ever lived somewhere I'm unsure about, the bulletproof move before a public deploy is to rotate it anyway. Rotation beats archaeology.) Phase 7b's theme was "everything that isn't your code lying to you." Phase 8's theme was quieter and more humbling: **my own typing, and tools doing exactly what I typed instead of what I meant.** Almost none of it was in the application code. My shiny new exception handler ran, the client got its 500 — but the server log was garbage: ``` Message: 'Unhandled error [$s] on $s %s' Arguments: ('a06f2a97', 'GET', '/debug/boom') ``` See the `$s` ? I'd typed `$s` instead of `%s` . Python's `logging` uses `%` -style formatting, so the interpolation failed and it dumped the raw message + args block. I fixed the middle one, re-ran — and it *still* broke, because I'd missed the one *inside the brackets*: `[$s]` . On a US keyboard `%` is Shift+5 and `$` is Shift+4, adjacent keys. I hit the wrong one, in the same line, twice. The lesson isn't "type carefully." It's that the whole *point* of the handler — a readable server-side log — was silently defeated by one wrong character, and the only reason I caught it was reading the actual log output instead of assuming it worked. Same energy as Phase 7b's silent `except` . I added `test_cross_user_isolation` , ran pytest, saw green, and almost celebrated. Then I read the first line: ``` collected 3 items ``` Three. I'd written a fourth. It wasn't failing — it *didn't exist* as far as pytest was concerned. The function had gotten indented so it was nested inside another, which makes it invisible to collection (pytest only finds top-level `test_*` functions). No error, no failure, just... absent. Now `collected N items` is the *first* thing I read, before the pass/fail. A green suite that silently skips your most important test is worse than a red one. Coming back after a few days, my saved token was stale (JWTs expire; shell variables die with the terminal). Fine — re-authenticate. Except the login kept returning a `KeyError: 'access_token'` . I tried `passwordd` . Failed. I *knew* the password was `Passwordd` with a capital P — tried that. Still failed. The fix, as always: stop guessing, strip the pipeline, read the raw response. The real problem was the *email* — I'd been typing `frontend1@test.com` when the account was `Frontend1@test.com` . Capital F. The design lesson underneath: my login returns the same generic `401` for a bad email *or* a bad password — which is correct (never tell an attacker which field was wrong) — but it means the error can't tell you which one you fat-fingered. That's exactly why "read the raw response" beats "stare at the error message." One curl command died with: ``` bash: $'\302\226\302\226curl': command not found ``` `\302\226` is the UTF-8 for an invisible control character. Two of them had ridden along when I pasted from a rich-text source, so bash tried to run a command literally named `‖‖curl` . The command was perfect; the paste smuggled in garbage bytes. Same family as Phase 7b's hidden comma — you can't debug what you can't see, so `cat -v` (which prints control characters) and re-typing by hand are the escape hatches. The through-line across all four: before I'd fixed a single line of *application* code this phase, my keyboard, my clipboard, my shell, and pytest's collector had each quietly done something other than what I intended. The bug is rarely where you first look — and it's often not even in your code. Same habit as always — try to break it before reality does: | Attack | Defense | |---|---| | POST a negative amount | Pydantic `gt=0` → 422, dies at the schema, never touches the DB | | Present a garbage/expired token | `get_current_user` → clean 401 with `WWW-Authenticate` | | Request another user's expense id | Owner-scoped query → 404 (not 403 — don't confirm it exists) | | Trigger an unhandled 500 | Global handler → generic body + correlation id, full trace stays server-side | Request `limit=999999` to dump everything | `Query(le=100)` → 422 | | Read a secret out of "deleted" git history | Swept by filename and value across all branches → clean | | I did (learning) | Production would | |---|---| | Manual curl audit, then automated it | Audit runs in CI on every push, blocks merge on a regression | | 5 focused tests | Broader coverage: every endpoint, edge cases, a real coverage target | In-memory `StaticPool` test DB | Same pattern scales fine — just more fixtures and factories | `print` /`logger` to stdout | Structured logging with levels + an alert on sustained 500s | | Deferred prod CORS to Phase 9 | Env-driven allowed origins, locked to the real frontend URL | | Called the secret history "clean" | Rotate secrets anyway before a public deploy — rotation beats archaeology | Ignored the `httpx` /`httpx2` deprecation warning | Resolve it before it becomes a hard break on a dependency bump | None of these are wrong for where I am. They're *known* — written down, not pretended away. `$s` log, the uncollected test, the capital-F login — every one was caught by `/health` first means any later red is a real failure, not a setup problem.`collected N items` is the first line you read.`KeyError` in a wrapper hides the `{"detail": ...}` that would've told you the answer.`dependency_override` poisons every other test. `yield` + teardown keeps them isolated.`200` control prevents a false-pass `404` .`alembic/env.py` was innocent — always eyeball what matched.The app is trustworthy now. The audit found no leaks, the unexpected 500 has a net, five tests lock in every guarantee that used to live only in my memory, and the git history is clean. What's left is the last mile: getting it *onto the internet*. Phase 9 is deployment — starting with the one item I consciously deferred (production CORS, once I know the real frontend URL), then Render itself, where my own lessons come home to roost: Linux means `venv/bin/activate` , not `Scripts/` , and paths are case-sensitive (the capital-F login bug, now at the OS level). Secrets become real environment variables in a dashboard, not a `.env` file. And I'll confirm debug is off, closing the loop on the exception handler. The features were the fun part. Turns out making them *trustworthy* was the part that actually taught me something. See you in the next one. silentcarry