{"slug": "the-full-stack-specs-skills-stewardship-in-a-single-system", "title": "The Full Stack: Specs + Skills + Stewardship in a Single System", "summary": "A developer building an AI-powered software factory integrated three layers—specs, skills, and evals—into a single system to build an order cancellation feature. The system caught three specification gaps before implementation, demonstrating the value of combining all layers. The developer credits Dan Shapiro and Nate B. Jones for the conceptual frameworks.", "body_md": "*Building the AI Dark Factory — Issue #19*\n\nI want to be upfront about something before we get into it. None of the frameworks in this article is mine. The ideas here come from two people who have been thinking about this stuff way harder and longer than I have — and they deserve full credit before I say another word.\n\nDan Shapiro — CEO of Glowforge, Wharton Research Fellow, and the person who gave this whole conversation a vocabulary. His blog post “The Five Levels: from Spicy Autocomplete to the Dark Factory” is the conceptual spine of everything I’m about to say. Read the original. It’s short, sharp, and will make you uncomfortable in the best way. [danshapiro.com](https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory)\n\nNate B. Jones — AI strategist, zero-hype practitioner, and the person whose YouTube channel made me realize I had been fooling myself about where I actually sat on this ladder. His video “The 5 Levels of AI Coding (Why Most of You Won’t Make It Past Level 2)” is what triggered this entire newsletter. [natebjones.com](https://www.natebjones.com/) — [Watch the video](https://youtu.be/bDcgHzCBgmQ)\n\nThis newsletter — The Level 5 Engineer — is my public learning log. I’m a Senior Software Engineer and a Tech Lead, currently somewhere between Level 2 and Level 3 (in context of the title of this newsletter) on a good day. The goal is Level 5. I’m documenting the climb in real time — the frameworks, the tools, the mindset shifts, and the moments where I realize I’ve been doing it wrong. If you’re on a similar journey, pull up a chair.\n\nSeventeen issues of building. One question left to answer: does it work?\n\nNot in isolation — each layer has been demonstrated in isolation. Layer 1's spec caught the payment field rename in Issue #4. Layer 2's skill review caught the routing signal length in Issue #12. Layer 3's eval caught the synchronous notification change in Issue #17.\n\nBut those were individual demonstrations of individual artifacts. This issue builds a complete new feature — order cancellation — using all three layers simultaneously, and compares the result to Issue #3, when the agent had only a spec and nothing else.\n\n`DELETE /orders/{order_id}`\n\nwith five business rules:\n\nFive rules. Three external service interactions. Two ADRs directly applicable. One new feature that touches every layer of the infrastructure.\n\nBefore any code, six Gherkin scenarios were written for `order_cancellation.feature`\n\n. Then the Gherkin quality skill v2.0 was applied to each one.\n\nThe skill caught three items the first draft left open.\n\n**Scenario 1 (happy path):** First draft — `Then the order status is \"CANCELLED\"`\n\n. Skill correction: missing HTTP status assertion and missing inventory release service call assertion. The scenario described the outcome but not the observable mechanism. Rewritten to: `Then the response HTTP status is 200`\n\n, `And the response body contains \"status\" equal to \"CANCELLED\"`\n\n, and `And the inventory service receives a release request for SHOE-RED-42 and BELT-BRN-M`\n\n.\n\n**Scenario 5 (idempotency):** First draft — `Then the response indicates the order is already cancelled`\n\n. Skill correction: UNDEFINED TERM — \"indicates\" is not an observable check. Rewritten to: `Then the response HTTP status is 200`\n\nand `And the response body contains \"status\" equal to \"CANCELLED\"`\n\n. No error. No new side effects. Two assertions that two agents would implement identically.\n\n**Scenario 6 (non-cancellable state):** First draft — `Given an order in PAYMENT_PENDING status`\n\n. Skill correction: UNDERSPECIFIED GIVEN — \"an order in PAYMENT_PENDING status\" does not specify how it got there. Rewritten to: `Given an order was created via POST /orders with order ID \"order-del-pend-001\" and payment is pending`\n\n.\n\nThree corrections before a single line of implementation. Each correction is a decision that would have been made silently in the implementation if the spec had been handed over uncorrected.\n\nThree skills were consulted before implementation began.\n\n**Gherkin quality skill v2.0** — applied to all six scenarios as described above. Three corrections produced.\n\n**Step definition style skill** — consulted before writing `tests/steps/test_order_cancellation.py`\n\n. The skill's five conventions were followed: fixture injection from `conftest.py`\n\n, mock server state asserted via call log rather than response body, the `reset_all_logs`\n\nautouse fixture, `time.sleep(0.3)`\n\nbefore async side-effect assertions, and `_delete_order`\n\nas the shared helper following the `_post_order`\n\nnaming convention from the existing files.\n\nOne structural consistency check from the skill: the new step definition file uses `def response(user_id, ...)`\n\nfixture injection, not `def test_cancellation(...)`\n\ntest function style. Without the skill, the file might have used a different fixture pattern — the other test files use this pattern consistently, but an agent reading only `order_cancellation.feature`\n\nwould not necessarily infer it.\n\n**Feature file audit skill** — run against `order_cancellation.feature`\n\nafter the quality skill corrections. Found zero additional debt items. The skill's Q5 check (\"what does this scenario NOT say that it should?\") flagged the absence of a scenario for concurrent cancellation attempts — two requests for the same order arriving simultaneously. Documented as a known gap rather than a spec debt item: this is a judgment call about scope, not an ambiguity about behavior.\n\nOne skill gap revealed: no skill exists for WireMock stub design. The decision to add `POST /inventory/release/{scenario}`\n\nas the release mechanism — rather than a `DELETE`\n\nor a patch to the existing inventory endpoint — was made without consulting any artifact. This is an implicit decision that a stub design skill would have caught.\n\nBefore any modification to `app/main.py`\n\n, the operation scope eval was run.\n\n**Q1 — ADR coverage:** The decision index identified two applicable ADRs.\n\nADR-001 agent check questions applied to the cancellation flow:\n\nADR-002 agent check questions applied to the notification call:\n\nBoth ADR checks passed before implementation began. The notification thread was implemented correctly on the first attempt — not because the agent inferred it, but because ADR-002 stated explicitly that asynchronous was non-negotiable and described the daemon thread pattern.\n\n**Q2 — Ordering of external service calls:** Cancellation flow ordering documented before implementation: inventory release first, then notification (fire-and-forget). Payment gateway not called. Ordering committed in the findings before the first line of code.\n\n**Q3 — Synchronicity:** The notification call — async. The inventory release call — synchronous. Cancellation confirmation depends on inventory release success; it does not depend on notification delivery.\n\nThe environment eval was not triggered — no infrastructure files were modified.\n\n`DELETE /orders/{order_id}`\n\nimplemented in `app/main.py`\n\n. The cancellation logic:\n\n``` python\n@app.delete(\"/orders/{order_id}\")\ndef cancel_order(order_id: str):\n    order = ORDERS.get(order_id)\n    if not order:\n        raise HTTPException(status_code=404,\n                          detail=f\"Order {order_id} not found\")\n\n    # Idempotency: already cancelled returns success\n    if order[\"status\"] == \"CANCELLED\":\n        return {\"order_id\": order_id, \"status\": \"CANCELLED\",\n                \"message\": \"Order already cancelled.\",\n                \"status_code\": 200}\n\n    # Only CONFIRMED orders can be cancelled\n    if order[\"status\"] != \"CONFIRMED\":\n        return {\"order_id\": order_id, \"status\": order[\"status\"],\n                \"message\": f\"Order cannot be cancelled. Current status: {order['status']}\",\n                \"status_code\": 422}\n\n    # Release inventory reservation — synchronous, cancellation depends on it\n    try:\n        inv_resp = httpx.post(\n            f\"{INVENTORY_URL}/inventory/release/success\",\n            json={\"order_id\": order_id,\n                  \"items\": [i[\"sku\"] for i in order[\"items\"]]},\n            timeout=5.0\n        )\n        if inv_resp.status_code != 200:\n            raise HTTPException(status_code=503,\n                              detail=\"Inventory release failed\")\n    except httpx.RequestError as e:\n        raise HTTPException(status_code=503,\n                          detail=f\"Inventory service unavailable: {e}\")\n\n    # Update order status\n    order[\"status\"] = \"CANCELLED\"\n    order[\"cancelled_at\"] = datetime.utcnow().isoformat()\n\n    # Notify — fire-and-forget per ADR-002\n    def _notify():\n        try:\n            httpx.post(\n                f\"{NOTIFICATION_URL}/notifications/order-cancelled\",\n                json={\"order_id\": order_id, \"user_id\": order[\"user_id\"]},\n                timeout=5.0\n            )\n        except Exception:\n            pass  # Notification failure does not affect cancellation\n\n    threading.Thread(target=_notify, daemon=True).start()\n\n    return {\"order_id\": order_id, \"status\": \"CANCELLED\",\n            \"message\": \"Order successfully cancelled.\",\n            \"status_code\": 200}\n```\n\nTest results after first implementation attempt:\n\n```\npytest tests/steps/test_order_cancellation.py -v\n\ntest_successful_cancellation_of_confirmed_order PASSED\ntest_idempotent_cancellation_of_already_cancelled_order PASSED\ntest_cancellation_of_non_existent_order PASSED\ntest_rejection_of_cancellation_for_payment_pending_order PASSED\ntest_inventory_release_on_cancellation PASSED\ntest_fire_and_forget_notification_on_cancellation PASSED\n\n6 passed in 4.23s\n```\n\n6/6 on the first attempt. Full suite:\n\n```\npytest tests/steps/ -v    → 21 passed\npytest tests/pact/ -v     → 2 passed\npython scripts/can_i_deploy.py → SAFE TO DEPLOY\n```\n\nIssue #3 was the first agent implementation session. The agent was handed the Gherkin scenarios for the order creation endpoint and told to build. It derived the entire API contract correctly and found a portability bug in the human's code. It also made four implicit decisions.\n\nHere is the comparison across five dimensions.\n\n**Implicit decisions made**\n\nIssue #3: 4 implicit decisions — HTTP 404 for missing orders, ISO string timestamp format, in-memory store, no failure scenario for partial availability.\n\nIssue #19: 1 implicit decision — the `POST /inventory/release/success`\n\nURL path convention for the release stub. The stub design skill does not exist; this decision was made without consulting any artifact.\n\nThe three layers reduced implicit decisions from 4 to 1. The remaining one identifies a gap in the skill infrastructure, not a gap in the implementation.\n\n**Spec quality before implementation**\n\nIssue #3: the spec was handed to the agent without review. The timeout ambiguity (`And the response is returned within 12 seconds`\n\n) was introduced in Issue #2 and silently inherited for three sessions before Issue #8 caught it.\n\nIssue #19: the Gherkin quality skill caught three items before implementation. The idempotency scenario's \"indicates the order is already cancelled\" would have produced an ambiguous assertion in the step definition — the agent would have invented a field name. The skill replaced it with two concrete assertions before the step definition was written.\n\n**Stewardship artifacts consulted**\n\nIssue #3: none existed.\n\nIssue #19: three artifacts prevented three potential failures.\n\nADR-002 prevented the notification call from being made synchronous. Without it, the agent's default for a \"confirm delivery\" side effect would have been to await the response. ADR-002 stated the daemon thread pattern explicitly; the implementation used it on the first attempt.\n\nADR-001's Q2 check prevented an inventory-before-payment ambiguity in the cancellation flow — not because payment was involved, but because the ordering question was asked before implementation and documented. The inventory release was confirmed as the primary side effect before a line of code was written.\n\nThe operation scope eval's Q3 caught the notification synchronicity risk before implementation began. Not after a test failure — before the function was written.\n\n**Time spent on clarification vs implementation**\n\nIssue #3: clarification was zero — the agent inferred everything. Implementation took one pass. Four implicit decisions were made invisibly.\n\nIssue #19: pre-flight work — spec review, eval, ADR checks — took approximately the same time as the implementation itself. The implementation took one pass. One implicit decision was made, and it was documented in the findings rather than embedded silently in the code.\n\nThe overhead is real. The pre-flight is not faster than inferring. What it produces is different: documented decisions rather than invisible ones.\n\n**Test results after first implementation attempt**\n\nIssue #3: 5/5 passed — an exceptional result that Issue #3's article noted was partly due to the simplicity of the API.\n\nIssue #19: 6/6 passed — on an endpoint with more complexity (idempotency, two side effects, state machine), tested against six scenarios that the Gherkin quality skill had already hardened.\n\nOne implicit decision made in this session that none of the three layers caught: the stub URL path convention for inventory release.\n\nThe decision: `POST /inventory/release/{scenario}`\n\nmirrors the existing stub pattern of `POST /inventory/check/{scenario}`\n\n. This is the right choice — it is consistent with the project's conventions. But it was made without consulting any artifact. A stub design skill that documented the URL convention for mock endpoints would have made this explicit rather than inferred.\n\nThis is the only true gap. Everything else was either covered by an artifact or was a judgment call that should remain with the human: which scenarios to write (scope), whether to add a concurrent cancellation scenario (product decision), what error message text to use (editorial).\n\nThe distinction that matters: some judgment calls are implicit decisions that should be documented. Others are genuinely human judgment that no artifact can or should replace. The stub URL convention is the first category — it has a right answer that the project has already established. The concurrent cancellation scenario is the second category — it is a product decision about scope, not an engineering ambiguity about behavior.\n\nAfter seventeen issues of infrastructure, the remaining gap is small, specific, and nameable. That is the point.\n\nIssue #3, but harder.\n\nThe agent would have built a working cancellation endpoint. It would have made the notification call synchronous — because synchronous is the natural default for a side effect you want to confirm. It would have written `Then the order is cancelled`\n\nin the Gherkin scenarios — which would have produced an ambiguous step definition. It would have made the inventory release a flag rather than a service call — because `inventory_released: true`\n\nin the response body is simpler than an HTTP call to a separate service.\n\nAll three tests would have passed. The notification synchronicity violation would not have been caught until the first 2am notification incident. The spec ambiguity would have been inherited by the next agent session. The inventory flag would have stayed a flag until someone noticed it was not actually releasing inventory.\n\nThe three layers did not produce a faster first implementation. They produced a better one — with fewer invisible decisions, and one documented gap that the next session can close.\n\n*Next issue: The Productivity J-Curve — the honest accounting of what seventeen issues of infrastructure building actually cost, and whether the answer to \"is it worth it\" is yes.*\n\n**Sources & Further Reading**\n\n*This article was written with the assistance of AI tools.*", "url": "https://wpnews.pro/news/the-full-stack-specs-skills-stewardship-in-a-single-system", "canonical_source": "https://dev.to/diyaburman/the-full-stack-specs-skills-stewardship-in-a-single-system-3oa2", "published_at": "2026-08-20 13:30:00+00:00", "updated_at": "2026-08-20 13:45:25.366449+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "developer-tools"], "entities": ["Dan Shapiro", "Glowforge", "Nate B. Jones"], "alternates": {"html": "https://wpnews.pro/news/the-full-stack-specs-skills-stewardship-in-a-single-system", "markdown": "https://wpnews.pro/news/the-full-stack-specs-skills-stewardship-in-a-single-system.md", "text": "https://wpnews.pro/news/the-full-stack-specs-skills-stewardship-in-a-single-system.txt", "jsonld": "https://wpnews.pro/news/the-full-stack-specs-skills-stewardship-in-a-single-system.jsonld"}}