{"slug": "show-hn-stripeworld-a-synthetic-stripe-api-for-rl-and-testing", "title": "Show HN: StripeWorld – a synthetic Stripe API for RL and testing", "summary": "Kiln-AI released StripeWorld, a stateful synthetic copy of Stripe's Billing and Payments API built on its Seahaven framework, covering 155 Stripe API operations across 25 tables pinned to API version 2026-08-26.dahlia. StripeWorld runs hundreds of private Stripe accounts in one process, each starting from a chosen fixture with controlled clock, ids and randomness, and logs every row a run changes so agent evals, RL and tests can be graded on outcomes rather than transcripts. Unlike Stripe test mode, which shares one account, or stripe-mock, which keeps no state, StripeWorld is stateful across calls and speaks Stripe's HTTP API and MCP tools, though it omits webhook delivery and covers only Billing and Payments core.", "body_md": "### \n  A stateful Stripe sandbox for agent evals, RL and tests\n\n  Built with [Seahaven](https://github.com/Kiln-AI/Seahaven).\n\n[**Quick Start**](#getting-started) •\n  [**Examples**](#examples) •\n  **Built with Seahaven**\n\nAgents that use Stripe need somewhere to practise. Test suites need somewhere to charge cards. Real Stripe is one shared account you can't reset, and a mock forgets every write the moment it answers.\n\nStripe World is a stateful copy of Stripe's Billing and Payments API. Run a hundred private Stripe accounts in one process, each starting from a state you chose, with every row each one changed logged for you to check.\n\n**Stripe World is a Seahaven demo.** It was built to show what a [Seahaven](https://github.com/Kiln-AI/Seahaven) world can be. If you want a realistic copy of *your* system, for your agents or your tests, that's what Seahaven is for.\n\n**Not affiliated with Stripe.** Stripe's field names, enum values and id prefixes are reused as API\nvocabulary under the MIT licence of Stripe's OpenAPI spec; see\n[`THIRD_PARTY_LICENSES.md`](https://github.com/Kiln-AI/stripe_world/blob/main/THIRD_PARTY_LICENSES.md).\n\n[Try it on Hugging Face Spaces →](https://huggingface.co/spaces/scosman/stripe_world)\n\n- **155 Stripe API operations across 25 tables** , pinned to API version`2026-08-26.dahlia` . Every\nobject shape aligned to Stripe's OpenAPI spec.\n- **A billing engine, not just CRUD.** Subscriptions move through all eight statuses. Invoices go\nfrom draft to open to paid, void or uncollectible. Proration, dunning, credit notes and a balance\nledger behave the way Stripe's do, and Stripe's test cards (`tok_visa` ,`tok_visa_chargeDeclinedInsufficientFunds` , …) succeed, decline and dispute on cue.\n- **Stripe's own MCP tools.** Agents get the same tools as Stripe's MCP server, so an agent that works\nhere works against the real thing.\n- **Stripe's own API/SDK.** It also speaks Stripe's HTTP API. Point an official Stripe SDK at it and\nyour code runs unchanged, apart from webhooks.\n- **Checked against the real API.** recordings of real Stripe API sessions are replayed against\nit in CI, and every object it returns is validated against Stripe's OpenAPI spec.\n\n|  | Stripe World | Stripe test mode | stripe-mock | \n|---|---|---|---|\n| Stripe's real object shapes | ✅ | ✅ | ✅ | \n| Stateful across calls | ✅ | ✅ | ❌ | \n| Billing behaviour: subscriptions, invoices, declines | ✅ | ✅ | ❌ | \n| A private account for every run | ✅ | ❌ | ✅ | \n| Hundreds of runs in parallel | ✅ | ❌ rate-limited | ✅ | \n| Every run starts from a known state | ✅ | ❌ | ✅ | \n| Reproducible: same ids, same timestamps | ✅ | ❌ | ✅ | \n| Every change logged for grading | ✅ | events only | ❌ | \n| No API key or network needed | ✅ | ❌ | ✅ | \n| Webhook delivery | ❌ events are queryable | ✅ | ❌ | \n| All of Stripe's API | ❌ Billing and Payments core | ✅ | ✅ | \n\n[stripe-mock](https://github.com/stripe/stripe-mock) is Stripe's own mock server. It answers every\nendpoint with a valid, canned response but keeps no state: a customer you create is not there when\nyou list customers. Stripe test mode is the real thing, but every run shares one account,\nand your code can't reset it.\n\nStripe World is built on [Seahaven](https://github.com/Kiln-AI/Seahaven), a framework for building\nsynthetic worlds for agent evals, RL and testing. Seahaven provides:\n\n- **Instances in milliseconds.** Every run gets a private copy of a frozen starting state, a*fixture* , with its own SQLite database.\n- **Reproducibility.** The clock, ids and randomness are controlled: the same fixture and seed\nreplay the same run.\n- **A change log.** Every row a run changed is recorded, so you grade the outcome, not the\ntranscript.\n- **Serving.** OpenEnv for eval and RL harnesses and HTTP for software tests, each hosting hundreds\nof instances in one process; MCP for chat apps; and a web console to drive a world by hand.\n\nStripe World supplies only what is specific to Stripe: the tables, the API operations and the billing rules.\n\nEach example runs from a checkout of this repository; [Getting started](#getting-started) has the\nthree commands that make one.\n\n`seahaven serve` hosts Stripe World as an [OpenEnv](https://github.com/huggingface/OpenEnv)\nenvironment, the open standard for RL environments. Every connection gets its own private Stripe\naccount. Open `http://127.0.0.1:8000/console` to drive one by hand.\n\n```\nuv run --extra serve seahaven serve\n```\n\nThis script, `rollouts.py`, runs 100 episodes at once against that server. Each starts from the same fixture and\nis graded on what changed in its account:\n\n``` python\nimport asyncio\n\nfrom seahaven.openenv import SeahavenClient\n\nTASK = \"Sign up jenny@example.com as a customer.\"\n\nasync def run_agent(env, tools, task):\n    # Your agent goes here: hand `tools` to your model and run its tool calls with `env.call`.\n    # This stand-in makes the two calls a good agent would.\n    account = (await env.call(\"list_available_accounts_or_orgs\")).result[\"accounts\"][0]\n    await env.call(\n        \"stripe_api_write\",\n        stripe_api_operation_id=\"PostCustomers\",\n        parameters={\"email\": \"jenny@example.com\"},\n        stripe_context=account[\"stripe_context\"],\n        livemode=account[\"livemode\"],\n    )\n\ndef grade(state) -> float:\n    # Grade what changed in the account, not what the agent said it did.\n    rows = state.state[\"db\"][\"log\"]\n    signed_up = any(\n        row[\"table\"] == \"customers\"\n        and row[\"op\"] == \"insert\"\n        and row[\"after\"][\"email\"] == \"jenny@example.com\"\n        for row in rows\n    )\n    return 1.0 if signed_up else 0.0\n\nasync def episode(seed: int) -> float:\n    async with SeahavenClient(base_url=\"http://127.0.0.1:8000\") as env:\n        # A private Stripe account, copied from the `empty` fixture.\n        await env.reset(fixture=\"empty\", seed=seed)\n        tools = await env.list_tools()  # the eight Stripe MCP tools, as JSON schemas\n        await run_agent(env, tools, TASK)\n        return grade(await env.state())\n\nasync def main():\n    rewards = await asyncio.gather(*(episode(seed) for seed in range(100)))\n    print(f\"{len(rewards)} episodes, mean reward {sum(rewards) / len(rewards)}\")\n\nasyncio.run(main())\nuv run --extra serve python rollouts.py\n```\n\n[Kiln](https://kiln.tech) connects to the same server as an OpenEnv client.\n\nTo use the hosted copy instead of a local server, see\n[Connecting to the published world](#connecting-to-the-published-world).\n\n`serve_http.py` serves Stripe's HTTP API. Every `/worlds/<id>` is its own Stripe account, created\nby the first request that uses it, so every test can have a fresh one. It holds 100 accounts at\nonce by default (`--max-instances N`, `0` for no limit), so the fixture below deletes each one\nwhen its test ends.\n\n```\nuv run serve_http.py --reset-options '{\"startup\": {\"livemode\": false}}'\n```\n\nSave this as `test_billing.py` and run `uv run pytest test_billing.py`:\n\n``` python\nimport uuid\n\nimport httpx\nimport pytest\nimport stripe\n\nSERVER = \"http://127.0.0.1:8000\"\n\n@pytest.fixture\ndef stripe_client():\n    # Every test gets its own private Stripe account, created on first use.\n    account = f\"{SERVER}/worlds/{uuid.uuid4().hex}\"\n    yield stripe.StripeClient(\"sk_test_123\", base_addresses={\"api\": account})\n    httpx.delete(account)  # and throws it away afterwards\n\ndef test_subscription_charges_the_card(stripe_client):\n    customer = stripe_client.v1.customers.create(params={\"email\": \"jenny@example.com\"})\n    card = stripe_client.v1.payment_methods.create(\n        params={\"type\": \"card\", \"card\": {\"token\": \"tok_visa\"}}  # Stripe's test cards work\n    )\n    stripe_client.v1.payment_methods.attach(card.id, params={\"customer\": customer.id})\n    price = stripe_client.v1.prices.create(\n        params={\n            \"currency\": \"usd\",\n            \"unit_amount\": 2000,\n            \"recurring\": {\"interval\": \"month\"},\n            \"product_data\": {\"name\": \"Pro plan\"},\n        }\n    )\n\n    subscription = stripe_client.v1.subscriptions.create(\n        params={\n            \"customer\": customer.id,\n            \"items\": [{\"price\": price.id}],\n            \"default_payment_method\": card.id,\n            \"expand\": [\"latest_invoice\"],\n        }\n    )\n\n    assert subscription.status == \"active\"\n    assert subscription.latest_invoice.status == \"paid\"\n    assert subscription.latest_invoice.amount_paid == 2000\n\ndef test_declined_card(stripe_client):\n    card = stripe_client.v1.payment_methods.create(\n        params={\"type\": \"card\", \"card\": {\"token\": \"tok_visa_chargeDeclinedInsufficientFunds\"}}\n    )\n\n    with pytest.raises(stripe.CardError) as declined:\n        stripe_client.v1.payment_intents.create(\n            params={\n                \"amount\": 2000,\n                \"currency\": \"usd\",\n                \"payment_method\": card.id,\n                \"confirm\": True,\n            }\n        )\n\n    assert declined.value.http_status == 402\n    assert declined.value.code == \"card_declined\"\n    assert declined.value.user_message == \"Your card has insufficient funds.\"\n    assert declined.value.error.decline_code == \"insufficient_funds\"\n```\n\nThe second test is failure injection: Stripe's test cards decline here exactly as they do in test\nmode, so you can test the unhappy paths that are hard to reach against the real API. Any Stripe SDK\nworks the same way: point its API base at `http://127.0.0.1:8000/worlds/<id>`, with any key.\n\n`seahaven mcp` connects Stripe World to Claude Desktop, or any MCP client, so you can work with it\nby hand. Add this to `claude_desktop_config.json`, with the path to your checkout:\n\n```\n{\n  \"mcpServers\": {\n    \"stripe-world\": {\n      \"command\": \"uv\",\n      \"args\": [\n        \"run\", \"--directory\", \"/path/to/stripe_world\", \"--extra\", \"mcp\",\n        \"seahaven\", \"mcp\", \"--reset-options\", \"{\\\"startup\\\": {\\\"livemode\\\": false}}\"\n      ]\n    }\n  }\n}\n```\n\nThen ask Claude things like:\n\n- *\"Set up a Pro plan at $20 a month, and sign up [jenny@example.com](mailto:jenny@example.com) on it, invoiced monthly.\"*\n- *\"Give Jenny 20% off for three months.\"*\n- *\"Which invoices haven't been paid yet?\"*\n\nClaude gets a fresh test-mode account each time it starts the server, and nothing is kept when it stops. The world provides 8 of the 10 tools on Stripe's MCP server; the documentation search, implementation planner and feedback tools are not reproduced.\n\nStripe World is published as the Hugging Face Space\n[scosman/stripe_world](https://huggingface.co/spaces/scosman/stripe_world). Connect to it with\nOpenEnv's generic client. Your side needs the `openenv` package (`pip install openenv`) and nothing\nfrom this repository:\n\n``` python\nfrom openenv import AutoEnv\n\nwith AutoEnv.from_hub(\"scosman/stripe_world\", skip_install=True) as env:\n    env.reset(seed=7, startup={\"livemode\": False})\n    tools = env.step({\"type\": \"list_tools\"}).observation[\"tools\"]\n    account = env.step(\n        {\"type\": \"call_tool\", \"tool_name\": \"list_available_accounts_or_orgs\", \"arguments\": {}}\n    ).observation[\"result\"][\"accounts\"][0]\n    created = env.step(\n        {\n            \"type\": \"call_tool\",\n            \"tool_name\": \"stripe_api_write\",\n            \"arguments\": {\n                \"stripe_api_operation_id\": \"PostCustomers\",\n                \"parameters\": {\"email\": \"jenny@example.com\"},\n                \"stripe_context\": account[\"stripe_context\"],\n                \"livemode\": account[\"livemode\"],\n            },\n        }\n    ).observation\n    print(created[\"result\"], env.state())\n```\n\nAlways pass `skip_install=True`. Without it, `from_hub` tries to install this repository as a\nPython package, and that fails. For a server you run yourself, add `base_url=\"http://host:8000\"`.\n\nA fixture is a frozen Stripe account that every run starts from. Build one with the same tools an agent uses, and freeze it:\n\n``` python\nfrom seahaven_stripe_world import world\n\nTEST_MODE = {\"livemode\": False}\n\ndef stripe_write(inst, operation, **parameters):\n    account = inst.call(\"list_available_accounts_or_orgs\")[\"accounts\"][0]\n    return inst.call(\n        \"stripe_api_write\",\n        stripe_api_operation_id=operation,\n        parameters=parameters,\n        stripe_context=account[\"stripe_context\"],\n        livemode=account[\"livemode\"],\n    )\n\nwith world.instance(now=\"2026-09-01T14:00:00.000Z\", clock_mode=\"fixed\", startup=TEST_MODE) as inst:\n    pro = stripe_write(\n        inst,\n        \"PostPrices\",\n        currency=\"usd\",\n        unit_amount=2000,\n        recurring={\"interval\": \"month\"},\n        product_data={\"name\": \"Pro plan\"},\n    )\n    for n in range(3):\n        customer = stripe_write(inst, \"PostCustomers\", email=f\"customer{n}@example.com\")\n        stripe_write(\n            inst,\n            \"PostSubscriptions\",\n            customer=customer[\"id\"],\n            items=[{\"price\": pro[\"id\"]}],\n            collection_method=\"send_invoice\",\n            days_until_due=30,\n        )\n    inst.freeze(\"pro_plan\", \"Three customers on the $20/month Pro plan, each with a draft invoice.\")\n```\n\nThen start every run from it. Each gets a private copy in milliseconds, and the same seed replays the same ids and timestamps:\n\n```\nfor seed in range(100):\n    with world.instance(\"pro_plan\", seed=seed, clock_mode=\"tick\", startup=TEST_MODE) as inst:\n        run_agent(inst)  # your agent\n        reward = grade(inst.state())  # a dict; the rows it changed are in [\"state\"][\"db\"][\"log\"]\n```\n\nThe servers take it too: `env.reset(fixture=\"pro_plan\", ...)` over OpenEnv, and\n`uv run serve_http.py --reset-options '{\"fixture\": \"pro_plan\", \"startup\": {\"livemode\": false}}'`\nfor the HTTP API. Commit the script that builds a fixture alongside it, so it can be rebuilt.\nTwo ready-made fixtures with real history ship beside `empty`, both test-mode accounts:\n`small_co` (Wrenfield Labs, 45 hand-written customers) and `big_co` (Ostrander Systems, about 380\nsimulated business customers over 18 months). [docs/fixtures.md](https://github.com/Kiln-AI/stripe_world/blob/main/docs/fixtures.md) describes each\ncompany and its loose ends.\n\n**Resources:** customers, payment methods, products, prices, coupons, promotion codes, tax rates,\npayment intents, charges, refunds, disputes, setup intents, balance, balance transactions,\npayouts, subscriptions, subscription items, subscription schedules, invoices, invoice items, credit\nnotes, customer balance transactions and events.\n\n**Billing engine:**\n\n- **Subscriptions:** all eight statuses and their transitions, trials, cancel now or at period end,\npause, and incomplete expiry.\n- **Invoices:** draft, open, paid, void and uncollectible, with auto-advance and both collection\nmethods.\n- **Proration:** credit and debit lines, rounded the way Stripe rounds them.\n- **Dunning:** retry settings, end-of-schedule outcomes and hard declines.\n- **Credit notes:** settled by refund, customer balance or out of band.\n- **Balance ledger:** a balance transaction for every money movement, with fees, pending and\navailable funds, and payouts.\n- **Test cards:** Stripe's test card numbers and tokens for declines, disputes and 3D Secure.\n\n**API behaviour:** search on seven resources with Stripe's query language, idempotency keys,\ncursor pagination, `expand[]`, and an event for every state change.\n\n```\ngit clone https://github.com/Kiln-AI/stripe_world.git\ncd stripe_world\nuv sync\n```\n\nThen run any example above from the checkout. To work on the world itself, run the checks CI runs:\n\n```\nuv run ruff format --check && uv run ruff check\nuv run ty check\nuv run pytest\nuv run seahaven check\n```\n\nThe [technical reference](https://github.com/Kiln-AI/stripe_world/blob/main/docs/technical.md) covers the agent surface, coverage, conformance,\nconventions and project layout. Coding agents start at [AGENTS.md](https://github.com/Kiln-AI/stripe_world/blob/main/AGENTS.md).\n\nStripe World is released under the [MIT License](https://github.com/Kiln-AI/stripe_world/blob/main/LICENSE), the same licence as Seahaven.\n\nStripe World is not affiliated with Stripe. Stripe's API vocabulary is used under the MIT licence of\nits OpenAPI spec; see [`THIRD_PARTY_LICENSES.md`](https://github.com/Kiln-AI/stripe_world/blob/main/THIRD_PARTY_LICENSES.md).\n\nStripe World and Seahaven are built by the team behind [Kiln](https://kiln.tech), a free app and\nopen-source library for building better AI products. Kiln connects to any Seahaven world: write\nscenarios against a fixture, [evaluate](https://kiln.tech/features/evals) your agent on the state\nit leaves behind, then [auto-optimize](https://kiln.tech/features/auto-optimize) prompts and models\nagainst those evals.", "url": "https://wpnews.pro/news/show-hn-stripeworld-a-synthetic-stripe-api-for-rl-and-testing", "canonical_source": "https://github.com/Kiln-AI/stripe_world", "published_at": "2026-11-20 14:05:46+00:00", "updated_at": "2026-11-20 14:15:22.383629+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "agent-protocols", "developer-tools", "ai-startups"], "entities": ["StripeWorld", "Stripe", "Seahaven", "Kiln-AI", "stripe-mock", "Hugging Face Spaces"], "also_reported_by": [], "takeaway": "StripeWorld gives agents and tests a stateful Stripe Billing and Payments sandbox with 155 API operations across 25 tables, reproducible ids and timestamps per fixture and seed, and a per-row change log for grading, but no webhook delivery.", "recommendation": null, "alternates": {"html": "https://wpnews.pro/news/show-hn-stripeworld-a-synthetic-stripe-api-for-rl-and-testing", "markdown": "https://wpnews.pro/news/show-hn-stripeworld-a-synthetic-stripe-api-for-rl-and-testing.md", "text": "https://wpnews.pro/news/show-hn-stripeworld-a-synthetic-stripe-api-for-rl-and-testing.txt", "jsonld": "https://wpnews.pro/news/show-hn-stripeworld-a-synthetic-stripe-api-for-rl-and-testing.jsonld"}}