{"slug": "verifying-that-an-approved-correction-reaches-the-next-agent-run", "title": "Verifying that an approved correction reaches the next agent run", "summary": "A developer working on Areev published a fixture-based integration test showing that an approved correction persists across agent runs and changes later output. In the invoice-to-accounting Python example, a user's fix of \"Cobolt Cloud\" to \"Cobalt Cloud\" is saved as an `mg:alias_of` relation scoped to the client's vendor namespace, then retrieved by an `extract_ctx` saved query so a later run resolves the misspelling to the canonical vendor and auto-approves it. The smoke and improve scripts passed against the published wheel, though the author notes the tests use synthetic fixtures and do not validate extraction accuracy on real invoices.", "body_md": "Disclosure: I work on Areev. This article was written by AI agents and checked against the public source and executed fixture tests.\n\nA corrected answer isn’t guaranteed to influence future agent runs. A meaningful integration test must trace the correction from approval, through persistent storage, into the next run’s context, and finally into the output.\n\nThis path is demonstrated in the public Areev invoice-to-accounting Python example. The example uses synthetic fixtures and mock integrations; it is not a live accounting deployment or evidence of model learning.\n\nThe test fixture starts with a misspelled vendor: “Cobolt Cloud.” A user corrects it to “Cobalt Cloud” and approves the change. In the reply handler, the alias is only saved when the decision is approval, the state is marked as revised, and the corrected vendor differs from the original input. A correction message alone does not create the alias.\n\nThe resulting relation is `mg:alias_of`, scoped to the client’s vendor namespace. This ensures the correction persists independently of the conversation event that triggered it. Source: [reply handler](https://github.com/AreevAI/areev/blob/64db3302e5df85fddead7bc7fc7c80f1775eee99/examples/agents/invoice-to-accounting/python/agent.py#L448-L455).\n\nThe trigger uses an `extract_ctx` saved query. Its assembly includes instructions, organizational facts, and recent thread events. The extraction tool processes the provided grains and constructs an alias lookup:\n\n```\naliases = {g[\"subject\"]: g[\"object\"] for g in grains\n           if g.get(\"relation\") == \"mg:alias_of\"}\nif vendor in aliases:\n    vendor = aliases[vendor]\n    confidence = max(confidence, 0.95)\n```\n\nThis fixture extractor uses predefined fields and does not invoke a model. Increasing confidence after an alias match is a defined rule in this example, not a general assertion that known vendor names justify automatic approval. Other fields and the workflow’s validation steps remain relevant. A production system must define its own validation and approval logic.\n\nSource: [extraction and context assembly](https://github.com/AreevAI/areev/blob/64db3302e5df85fddead7bc7fc7c80f1775eee99/examples/agents/invoice-to-accounting/python/agent.py).\n\nWith Git, Bash, and Python available, initialize a fresh directory and virtual environment:\n\n```\ngit clone https://github.com/AreevAI/areev.git\ncd areev\ngit checkout 64db3302e5df85fddead7bc7fc7c80f1775eee99\npython3 -m venv .venv\n. .venv/bin/activate\npython3 -m pip install areev\ncd examples/agents/invoice-to-accounting\nbash python/smoke.sh\nbash python/improve.sh\n```\n\nPackage installation requires network access; the fixture walkthrough does not need a model key or live accounting credentials. These commands use the published Python package, so record the installed version when comparing runs.\n\nThe smoke script runs the revision and subsequent approval. The improve script processes later fixtures and verifies that the repeated misspelling results in the canonical vendor and an automatic approval in this controlled scenario:\n\n```\nassert again[0][\"vendor\"] == \"Cobalt Cloud\", again[0]\nassert again[0][\"approved_by\"] == \"auto\", \\\n    \"it should not have needed a person this time: %r\" % again[0]\n```\n\nSource: [later-run assertions](https://github.com/AreevAI/areev/blob/64db3302e5df85fddead7bc7fc7c80f1775eee99/examples/agents/invoice-to-accounting/improve.sh#L40-L56). The smoke and improve scripts passed in an isolated environment using the published wheel during the article’s preparation.\n\nThe tests confirm that the saved alias persists across runs and affects output. They do not validate extraction accuracy on real invoices. For other workflows, test the same boundaries: what triggers persistence, which context query retrieves the stored data, which step uses it, and which output changes result. A memory write can succeed while a later boundary fails.", "url": "https://wpnews.pro/news/verifying-that-an-approved-correction-reaches-the-next-agent-run", "canonical_source": "https://dev.to/sathish_palanisamy/verifying-that-an-approved-correction-reaches-the-next-agent-run-14ci", "published_at": "2026-10-21 19:56:02+00:00", "updated_at": "2026-10-21 20:11:31.400280+00:00", "lang": "en", "topics": ["ai-agents", "artificial-intelligence", "ai-tools"], "entities": ["Areev", "AreevAI", "GitHub", "Python", "Cobalt Cloud", "Cobolt Cloud"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/verifying-that-an-approved-correction-reaches-the-next-agent-run", "markdown": "https://wpnews.pro/news/verifying-that-an-approved-correction-reaches-the-next-agent-run.md", "text": "https://wpnews.pro/news/verifying-that-an-approved-correction-reaches-the-next-agent-run.txt", "jsonld": "https://wpnews.pro/news/verifying-that-an-approved-correction-reaches-the-next-agent-run.jsonld"}}