{"slug": "what-broke-when-we-turned-government-open-data-into-agent-callable-evidence", "title": "What Broke When We Turned Government Open Data Into Agent-Callable Evidence", "summary": "CivicDataForge, a project by an unnamed developer, has built an architecture to turn government open data into reliable evidence for AI agents and software systems. The project addresses failure modes such as pagination limits, ambiguous absence of records, and stale data, using a fail-closed decision vocabulary and an evidence envelope that preserves lineage to official sources. The developer emphasizes that APIs should not manufacture certainty and that downstream policies should map evidence states to actions.", "body_md": "Government open data looks deceptively easy from a distance.\n\nFind an endpoint. Make a request. Normalize the JSON. Ship an API.\n\nThat is enough for a demo. It is not enough for a production system that has to answer a consequential question next month, after the publisher changes a field, caps a response, moves an endpoint, or returns an empty page that looks exactly like “there are no records.”\n\nWe have been building CivicDataForge around one question:\n\nWhat has to exist between an official government publisher and a software system—or an AI agent—before the result deserves to be called evidence?\n\nThis post is the practical answer we have reached so far. It is not a claim that government data can be made perfectly clean. It is an account of the failure modes we hit, the architecture we adopted, and the boundaries we refuse to erase.\n\nThe first serious failure mode is pagination.\n\nGovernment-data platforms often impose a maximum record count. A response can be valid JSON, have a `200`\n\nstatus, and still represent only the first page.\n\nArcGIS feature services expose controls such as `resultOffset`\n\nand `resultRecordCount`\n\n, and may signal that a transfer limit was exceeded. Socrata datasets similarly support paging and query controls. If a collector ignores those contracts, “the source returned 1,000 rows” can quietly become “the jurisdiction has 1,000 records.”\n\nThose are not the same statement.\n\nOur collection rule became:\n\nThe important product decision is step four. A partial page is useful for exploration, but it cannot prove that a record disappeared between two runs.\n\nOfficial references:\n\nSuppose an address lookup returns no permit record.\n\nIt is tempting to return `NOT_PERMITTED`\n\nor `ILLEGAL`\n\n. That is usually stronger than the evidence.\n\nThe selected publisher may not cover the full legal regime. The address may be formatted differently. A local permit may exist in another system. The source may be stale. The query may be incomplete. Or there may simply be no published match.\n\nWe use an explicit, fail-closed decision vocabulary:\n\n```\nEVIDENCE_FOUND\nNO_PUBLISHED_MATCH\nREVIEW_REQUIRED\nSOURCE_UNAVAILABLE\nSCOPE_INCOMPLETE\n```\n\nThis vocabulary is deliberately less dramatic than a binary yes/no result. That is the point.\n\nAn API should not manufacture certainty because a consumer wants a Boolean. If a downstream workflow requires a Boolean, the customer should own the policy that maps evidence states to an action—and should preserve the original state for review.\n\nNormalization is necessary. It is also where evidence can lose its identity.\n\nWe normalize names, addresses, dates, identifiers, and source-specific statuses, but retain enough lineage to answer:\n\nA minimal evidence envelope looks roughly like this:\n\n```\n{\n  \"source\": {\n    \"publisher\": \"issuing authority\",\n    \"url\": \"https://official.example/dataset\",\n    \"retrieved_at\": \"2026-08-24T12:00:00Z\"\n  },\n  \"scope\": {\n    \"query\": \"bounded source query\",\n    \"complete\": true\n  },\n  \"record\": {\n    \"source_id\": \"publisher-owned-key\",\n    \"normalized_identifier\": \"stable-identifier\",\n    \"record_hash\": \"sha256:...\"\n  },\n  \"decision\": {\n    \"state\": \"EVIDENCE_FOUND\",\n    \"receipt_hash\": \"sha256:...\"\n  }\n}\n```\n\nThe exact fields vary by source. The invariant is that the normalized result does not sever the path back to the official record.\n\nA source can be reachable and stale.\n\nIt can have a stable schema and return incomplete data. It can publish a recent file whose internal dates are old. It can change content without changing a URL. It can change a schema without failing a request.\n\nSo one green check is not enough.\n\nWe monitor independent dimensions:\n\nWhen freshness cannot be measured, we say `freshness_not_measured`\n\n. We do not let an availability check imply that the data is current.\n\nThis distinction matters because a production monitor should be able to quarantine a bad update instead of replacing a known baseline with an empty or damaged response.\n\n“Send a webhook when something changes” is only the visible edge of a reliable change system.\n\nThe harder questions are:\n\nApify webhooks retry failed deliveries. That is helpful, but it means consumers must be idempotent. Our recommended key is the Actor run ID plus event type, with the downstream handler acknowledging quickly and queueing the expensive work.\n\n```\nidempotency_key = actor_run_id + \":\" + event_type\n```\n\nWe treat a webhook as a notification about an evidence object, not as the evidence object itself. The durable dataset, record hashes, scope, and decision receipt remain the inspectable source of truth.\n\nOfficial reference: [Apify webhook actions and retry behavior](https://docs.apify.com/integrations/webhooks/actions).\n\nPublishing an MCP server does not automatically make a tool safe or useful.\n\nAn agent still needs to know:\n\nWe expose a stable Streamable HTTP MCP endpoint and machine-readable catalogs, but the more important artifact is the task-to-tool selection contract.\n\nFor example:\n\n```\n{\n  \"task\": \"verify India company evidence\",\n  \"preferred_identifier\": \"Corporate Identification Number\",\n  \"minimum_input\": \"exact CIN preferred\",\n  \"boundary\": \"research evidence, not automatic KYC or eligibility clearance\"\n}\n```\n\nThe transport gets the tool into the agent. The selection contract helps the agent call the right tool and preserve the result's meaning.\n\nThe MCP specification recommends Streamable HTTP for remote servers and defines the transport behavior. Authentication, origin validation, and secret handling still belong to the implementation and client configuration—not to public examples with embedded credentials.\n\nOfficial reference: [MCP transports specification](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports).\n\nThe architecture now has six explicit services:\n\nThat framing changes what we build.\n\nA new source does not enter production because we found an endpoint. It needs a source and rights passport, a completeness test, a normalization contract, health probes, a bounded decision contract, and an acceptance-tested delivery path.\n\nLikewise, a connector is not “available” because we can imagine it. Public APIs, datasets, schedules, webhooks, file exports, and the remote MCP endpoint are live interfaces. A customer-specific adapter, SFTP route, or platform-native app remains scoped work until its real downstream path passes an acceptance test.\n\nThe [service catalog](https://civicdataforge.pages.dev/service-catalog.json) and [integration recipes](https://civicdataforge.pages.dev/integration-recipes) publish that boundary in human- and machine-readable form.\n\nCivicDataForge exists to reduce the hours lost between “the government publishes this somewhere” and “a person or system can use it without pretending it says more than it does.”\n\nOur intention is to build a reusable evidence layer across public records: official-source first, inspectable, correction-friendly, and callable by both software and AI agents.\n\nOur want is straightforward: make source quality and uncertainty visible enough that developers do not have to choose between a brittle one-off script and an opaque vendor answer.\n\nWe are open to collaboration and grateful for serious support, correction, and feedback. We are not asking for permission to build; we are inviting developers, public servants, data owners, researchers, and AI teams to make the contracts stronger with us.\n\nWhat we need most from the developer community is adversarial input:\n\nThose questions are more valuable than generic feature requests because they expose the real contract the system has to survive. If you bring one, we will meet it with respect, evidence, and a willingness to change what the facts prove should change.\n\nIf we started again, we would make three decisions earlier:\n\nThe code that fetches rows is often the smallest part of the system. The defensible work is keeping a source-linked statement truthful after paging, normalization, retries, source drift, and downstream automation.\n\nThat is the layer we are building.", "url": "https://wpnews.pro/news/what-broke-when-we-turned-government-open-data-into-agent-callable-evidence", "canonical_source": "https://dev.to/civicdataforge/what-broke-when-we-turned-government-open-data-into-agent-callable-evidence-1m81", "published_at": "2026-08-24 05:25:11+00:00", "updated_at": "2026-08-24 05:42:53.068643+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "developer-tools"], "entities": ["CivicDataForge", "ArcGIS", "Socrata"], "alternates": {"html": "https://wpnews.pro/news/what-broke-when-we-turned-government-open-data-into-agent-callable-evidence", "markdown": "https://wpnews.pro/news/what-broke-when-we-turned-government-open-data-into-agent-callable-evidence.md", "text": "https://wpnews.pro/news/what-broke-when-we-turned-government-open-data-into-agent-callable-evidence.txt", "jsonld": "https://wpnews.pro/news/what-broke-when-we-turned-government-open-data-into-agent-callable-evidence.jsonld"}}