Accessibility is the first-class interface for AI agents

AI agents perform better when using the web's accessibility tree instead of screenshots, reducing latency from 2-5 seconds to under 500ms and cutting token costs by an order of magnitude, according to a developer's analysis. The accessibility tree provides structured machine-readable interfaces, but gaps remain for execution contracts, which emerging standards like WebMCP aim to address. Modern component architectures degrade semantic quality, creating failures for both accessibility and AI agents.

When I started evaluating browser agents, most of the conversation around me focused on multimodal models, computer-use systems and screenshot-based automation. Almost every framework I evaluated assumed agents needed to perceive the web the way humans do, visually, pixel by pixel. The more time I spent shipping agents against real web applications, the more I became convinced we were solving the wrong problem. AI agents would stall on checkout forms because a button had no ARIA role. They would waste seconds and thousands of tokens taking screenshots to figure out what was on the screen. The problem was never the Agent. It was that we kept treating the web as a visual surface, even though it already has a machine-readable interface. We have had one for decades. It is called the accessibility tree. Most developers think of accessibility as a feature for people. Technically, accessibility required the web platform to solve a deeper problem https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA : Exposing interfaces in a machine-readable form. Long before AI agents existed, screen readers were already consuming the web through a structured semantic representation of roles, labels, states and relationships. There was no pixel interpretation and no screenshot. That’s not adjacent to what AI agents need. That is what AI agents need. Long before LLMs existed, assistive technologies proved the core thesis: Machines can navigate interfaces, semantics can outlive presentation and structure can substitute for vision. Screenshot-based agents spend tokens rediscovering facts the browser already knows. The accessibility tree already contains role, name and state in structured form. In my own agent work, switching from screenshot-based to DOM-native execution cut per-action latency from 2–5 seconds to under 500ms and token cost by an order of magnitude. The most clarifying realization I had was this: Accessibility had already solved a large portion of the problem agents face. Accessibility gives machines a way to discover interfaces. It exposes available controls, their names, their states and their relationships. But discovery is not execution. The accessibility tree can identify a button named “Checkout” and indicate whether it is disabled. What it cannot provide is a contract for the action itself. For example, what inputs it accepts, what preconditions are required and what state changes it produces. One emerging response to this gap is WebMCP https://webmachinelearning.github.io/webmcp/ , which introduces a browser-native way to expose typed capabilities that agents can invoke directly. When a form field has no explicit agent annotation, Chrome’s declarative API derives the parameter description from the associated Modern component architectures actively degrade the semantic quality that accessibility and agents both depend on. When a design system wraps a native button in a custom component, what reaches the DOM is often a div with generated class names and no semantic role. The accessibility tree gets “generic” instead of “button.” Under WebMCP’s declarative API, a form field with no label has no parameter description for the browser to inherit. Either way, the agent has nothing to work with. Beyond div soup, virtualized lists https://tanstack.com/virtual/latest only render visible rows, making out-of-viewport content completely unreachable. Client state that updates visually but never updates ARIA attributes leaves agents acting on stale snapshots. The common thread is that accessibility was treated as a concern for human users only, and the semantic layer got quietly destroyed in the abstraction. That’s now a double failure. Humans can tolerate ambiguous UI. Agents cannot. Every point of ambiguity is a probability distribution over possible actions, and probability distributions can produce wrong actions at scale. For frontend teams thinking about this now, there are three places to start. I have started thinking of agent operability as a strict superset of accessibility. Tools like axe-core https://github.com/dequelabs/axe-core already catch a meaningful share of agent failures because they validate the semantic layer agents depend on. The WebMCP team’s proposed Lighthouse audit for the agentic web is the natural next layer. HTML gave us a machine-readable structure. ARIA and the Accessibility Object Model gave us machine-readable meaning. What agents need next is machine-readable capability: Not just what a control is , but what it does , under what conditions and with what effect. Teams that invested in accessibility did not just build more inclusive products. They also built the closest thing to agent-compatible UIs on the web. WebMCP makes that inheritance explicit: Labels become parameter descriptions, ARIA metadata becomes agent metadata and semantic structure becomes the foundation for machine execution. Assistive technologies proved the thesis decades ago: Machines can navigate interfaces, semantics can outlive presentation and structure can substitute for vision. This isn’t a new protocol. It is the completion of work that ARIA and the Accessibility Object Model started – turning machine-readable descriptions into contracts that agents can execute against reliably. . This article is published as part of the Foundry Expert Contributor Network. Want to join? https://www.infoworld.com/expert-contributor-network/