Google issued version 1.0 of Modern Web Guidance, which is a free skill set that web development teams may use as an enhancement to their AI coding tools, in order for those tools to write code based on current browser features. The skills are in the form of packaged guidelines that the agent will look at while writing the code. The code generated by the agent will also be sent to the same pages that are evaluated for metrics such as Largest Contentful Paint (LCP), Interaction to Next Paint (INP), and accessibility.
Philip Walton, a senior staff developer relations engineer at Google, announced the release in a Chrome for Developers post. Version 1.0 adds an audit for existing code, and Firefox and Edge are joining the project. Chrome first released the guidance as an early preview at Google I/O in May.
Teams can install it by running npx modern-web-guidance@latest install in a project directory. It’s also available through the plugin marketplaces of most popular coding agents, according to the post.
What’s New In Version 1.0 #
Version 1.0’s audit will go through a project’s HTML, CSS, JavaScript files as well as its package.json to identify areas where developers are able to safely update their code, then walk the agent through the changes. This first iteration of the audit uses static analysis to look at a limited number of code patterns, according to the post.
The post says the library now targets “hundreds of verified AI knowledge gaps” across its guides. Additionally, current coding models continue to fall short in Google’s tests of accessibility, security headers, and priority for scripts and images.
According to Google, each of the guides is now reviewed and approved by a domain tech lead. For user-facing feature guides there is a dedicated accessibility review done by experts who use assistive technologies daily.
A governance document was uploaded to the project’s GitHub repository on Nov. 9 which outlines six owner seats for the project. Two of those seats belong to Chrome, Microsoft Edge and Mozilla each have one, and two are given to independent web platform experts. There is currently one open independent seat and no seat belongs to Apple.
What Google’s Evals Measure #
Google’s announcement links to a public eval dashboard where coding agents from Anthropic, OpenAI and Google run the same set of tasks every night, with and without the guidance. A real browser then checks the DOM, styles and source code of each finished app.
When I checked Tuesday morning, guided runs were passing 88% of the automated checks, whereas unguided runs were passing 56%. These figures are based upon each agent’s combined results for nightly runs completed Nov. 8; the range is 199 to 215 tasks per agent. The dashboard provides a limited review (not a full WCAG audit, it says) of accessibility features such as support for keyboards, focus, and form labels.
Its task prompts don’t mention the guidance, so each agent has to work out on its own when a guide applies. In most of the recent suites listed, guided runs ran slower and burned more tokens.
Accessibility consultant Adrian Roselli evaluated the preview in May by giving Antigravity (Google’s AI coding tool) an animated accordion prompt found in the documentation. However, according to Roselli, the output did not comply with WCAG and its animation was not functional in Firefox. He also found no assurance that agents would be able to follow the guidelines. This was for the preview, not v1.0.
What Early Users Reported #
The new guidelines were tested by Japan’s creator platform, note inc. using two coding agents that ran through six user interface (UI) tasks across 180 test runs on clone versions of note inc.’s production code base. As explained in a Chrome blog, both agents’ scores on 19 browser checks rose after installing the skill without naming it in prompts, then again once prompts named it. Agent One went from 60.8% to 76.5% to 94.0%. Similarly, Agent Two had scores of 75.8%, 86.2%, and 90.0% under the same three test conditions.
One of note’s tasks checked whether the largest image in view on an article page had high priority and wasn’t lazy-loaded. The comparison table in the post shows that unguided agents wrote hero image code using both high priority and lazy .
The guidance is run by MetaMask, as part of an automatic review process for each pull request submitted to its open-source browser extension, according to a second Chrome post. Through Oct. 21 (the first five weeks), the bot identified 102 issues across 57 feature and fix pull requests. The engineers fully resolved 43 of them and partially resolved three of them. Another 16 were either declined or false positives. Twenty of them went unaddressed. The outcome of the other 20 is unknown.
Why This Matters #
Modern Web Guidance provides a no-cost means for teams to encourage agents writing code to use native browser capabilities and that includes code in existing legacy code bases. Most of the results that have been published are based on testing conducted by Google as well as customer testing that Google highlighted. Note Inc. tested the two agents using the same prompt with and without telling them to use the guidance; they both performed better when the prompt did tell them to use the guidance. The team can also label their prompts to include the guidance and keep field data and manual accessibility testing in the review.
Looking Ahead #
Google says it plans to expand this process of auditing existing code at some point, as well as performing audits in which the agent is reasoning about the actual code being audited.
The company suggests pairing the guidance with Chrome DevTools for agents to allow the agent to run its own created code and test it inside of the browser. Preview users can use the command npx modern-web-guidance@latest update.
Featured Image: Accogliente Design/Shutterstock
Source: Google.