Content Signals, a proposed extension to robots.txt backed by the IETF AI Preferences working group and the IAB Tech Lab, adds directives to declare whether AI crawlers may search, ingest, or train on…
The W3C Web Machine Learning Community Group is incubating WebMCP, a browser API that lets web pages register structured tools for in-browser AI agents to call directly via a JavaScript API, eliminati…
Internal linking, using descriptive anchor text and avoiding orphan pages, is the strongest SEO signal site owners control for crawlers and AI agents. Proper implementation involves linking to canonic…
Jeremy Howard proposed /llms.txt, a markdown file served at a site's root to give large language models a curated index of important content. The convention, hosted at llmstxt.org, is not a ratified s…
A new specification proposes that documentation sites expose raw Markdown source at predictable URLs via a .md suffix or content negotiation, enabling AI agents and tools to fetch lossless, smaller pa…
HTML headings must form a nested outline, never skip levels, and never be used for visual styling alone, as they are the only structural signal consistent across browsers, screen readers, search engin…
The Linux Foundation working group released the Agentic Resource Discovery (ARD) specification, a draft discovery layer that allows domains to advertise agent capabilities via a manifest at /.well-kno…
The Open Knowledge Format (OKF) bundle is a new convention for packaging knowledge bases as a tree of Markdown files with typed front matter, enabling AI agents to ingest an entire corpus in a single …
Websites should announce RSS, Atom, or JSON feeds using <link rel="alternate"> in the <head> to enable automatic discovery by feed readers, browsers, and AI agents. The practice improves subscription …
JSON-LD structured data using schema.org vocabulary is the standard for machine-readable page annotations, enabling search engines to display rich results and AI agents to extract factual ground truth…
The W3C Web Application Security Working Group has standardized the /.well-known/change-password URL as a redirect endpoint that points password managers and users to a site's actual change-password p…
The Model Context Protocol (MCP), an open standard proposed by Anthropic in late 2024, enables websites to expose queryable tools directly to AI agents via JSON-RPC, replacing UI scraping with structu…
The HTML `hidden="until-found"` attribute allows developers to hide collapsible content while keeping it accessible to browser find-in-page, assistive technologies, and search engines. When a user sea…
A new website specification, "What a good website does," has been published as a platform-agnostic technical standard covering ten categories including SEO, accessibility, security, and agent readines…