# United Separates Search Indexing From AI Agent Access in New Policy

> Source: <https://dev.to/alifar/united-separates-search-indexing-from-ai-agent-access-in-new-policy-5fbi>
> Published: 2026-11-16 17:45:30+00:00

United Airlines has updated its policies to distinguish between two forms of automated access that are often grouped together in discussions about AI search: indexing public web content and allowing AI agents to act on a user’s behalf.

Under the policy update announced on November 11, 2026, United says unauthorized third-party personal AI agents or assistants are restricted from accessing its digital platforms. At the same time, public pages can remain available for search-engine indexing. Agents that search, book or purchase through united.com or the United app must have written permission, according to reporting on the company’s terms and related travel policies.

The distinction matters because visibility in search results does not automatically mean a site permits automated agents to use its content, retrieve inventory or complete transactions.

In its [November 11 policy announcement](https://united.mediaroom.com/news-releases?item=125496), United said it was restricting access by unauthorized third-party personal AI agents and assistants across its digital platforms, as well as addressing access through travel agencies and aggregators. The airline described updates to its booking, ticketing and distribution policies.

Search Engine Journal’s [review of United’s updated terms](https://www.searchenginejournal.com/united-ai-agents-permission-search-indexing/597418/) identified a more specific split relevant to search professionals:

Those are related policies, but they govern different activities. A traditional crawler can discover and index an accessible public page for search results. An agent that navigates a site, retrieves travel options, submits a request or attempts a purchase is performing a more active form of automated access.

The rise of AI assistants has made the line between crawling, retrieval and action more important. A search engine may crawl public content to create an index. A generative system may retrieve content to answer a question. An AI agent may go further by comparing options, querying availability or attempting to complete a transaction.

For a travel company, the latter can involve dynamic prices, inventory, passenger data, loyalty accounts, booking rules and distribution agreements. United’s policy reflects that these operational and contractual concerns are not identical to being discoverable in search.

The policy also separates visibility from data-use permission. Allowing a page to appear in search results should not be interpreted as consent for its content to be used for AI model training or grounding. United’s terms, as reported by Search Engine Journal, address that distinction directly.

This is especially relevant for organizations trying to improve presence in AI-powered discovery. Content accessibility can support discoverability, but it does not settle whether an external assistant can access a service, reuse its data or execute a workflow.

Search Engine Journal reported that United’s robots.txt did not name AI crawlers as of mid-November 2026. That does not contradict the company’s policy position. Robots.txt is a crawler directive, while terms of use, contracts and distribution policies can set separate conditions for automated access and data use.

A robots.txt file can communicate crawl preferences to compliant bots. It is not, by itself, a complete governance framework for AI systems, agentic tools or commercial data relationships. Conversely, a legal or contractual restriction does not necessarily mean a page is unavailable for conventional search indexing.

For site owners, the practical lesson is to avoid treating a single technical file as the whole AI-access strategy. The relevant rules may be distributed across robots.txt, website terms, API documentation, partner agreements, login controls and product-specific conditions.

United’s approach provides a useful framework for reviewing a company’s own policies. The goal is not necessarily to copy its restrictions. It is to make deliberate distinctions between content that should be discoverable and activities that require controls or permission.

Start by identifying the automated activities your organization permits, restricts or has not yet addressed. These can include:

This exercise can expose policy gaps. A business may welcome search visibility while wanting to limit automated purchases, scraping of dynamic inventory or reuse of proprietary content in AI systems.

SEO teams should compare robots.txt directives with website terms, data-use language, [API rules and agency agreements](https://blog.geosearchleads.com/api-access-website-scans-pdf-reports/). Misalignment can create confusion internally and externally. For example, a site might technically allow broad crawling while its contractual terms are silent about AI data use, or it might restrict agent actions without clearly defining what permission process applies.

Teams should involve legal, product, security and commercial stakeholders where the site includes transactions, sensitive data or partner distribution. An SEO-led crawl policy cannot independently resolve obligations in contracts or regulated workflows.

Public educational, support and destination content may be appropriate for indexing because it helps users find the brand through search. Account areas, real-time availability, checkout flows and other high-risk interactions may warrant stronger access controls or explicit permission requirements.

That separation can preserve organic discoverability while giving the organization clearer governance over automated actions. It also helps content and technical teams explain why an initiative aimed at AI visibility should not be assumed to authorize every type of AI-agent interaction.

The operational impact of United’s policy remains uncertain. The company has not publicly detailed all methods it will use to identify or block automated agents, and the practical effects on agencies and third-party tools will depend on implementation and any permission processes. The announcement is therefore clearer about the policy boundary than about how consistently it will be enforced.

If your site is assessing how AI discovery, crawling and automated actions intersect, [create a GEO Search Leads account](https://app.geosearchleads.com/pages/register) to begin evaluating your website’s GEO readiness and identify areas where [content accessibility and AI-access policies](https://blog.geosearchleads.com/ai-visibility-tool-seo-agencies/) may need closer review. A structured assessment can help distinguish visibility opportunities from workflows that need separate technical, legal or commercial controls.

**Does United still allow search engines to index its public pages?**

Yes. Search Engine Journal reported that United’s updated terms preserve permission for search-engine indexing of public pages, while imposing separate restrictions on third-party AI agents.

**Do AI agents need permission to use United’s website or app?**

United’s policy requires written permission for automated AI agents that search, book or purchase on united.com or the United app. United also announced restrictions on unauthorized third-party personal AI agents and assistants.

**Does a robots.txt file determine all AI access rights?**

No. Robots.txt communicates crawler preferences, but website terms, contracts, API conditions and technical access controls may also govern automated access, data use and transactions. Search Engine Journal reported that United’s robots.txt did not name AI crawlers as of mid-November 2026.

**Does allowing indexing mean a company permits AI model training?**

No. United’s policy, as reported by Search Engine Journal, distinguishes search indexing from use of United site data to train or ground AI models, which its terms prohibit.

United’s update illustrates a growing need to separate public search visibility from AI-agent permissions. For SEO and digital teams, the key task is to make crawl directives, terms, partner contracts and transactional controls express the same underlying policy, rather than assuming that indexing rules answer every question about AI access.
