United Airlines bans AI agents in its terms but not in robots.txt
Summary
United Airlines' new terms let search engines index its public pages but require written permission for any automated agent, while its robots.txt names no AI crawler at all.
Check that your own terms, robots.txt and edge bot rules agree on what an agent may do on your site.
United Airlines updated its website terms on 11 November to require written permission from any automated agent that searches, books, buys or manages anything on united.com or in the airline’s app, while leaving search engines free to index its public pages, according to Search Engine Journal’s reading of the terms. The same terms bar anyone from using United’s fares, schedules, availability or pricing to “train, fine-tune, ground, evaluate, or otherwise develop” an AI model. The change arrived with an announcement from United that it is restricting third-party AI agents on its own platforms and through travel agencies and booking sites.
Search Engine Journal checked United’s robots.txt on 14 November and found no AI crawler or agent named in it. The legal half of the policy is written and the technical half is not. For the agents United says it is worried about, there may be nothing to write: a personal assistant working through a browser session publishes no user agent for a Disallow line to name.
United’s definition is wide enough to cover most of that traffic anyway. An Automated Agent is any software that acts with full or partial autonomy to search, book, buy, manage or extract data, which the terms spell out as including AI assistants, bots, scrapers, browser extensions and features built into browsers, even when a person supervises or approves each action. Standard browsers and assistive technology are exempt. A browser’s built-in agent feature is not, and using an AI tool to plan a trip stays allowed as long as the tool never touches united.com or the app.
The reason United gives is account security rather than data licensing. The announcement says sharing credentials with a third-party assistant can expose sensitive information and that automated actions can lead to “unintended purchases, account changes, or use of MileagePlus miles”. It adds that automated actions could breach Federal Aviation Administration and Department of Transportation rules, pointing to check-in, where a person has to confirm their bags hold no banned items such as explosives.
The rule reaches past United’s own site. Search Engine Journal’s piece reports that Rule 6.J.6 of the contract of carriage adds using an AI agent to make, modify or service a ticket to the list of prohibited practices, next to hidden-city ticketing, and that Rule 5.H puts agent-made bookings among the improper bookings United will cancel without notice. Travel agencies and booking platforms need permission before pointing their own AI tools at United.
The exemption for search engines leaves a gap. Indexing is allowed, grounding is barred, and the terms say nothing about answers that AI systems build out of a search index. A fare that Google can index is a fare that can be summarised in an AI Overview, so on the likeliest reading the grounding ban binds only systems that collect the data themselves. Anyone reusing United’s wording should settle that question in the document rather than leave it to be argued later.
What to do
Treat the two layers as doing different jobs, and write both:
- Keep the permission rule in the terms. A clause binds the user, whoever operates the agent and the party it acts for, which robots.txt cannot do.
- Name the bots that declare themselves. Training crawlers and retrieval bots publish user agents that robots.txt can address, though the scope of an AI training opt-out still differs by company, so check what each one covers before relying on it.
- Push undeclared agent traffic to the edge, where Cloudflare now lists AI agents by name and lets you block each one.
- Say in writing what you intend for AI answers built from a search index, since an indexing exemption will otherwise be read as permission.
Watch out for
United’s announcement, FAQ, terms and contract do not say how it detects an agent, which leaves a shopper using an agentic browser unable to tell what will trigger a cancellation. Writing the same silence into your own terms makes enforcement look arbitrary, and blocking agent-driven sessions at the edge can catch the assistive technology those terms meant to exempt.