Google, Microsoft, OpenAI, and Anthropic support a new way for websites to say no to AI training. Today, website owners must write separate rules for each company. The new plan lets them share their choices with several companies through one rule.
Owners put the rule in robots.txt, a file on their website. It tells companies which content they can collect to train AI. Owners can say no for the whole site or allow some parts. The rule does not remove pages from regular search results. It also cannot take back content companies already collected.
Anthropic has started work on the change. OpenAI plans to add support in the coming months. Google plans to do so in early 2027. Microsoft will start on February 1, 2027.
Kevin Kelley of Anthropic wrote the plan. The Internet Engineering Task Force, a group for internet standards, has not approved it. The rule shows an owner's wishes; it does not itself stop programs from visiting a site. The companies promise to respect those wishes.
