An AI training opt-out that works across several AI companies at once is finally on the way. On October 30, 2026, Google, Microsoft and OpenAI announced plans to honor one shared line in robots.txt that tells participating providers you do not want your content used to train generative AI models. The proposal comes from Anthropic’s Kevin Kelley, and Anthropic says it has already started building support. This guide explains how the new token works, what each company has committed to, what it does not do and seven smart steps to take before support goes live.
Our sources are the Internet Draft itself, Kelley’s note to the IETF working group, the official posts from Google, Microsoft and OpenAI, and reporting from Search Engine Journal and The Verge. Every date and quote below is attributed to one of them.
Table of Contents
What Is the Shared AI Training Opt-Out?
Today, a site that wants to keep its pages out of AI training has to name each company’s crawler or control one by one. OpenAI described the old routine plainly: “Previously, website operators would identify individual crawlers and maintain separate rules for each and every provider.” The shared AI training opt-out replaces that list with a single group in robots.txt that any participating provider can read.
The mechanism is defined in an Internet Draft by Kevin Kelley of Anthropic, titled “Machine-Readable Training Preference Signal” and published on 30 October 2026. Search Engine Journal notes that the IETF tracker lists it as an individual draft, which does not represent official IETF standardization activity. The draft expires on 3 May 2027 unless a new version replaces it.
So this is not yet a finished standard. It is a working proposal that several of the largest AI providers have said they will honor, which is why it matters now.
How the Policy Token Works
The draft adds no new directive. Instead, you write an ordinary robots.txt group whose User-Agent line names a “policy token” rather than a crawler. The token is __usage_ai-training. The draft’s own example looks like this:
User-Agent: __usage_ai-training
Disallow: /
Allow: /press-kit/
That example expresses a preference against AI training for every path on the site except the press kit folder, where it expresses a preference in favor. Normal robots.txt path matching and longest match precedence apply, so you can set different preferences for different sections.
Two design choices protect sites that do nothing. First, wildcard groups never match the token. Kelley wrote that “existing User-Agent: * rules aren’t read as expressing a preference.” Second, a file without a policy token group expresses no preference at all, which he says is true of every robots.txt file published today. Your current rules will not be reinterpreted as a training preference by accident.
The meaning of “AI training” comes from the working group’s vocabulary draft. As Microsoft summarizes it, it covers using an asset “to modify the learned parameters of a generative AI model.” Training models used only for non generative tasks, such as classification or ranking, is not included.
Who Supports It and When
None of the companies supports the line yet, so timing matters as much as the announcement. Here is what each one said.
Google. In a Search Central post by John Mueller, Google said that when a site uses the token to opt out and Google-Extended is not explicitly specified, Google plans to interpret that “as if the publisher had disallowed Google-Extended.” Google says it aims to support the control in 2027, and its crawler documentation says early 2027.
Microsoft. Bing said it will honor the new preferences as it crawls, starting February 1, 2027. It is also replacing its older NoArchive and NoCache controls for training and grounding, and asks sites to publish their new settings before March 1, 2027.
OpenAI. OpenAI said, “We aim to implement the signal in the coming months,” and added that its existing crawler controls “will continue to apply and be respected.”
Anthropic. Kelley told the working group, “Anthropic has already started implementing support for this draft,” while warning that “It’ll take us some time to add support across our services.”
Meta. According to The Verge, Meta communications head Andy Stone signaled support for the setting but did not say whether Meta will honor it.
Training vs Grounding: The Part Most Sites Miss
Training and grounding are different uses. Training changes a model. Grounding feeds content to a model when it answers a question, often with a link back. The shared AI training opt-out only speaks to training, but each provider ties it to grounding differently.
At Google, both uses sit under Google-Extended. The crawler documentation says Google-Extended governs training for future Gemini models and grounding in Gemini Apps and Grounding with Google Search on Vertex AI. Google’s post adds that “Websites that specifically allow Google-Extended are eligible to appear in responses and receive clicks via grounding in Gemini Apps.” In other words, an opt-out read as a Google-Extended block may also affect that grounding visibility.
Microsoft splits the two. The Bing announcement keeps the shared token for training and adds a separate, page level AI Grounding preference. A new DisallowExtended option limits grounding to Microsoft operated experiences such as Copilot and excludes grounding services Microsoft provides to third parties. Microsoft also lets you set training at the page level with a tag such as <meta name="Bingbot" content="ai-training:disallow">, which overrides robots.txt for that page.
What It Does Not Do
The token is not a crawl block. The draft states, “This policy token is not a directive for crawling behavior.” If you want to stop a crawler from fetching pages, you still write rules for that crawler’s own product token. Microsoft says the same about its settings: they govern use of content, not crawl access.
It does not change search eligibility either. Google’s documentation says Google-Extended “does not impact a site’s inclusion in Google Search nor is it used as a ranking signal in Google Search.” Microsoft says its preferences do not change eligibility for traditional Bing results.
It is also not a payment system. Microsoft is blunt here: “Neither choice guarantees inclusion, traffic, or payment.” If you want compensation, you still need a licensing route, which we cover in our guide to AI content licensing.
Why It Matters for Publisher Revenue
For an ad funded site, the shared signal is a negotiating position as much as a technical setting. A clear, machine readable “no” to training, honored by several providers at once, makes it easier to separate the AI uses you want, such as cited answers that can send clicks, from the uses you do not.
The trade off is visibility. Microsoft warns that if you disallow grounding, “you will see reduced brand visibility in AI experiences.” At Google, opting out through the shared token without a separate Google-Extended rule may also remove you from Gemini Apps grounding. Decide which outcome you want before you add the line.
Interest spiked after the announcement. Our count of original public posts on X mentioning AI training alongside robots.txt, opt out or policy token found 1,153 posts in the seven days to the morning of November 2, 2026, with the busiest day at 488 posts on October 31 (UTC). Google Trends interest in the exact phrase “ai training opt out” stayed low over the last three months, which fits a story driven by trade press rather than consumer search.
For background on what robots.txt can and cannot control, read our guide to AI crawlers and robots.txt, and see how to block LLM bots from scraping ad inventory if crawl load is your bigger problem.
7 Smart Steps Before Support Goes Live
1. Decide your training position by section
The token supports path level rules. You might allow training on a press kit or evergreen guides and opt out on premium or original reporting.
2. Keep your crawler rules separate
The policy token does not block fetching. Leave your existing crawler groups in place and add the new group alongside them.
3. Check your Google-Extended rule
If you want Gemini Apps grounding but not training, note that Google ties both to Google-Extended. An explicit Google-Extended rule overrides how Google reads the shared token.
4. Audit NoArchive and NoCache tags
Microsoft will stop reading those tags as training or grounding preferences after March 1, 2027. If you use them, plan the switch now.
5. Use the Bing transition window
Temporary grounding preferences open in Bing Webmaster Tools on December 15, 2026, and only for sites using NoArchive or NoCache as of that date.
6. Test without guessing
Adding the group early is low risk because parsers that do not implement it should ignore it. Still, validate the file and watch crawl logs after any change.
7. Track the draft
The draft can change before it expires on 3 May 2027. Google and OpenAI say they will share more details as support arrives.
The shared AI training opt-out is the simplest training control publishers have been offered so far: one line, several providers, no new grammar. It is not a crawl block, a ranking factor or a payment deal, and each company reads it a little differently. Set your position deliberately, keep your crawler rules intact and revisit the file as 2027 support rolls out.
Frequently Asked Questions
What is the shared AI training opt-out?
It is a robots.txt group keyed by a policy token, __usage_ai-training, that tells participating AI providers you prefer your content not be used to train generative AI models. It comes from an Internet Draft by Anthropic’s Kevin Kelley published on 30 October 2026.
Which companies plan to honor the AI training opt-out?
Google, Microsoft and OpenAI announced plans to honor it, and Anthropic says it has already started implementing support. Meta signaled support but did not say whether it will honor it.
When will the AI training opt-out start working?
Microsoft says it will honor the new preferences from February 1, 2027. Google aims to support it in 2027, OpenAI says in the coming months and Anthropic says it will take some time across its services.
Does the policy token block AI crawlers?
No. The draft says the policy token is not a directive for crawling behavior. To stop a crawler from fetching pages, you still need rules for that crawler’s own product token.
Will opting out of AI training hurt my Google Search rankings?
Google says Google-Extended does not impact a site’s inclusion in Google Search and is not used as a ranking signal. Google plans to read the shared opt-out as a Google-Extended block when Google-Extended is not specified.
Do my existing robots.txt rules count as an opt-out?
No. Wildcard groups never match the policy token, and a robots.txt file without a policy token group expresses no preference.
Related posts:
6 Top App Monetization Models Developers Need for Success
Ad Network Comparison for Low Traffic Sites: 2026 Guide
From Unknown to Recognized: How to Achieve a Verified TikTok Status
How Many Watch Time Hours Do You Need to Monetize YouTube? A Complete Guide
The Best Adsense Alternative - Infolinks
