Opt your WordPress site out of AI training with one robots.txt rule

On October 30, Google, Microsoft and OpenAI said they plan to honour one shared robots.txt line for AI training: __usage_ai-training. The same day, Kevin Kelley of Anthropic published it as an IETF draft. If you run a WordPress site, this is the first time one rule may speak to all four companies.

The short answer: add a small group to your robots.txt now if you don’t want your content used for training, but know what it does and doesn’t do. It is a preference, not a block. Support mostly starts in 2027. At Google, it can also cost you something you may want to keep, so read that section before you ship it.

How do I opt my WordPress site out of AI training with one robots.txt rule?

Give the token its own group with Disallow: /. These are the lines:

User-Agent: __usage_ai-training
Disallow: /

Paths work the normal way, so you can refuse training on the site but allow it for one folder. The draft’s own example uses a press kit:

User-Agent: __usage_ai-training
Disallow: /
Allow: /press-kit/

Your existing rules don’t count. The draft says User-Agent: * must not be read as matching the token, so even a site-wide wildcard block says nothing about training. With no group for the token, you have expressed no preference.

Where do these lines go on WordPress?

They go wherever your site’s robots.txt actually comes from, so find that first. WordPress only generates a virtual robots.txt when no physical file sits in the site root.

1. Look for a robots.txt file in your site’s root folder, using SFTP or your host’s file manager. If there is one, it is what visitors get, and you edit that file.
2. Use your SEO plugin’s editor if you already manage the file there. In Yoast SEO it is under Tools, then File Editor. In Rank Math, turn on Advanced Mode, then go to General Settings and Edit robots.txt. Rank Math’s guide says to delete any physical file first if you want to use its editor.
3. Otherwise, add the group with the robots_txt filter in a small must-use plugin, such as wp-content/mu-plugins/ai-training-preference.php. The code is below.
4. Purge your page cache and CDN, then open /robots.txt in a private window. It worked if you see the group exactly once, with its own User-Agent line.

<?php
add_filter( 'robots_txt', function ( $output ) {
	return rtrim( $output ) . "\n\nUser-Agent: __usage_ai-training\nDisallow: /\n";
}, 99 );

The priority of 99 makes it run late, after plugins using the default priority have changed the output. The filter does nothing if a physical file exists, because WordPress never runs in that case.

Who honours the token, and when?

Nobody has confirmed full support today, so treat this as setting your preference early. Here is what each company has said:

• Google: its crawler documentation says it intends to support the token in early 2027.
• Microsoft: Bing’s post says the new preferences begin taking effect on February 1, 2027.
• OpenAI: it plans to implement the signal in the coming months, and says the signal won’t replace its existing robots.txt controls.
• Anthropic: Kelley wrote that it has started implementing support and that adding it across its services will take some time. Engadget reported this as already supporting it, but Search Engine Journal’s reading is more careful and matches his words.

The document itself is an individual draft, version 00, and it expires on May 3, 2027 unless it is renewed. It has no formal standing at the IETF, so the name or the rules may change. Put the lines somewhere you’ll remember to revisit.

Will opting out of training hurt my visibility in AI answers?

At Google it can, in Gemini, because the token maps onto a control that covers more than training. John Mueller said that when a site uses the token and doesn’t specify Google-Extended, Google plans to treat that as if the site had disallowed Google-Extended.

Google’s documentation says Google-Extended covers both training future Gemini models and grounding in Gemini Apps. Mueller also said that sites which specifically allow Google-Extended are eligible to appear in responses and receive clicks through grounding in Gemini Apps. As I read that, the opt-out can take you out of those answers too. The same documentation says Google-Extended doesn’t affect inclusion or ranking in Google Search.

If being cited in Gemini matters more to you than refusing Gemini training, name Google-Extended in its own group, and accept that this keeps both uses open:

User-Agent: Google-Extended
Allow: /

This is the decision I’d make deliberately on any site where AEO or GEO is part of the job. Microsoft treats the two uses separately: the robots.txt token covers training only, and Bing says there is no site-wide robots.txt setting for grounding. You set grounding per page instead.

How is this different from Content-Signal lines?

Content-Signal is a separate format that Cloudflare introduced, and it sits inside a normal group rather than in a group of its own. You may already have one without having typed it. Cloudflare’s managed robots.txt adds a line like this when you turn on Bot Preference Sync and set a policy to block:

User-Agent: *
Content-signal: search=yes, ai-train=no, use=reference
Allow: /

The AI Content Signals plugin on WordPress.org writes Content-Signal preferences, through the robots_txt filter. The new draft doesn’t mention Content-Signal, and the four companies’ commitments are to the token. In July, Mueller said of Content-Signal that as far as he knew it had “no effects whatsoever for any crawler or LLM”. Cloudflare also warns that Search Console may report “Syntax not understood” for these lines.

The two can sit in the same file, because the token group is separate. I wouldn’t add Content-Signal by hand for training now. If Cloudflare or a plugin already adds it, the token goes alongside it, and the lines you should actually check are any Disallow groups Cloudflare manages for named bots.

Does the token stop AI crawlers visiting my site?

No, it says how your content may be used, not whether a bot may fetch it. The draft calls it “not a directive for crawling behavior”, and Bing says the same of its settings. You’ll keep seeing AI crawlers in your logs, and that doesn’t mean the preference failed, so logs can’t confirm it is working.

Keep the named rules you already have. Google says its company-specific controls continue alongside the token, and OpenAI says the same of its own. OpenAI hasn’t said what happens if the token and a GPTBot rule disagree, so make them agree. I also found no commitment from Meta, so the Meta crawler rules still need their own group. To see what these bots can read, use this crawler check.

What if my site uses NoArchive or NoCache?

You have a Bing deadline: after March 1, 2027, those tags stop counting as a training or grounding preference. An SEO plugin or a hand-written snippet may be setting them, so view a page’s source, look at its robots meta tag, and check its X-Robots-Tag header.

If you find them and want to stay out of grounding, Bing offers two routes. A temporary preference opens in Bing Webmaster Tools on December 15, 2026, only for sites still using those tags on that date, and it expires on September 30, 2027. The lasting one is a page-level tag. For training, the value is ai-training:disallow:

<meta name="Bingbot" content="ai-grounding:disallow">

Without either by March 1, Bing says those pages default to allowing grounding until it recrawls them.

This week, I’d open your live /robots.txt, decide what you want at Google before adding the token, and search your templates for NoArchive. Then set a reminder for early 2027 to check whether the draft has changed and whether the companies have shipped what they promised.