Cloudflare's New AI Bot Defaults

Cloudflare's New AI Bot Defaults: What Bloggers Need to Check Before September 15, 2026

Cloudflare just changed how it classifies AI bot traffic, and the update carries a real deadline. Starting September 15, 2026, new domains onboarding to Cloudflare will have two out of three AI traffic categories blocked by default on any page that displays ads — including AdSense. If you run a Blogger or WordPress site behind Cloudflare, this is worth ten minutes of your time now, before the defaults change under you or a future site.

Quick Win: Log into your Cloudflare dashboard today and check your current AI bot settings under Security → Bots (or the AI Crawl Control section if you're on a plan that has it). This feature is available to every Cloudflare customer, including the free tier, as of July 1, 2026 — you don't need to wait for September to see or use it.

What Actually Changed

Cloudflare used to treat "AI bots" as a single category — block them all or allow them all. That's gone. Traffic is now split into three distinct behaviors:

• Search — bots that index your content to answer questions about it later, the same basic job traditional search engines do. Cloudflare's position is that Search behavior should send referral traffic or other compensation back to you, so it stays allowed by default.
• Agent — an automated system acting on a real person's behalf in real time to complete a specific task right now. This includes chat-fetch bots like ChatGPT-User and browser-driving agents like Gemini or Claude that operate within Chrome. There's usually a human waiting on the other end for the result.
• Training — crawlers collecting your content specifically to train or fine-tune AI models, with no real-time task and no user waiting.

Proof Block — Screenshot This: Capture the new three-category toggle screen in Cloudflare's dashboard (Security → Bots or AI Crawl Control), plus the crawler list showing which specific bots fall under each category for your domain. Readers respond well to seeing the actual settings screen rather than a description of it.

The September 15 Deadline — What It Actually Applies To

This is the detail most coverage of this update glosses over: the new defaults only apply to domains newly onboarding to Cloudflare on or after September 15, 2026. If your blog is already on Cloudflare today, your existing settings won't silently change on that date. The defaults being introduced are:

Category	Default on ad-monetized pages (new domains, Sept 15+)
Search	Allowed
Agent	Blocked
Training	Blocked

The logic Cloudflare gives for this: an ad on a page signals the content was built for a human to land on and see, which is monetizable. Training and Agent bots don't send a person to read that ad — Search behavior generally does, through referral clicks — so Search stays open while the other two default closed.

If you're launching a new blog or moving an existing one to a new Cloudflare zone after September 15, you'll inherit these defaults automatically. If your current site is already connected, you're unaffected unless you change something yourself — but you should still actively review your settings now, for the reason below.

The Warning Every AdSense Blogger on Cloudflare Needs to Read

Warning: Blocking the "Training" category is not as isolated as it sounds. Cloudflare has confirmed that multi-purpose crawlers — bots that combine Search behavior with Training behavior in the same crawler — get blocked or allowed based on all of their combined behaviors. Googlebot, Applebot, and Bing's crawler all fall into this multi-purpose bucket. That means if you block Training thinking you're only stopping AI model scraping, you can accidentally block Googlebot itself — the crawler responsible for your actual organic Google Search rankings and indexing.

For a blog that depends on organic Google traffic to feed AdSense impressions, this is not a small misconfiguration. Before you touch any Training or Agent toggle, check Cloudflare's current crawler list for your domain and confirm Googlebot, Bingbot, and Applebot are explicitly allowed, separate from whatever you decide about dedicated AI training bots like GPTBot or Google-Extended.

How to Check and Configure This Today

1. Log in to the Cloudflare dashboard and select your domain
2. Go to Security → Bots, or AI Crawl Control if it appears as its own section on your plan
3. Review the Search, Agent, and Training toggles individually — don't accept a bundled "block all AI bots" switch without checking what it actually covers
4. Open the crawler list and confirm Googlebot, Bingbot, and Applebot show as allowed, independent of your Training/Agent decisions
5. Decide deliberately on Agent bots — if you want your content surfaced when someone asks ChatGPT or Gemini a question in real time, allowing Agent behavior is usually the better call for traffic; if you're purely defending against scraping with no real-time task involved, Training is the one to restrict

This connects directly to the crawler-permission work covered in our guide on getting cited in ChatGPT and Perplexity — that post focuses on robots. txt-level control over specific bots like PerplexityBot and GPTBot, while Cloudflare's new categories give you the same kind of control at the network level, which is worth double-checking against whatever robots.txt rules you already have so the two don't contradict each other.

Why Category-Level Thinking Beats Bot-by-Bot Rules

The old approach to this problem, which is still what most guides teach, is to maintain a list of individual bot user-agents and manually decide to allow or block each one as new AI crawlers appear. That list changes constantly and is easy to let go stale. Cloudflare's category system is trying to solve the actual underlying question a blogger has — "Does this bot send me traffic or attention, or does it just take my content?" — without requiring you to keep a spreadsheet of every crawler name in existence.

That said, category-level control isn't a substitute for checking your specific numbers. If your site sells physical products or leans on structured data for AI shopping visibility, the crawler-access considerations in our guide to optimizing e-commerce for AI shopping search are worth reviewing alongside this — allowing Search and Agent crawlers to matter even more when AI-driven purchase recommendations are part of your traffic mix.

FAQ-Cloudflare's New AI Bot Defaults

Q1. Will my existing Cloudflare site suddenly get AI bots blocked on September 15? 

No. The new defaults apply only to domains newly onboarding to Cloudflare on or after that date. Existing connected domains keep their current configuration unless you change it yourself.

Q2. Does blocking "Training" bots hurt my Google Search rankings? 

It can, indirectly. Googlebot is classified as a multi-purpose crawler that combines Search and Training behaviors. If you block Training, Cloudflare enforces that block across all of Googlebot's combined behavior, which can affect indexing. Always confirm Googlebot is explicitly allowed before touching Training settings.

Q3. Is this feature only for paid Cloudflare plans? 

No. The new Search, Agent, and Training controls are available to all Cloudflare customers, including the free tier, as of July 1, 2026.

Q4. Should I block Agent bots like ChatGPT-User? 

It depends on your goals. Agent bots visit your site in real time to complete a task for a specific user who's waiting on the result — often the same kind of visit that would eventually show your content inside an AI chat answer. If you want visibility inside AI assistants, allowing Agent traffic is usually the better default; if you're strictly trying to minimize any automated access to your content, blocking it is the more defensive option.

Q5. What's the difference between Agent bots and Training bots in practical terms? 

Training bots collect content in bulk to improve a model over time, with no real-time task attached. Agent bots are triggered by an actual person asking an AI assistant to do something right now — checking a price, summarizing a page, completing a booking — and there's a live task waiting on the result.

Final Thoughts

The headline number here is September 15, but the more useful action item is today: go check your current Cloudflare bot settings, confirm Googlebot isn't accidentally caught in a Training block, and make a deliberate choice about Agent traffic instead of leaving it on whatever the old bundled toggle left you with. For an AdSense-dependent blog, getting the Search versus Training versus Agent distinction right is now part of basic technical SEO hygiene, not an advanced edge case.