Free AI Crawler Checker: Is Your robots.txt Blocking GPTBot, ClaudeBot or PerplexityBot?
DigiMSM
Free Checker · 30 Seconds

Is Your Website Telling AI to Go Away?

Security plugins quietly block the exact bots that put you in ChatGPT answers. Paste your robots.txt below - see every AI crawler verdict instantly, and generate the right file.

100% freeNo signupRuns in your browser

Paste your robots.txt

Open yoursite.com/robots.txt, select all, copy, paste here.

Crawler verdict

Search bots feed live AI answers. Training bots feed model training. Different stakes.
-

Bot-by-bot result

Generate the right file

Two clean configurations - copy the one that fits your policy.

Frequently Asked Questions

How do I check if my robots.txt blocks AI crawlers?
Open yoursite.com/robots.txt in a browser, copy everything, and paste it into this checker. It parses the rules and shows a verdict for each major AI bot - distinguishing search bots that power live answers from training bots that collect model data.
Which AI bots should I never block?
Search-type bots: OAI-SearchBot and ChatGPT-User (ChatGPT answers), PerplexityBot, and Claude-SearchBot. Blocking these removes your business from live AI search. Training bots like GPTBot and Google-Extended are an optional separate decision.
My site blocks AI bots - how do I fix it on WordPress?
On WordPress robots.txt is usually generated by your SEO or security plugin, so edit it in the plugin settings (look for a robots.txt editor or an AI-blocking toggle) - a manually uploaded file can get overwritten. Also check the Cloudflare AI-blocking toggle if you use it, then re-check the live file.
Built by DigiMSM — Pakistan's 1st AI-Powered Digital Marketing Agency · Guides · WhatsApp us
Copied
Scroll to Top