What is AI visibility, and how is it different from SEO?

SEO decides whether your page appears in a list of blue links. AI visibility — also called Answer Engine Optimization or AEO — decides whether an AI assistant like ChatGPT, Gemini, Claude or Perplexity can reach your page, understand it, and name your business in its answer. A site can rank well on Google and still be invisible to AI, most commonly because its robots.txt blocks AI crawlers or its content only renders with JavaScript.

How do I check if ChatGPT can see my website?

ChatGPT reaches websites through crawlers named GPTBot, OAI-SearchBot and ChatGPT-User. If your robots.txt file disallows those user agents, ChatGPT can never read or cite your site. Enter your URL in this tool and the AI Visibility report shows each AI crawler's access individually, along with the exact robots.txt rules blocking them.

Why is my site blocked from AI crawlers when my robots.txt allows them?

A CDN can override your file. Cloudflare's AI Crawl Control feature injects its own managed block into robots.txt, disallowing GPTBot, ClaudeBot, Google-Extended and others before your own rules are read. Check Cloudflare's dashboard under AI Crawl Control and set AI crawlers to Allow. Some hosting firewalls and security plugins block these bots by IP as well.

What is llms.txt and do I need one?

llms.txt is an emerging plain-text file at the root of your site that gives AI models a curated summary of your business, services and key pages, in the same spirit as robots.txt for crawlers. It is not yet an official standard and no engine guarantees it will read one, but it costs nothing to add, very few sites have one, and it lets you state plainly who you are rather than leaving models to infer it.

Is this tool free, and does it need my password or website access?

The scan is free and requires no signup, no password and no access to your site. It reads only what is publicly available: your page HTML, robots.txt, sitemap and llms.txt — the same files any search engine or AI crawler fetches. You provide an email only to unlock the full dashboard.

Does this show my Google ranking position?

No, and any free tool claiming to is guessing. Real ranking positions come from Google Search Console for your own site, or a paid rank-tracking API. What this tool measures is topic match: how clearly a page signals that it is about a given query, scored from where the words appear across your title, H1, headings, copy, URL and schema. A page scoring zero is not in the running at all; seventy or higher is properly set up to compete.

How can I improve my AI visibility score quickly?

In order of impact: unblock AI crawlers in robots.txt and at your CDN; add JSON-LD structured data identifying your business; add an FAQ section answering the exact questions customers ask, marked up with FAQPage schema; make sure your content is present in the raw HTML rather than rendered by JavaScript; and publish an llms.txt file. The tool's report ranks these by the points each one would recover on your specific site.

How many pages of my website does the scan cover?

It scans the page you enter first, then finds every other page it can through your sitemaps and internal links and scores those too, up to 150 pages. Progress is shown live and you can stop at any point and keep the results gathered so far. Each page gets its own AI, SEO and speed score, and the dashboard shows the average across the whole site.

Is Your Website Blocked From ChatGPT? How to Check GPTBot in robots.txt

By LunieaJuly 20, 2026

ChatGPT reaches websites through three crawlers: GPTBot, OAI-SearchBot and ChatGPT-User. If your robots.txt file disallows any of them, ChatGPT cannot read or cite your pages — no matter how good your content or your Google rankings are. You can check this in under a minute by opening yoursite.com/robots.txt and searching for those names.

What each OpenAI crawler actually does

GPTBot collects content used to train and improve OpenAI's models. Blocking it keeps your content out of training data, which some publishers want, but it also means the model has less grounding in who you are.

OAI-SearchBot powers ChatGPT's search results and citations. This is the one that matters most for visibility: block it and your business cannot appear as a cited source when someone asks ChatGPT for a recommendation.

ChatGPT-User fetches a page in real time when a user asks ChatGPT to look at a specific link. Blocking it means your page fails to load when a potential customer explicitly asks about you.

How to check your robots.txt in 60 seconds

Open your browser and go to yoursite.com/robots.txt. Search the page for GPTBot, OAI-SearchBot and ChatGPT-User. If you find any of them followed by a line reading Disallow: /, that crawler is fully blocked.

Also check the User-agent: * group at the top. Rules there apply to any crawler that doesn't have its own group, so a blanket Disallow: / blocks the AI crawlers too.

Watch for order: a crawler follows the most specific group that matches its name, and ignores the rest. That means an AI-friendly group further down the file will not undo a block if the crawler already matched an earlier, more specific group.

The robots.txt rules that let AI crawlers in

Add an explicit group welcoming the AI crawlers. Being explicit matters: SEO plugins and security tools regularly rewrite robots.txt, and a stated rule survives those rewrites better than an absence of rules.

Add to robots.txt

User-agent: GPTBot
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: ClaudeBot
User-agent: PerplexityBot
User-agent: Google-Extended
Allow: /

Sitemap: https://yoursite.com/sitemap.xml

Your robots.txt says allow, but AI still can't read you

This is more common than a straightforward block, and it has three usual causes.

Your CDN overrides the file. Cloudflare's AI Crawl Control injects a managed block into robots.txt ahead of your own rules, disallowing GPTBot, ClaudeBot and Google-Extended. Your file says allow; what the crawler downloads says disallow.

Your content requires JavaScript. Most AI crawlers do not execute JavaScript. If your text only appears after a script runs, the crawler receives an effectively empty page. Press Ctrl+U on your own site — that raw HTML is what AI sees.

Your firewall blocks the bot by IP or user agent before robots.txt is ever consulted. Security plugins and bot-protection rules do this routinely, and it is invisible from the file itself.

Frequently asked questions

Does blocking GPTBot hurt my Google rankings?

▾

No. GPTBot is OpenAI's crawler and has no connection to Googlebot or Google's ranking systems. Blocking it only affects whether your content reaches ChatGPT.

Should I allow AI crawlers or block them?

▾

If you want customers to discover you through AI assistants, allow them — being absent from an answer means a competitor fills that space. Publishers whose business model is selling access to their content often choose to block training crawlers while allowing search crawlers like OAI-SearchBot, which is a reasonable middle ground.

How long until ChatGPT sees my site after unblocking?

▾

Crawlers revisit on their own schedule, typically days to weeks depending on how often your site changes and how much authority it has. There is no way to request an immediate recrawl the way you can with Google Search Console.