How to check if ChatGPT can read your website (5-minute test)
Open your terminal and type curl -A "GPTBot" https://yoursite.com/page. The -A "GPTBot" part is what makes the request look like it’s coming from OpenAI’s crawler instead of a browser, so what you get back is the raw code sitting behind the page, before any JavaScript has run. Open the same page in a regular browser next to it. If the two don’t match, GPTBot is missing content your visitors see just fine.
GPTBot isn’t the whole story here. OpenAI runs three separate crawlers, and GPTBot isn’t even the one that decides whether you get cited in a ChatGPT answer. GPTBot pulls material for training. OAI-SearchBot builds the index ChatGPT actually pulls citations from. ChatGPT-User only shows up when someone drops your link directly into a chat.
The test below checks GPTBot specifically, and it’s still where to start. The failure it catches most often, a JavaScript shell with no real content in the raw HTML, breaks all three crawlers at once. A page that fails this test is very likely invisible to OAI-SearchBot and ChatGPT-User too.
Step 1: See what GPTBot sees
curl -A "GPTBot" https://yoursite.com/pageThe -A flag swaps in the GPTBot user agent, so the server thinks it’s talking to OpenAI instead of a browser. Scan the output for whatever’s actually written on the page: your headlines, your body copy.
If instead you’re staring at basically nothing, something like <div id="root"></div> or <div id="__next"></div> with a stack of <script> tags underneath, that’s your answer. The content is real. It just doesn’t show up until JavaScript builds the page inside a browser, and GPTBot never runs that JavaScript, so as far as it’s concerned, the page is empty.
Step 2: No terminal? Use view source instead
Right-click the page and pick View Page Source, not Inspect. Source shows what the server actually sent. Inspect shows the page after JavaScript has already finished building it, which can look completely normal even on a page GPTBot can’t read.
Once you’re in source view, search for a sentence you know is on the page. Not there means invisible to AI crawlers; same conclusion as Step 1, no terminal needed.
Step 3: Check robots.txt
curl -s https://yoursite.com/robots.txtLook for User-agent lines naming GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, or PerplexityBot, and check whether any are set to Disallow. A User-agent: * block that disallows everyone blocks these crawlers too, so check for that as well. No robots.txt file at all isn’t a red flag. It defaults to allowing everyone in, so don’t mistake a missing file for a block.
Step 4: Try the bare domain
Type your domain into a browser without the www. A handful of sites we’ve audited have no A record set up on the apex domain (an A record is what points your domain to a server; no record means no destination), so anyone typing the name from memory, person, or crawler hits a connection error instead of the homepage.
Step 5: Check the metadata layer
Back in the page source, look for a title tag, a meta description tag, and structured data written in JSON-LD format. These are the labels AI engines lean on to understand what your page is about and how to describe your business when they mention you.
What does this test actually prove?
It confirms what GPTBot pulled from your server right now, nothing more. It doesn’t tell you whether ChatGPT has crawled the page recently, whether OAI-SearchBot has it indexed, or whether fixing what curl turned up gets you a citation. Crawling, indexing, and citation are three separate hurdles, and this test only checks the first, for one of OpenAI’s three crawlers.
When people say “the ChatGPT user agent,” they usually mean GPTBot. It’s the one that shows up in nearly every crawlability guide, and the one robots.txt rules tend to name first. If what you actually want is to show up inside a ChatGPT answer, swap OAI-SearchBot into that -A flag and run the same test. Nothing else changes, just the name you’re pretending to be.
For a broader look at what actually gets cited once a page clears this test, we ran 500 prompts across five AI engines to see what wins. Read the study →
After you fix it
Re-run Steps 1 through 3. Raw HTML matching the browser view means you’re done here.
What this looks like on a live site
We’ve audited live marketplaces with 20,000+ product listings where the homepage serves under a kilobyte of empty shell to any crawler that requests it. The content is real and visible to every visitor who loads the page. It just doesn’t exist yet for anything that doesn’t run JavaScript, which is most AI crawlers.
Skip the manual checks
Yogoo AI’s free GEO Site Audit runs this exact test automatically. Then a free Yogoo Score shows how ChatGPT, Claude, Perplexity, Gemini, and Google AI Overviews actually cite you. Paste your domain and get started in under a minute with a free account.