Do AI crawlers run JavaScript? What GPTBot and ClaudeBot see on a React site

2026-11-07

Mostly, no. The crawlers behind ChatGPT and Claude download your HTML, and even your JavaScript files, but they do not run the scripts. Vercel looked at real crawler traffic, and its results "consistently show that none of the major AI crawlers currently render JavaScript." Suppose your React, Vue or Angular site builds its text in the browser. Then these crawlers get only what your server sent before any script ran, which is often an empty root element and a list of script tags. Google is the main exception: Googlebot renders pages in a headless Chromium, and Google's Gemini uses the same setup. If you want the other AI crawlers to read a page, its content has to be in the HTML your server returns.

What AI crawlers fetch, and what they skip

Vercel's analysis covers OpenAI's GPTBot, Anthropic's Claude crawler and several others. Not running JavaScript doesn't mean they ignore your script files. JavaScript files made up a share of each crawler's fetches (ChatGPT: 11.50%, Claude: 23.84%), yet neither crawler executes them. So these crawlers spend requests on your bundles and still read only the server's HTML.

From that study, the crawlers fall into two groups:

Read only the server's HTML: the crawlers behind ChatGPT and Claude. Vercel's advice is direct: "ChatGPT and Claude don't execute JavaScript, so any important content should be server-rendered."
Also reads only the server's HTML: Common Crawl's CCBot. Vercel notes it is often used as a training dataset for large language models, and it does not render pages.
Renders JavaScript: AppleBot, which uses a browser-based crawler similar to Googlebot.
Renders JavaScript: Google's Gemini, which uses Googlebot's infrastructure and so gets full JavaScript rendering.

Vercel is specific about what needs to be in the server's HTML: main content (articles, product information, documentation), meta information (titles, descriptions, categories), and navigation structures. Suppose a product page's name, price and description only appear after a fetch call in the browser. A crawler that doesn't run scripts never sees them.

Why Googlebot sees more than AI crawlers

Google processes JavaScript web apps in three main phases: crawling, rendering and indexing. A fetched page waits in a render queue. Google's documentation says the page may stay on this queue for a few seconds, but it can take longer than that. Once Google's resources allow, a headless Chromium renders the page and executes the JavaScript. That is why a client-rendered site can rank in Google and still look empty to an AI crawler.

Google doesn't treat its own rendering as a reason to skip server output. Its JavaScript SEO guide says that server-side or pre-rendering is still a great idea, partly because not all bots can run JavaScript. The AI crawlers above are exactly those bots.

Check what a crawler receives from your pages

Don't guess from how the site looks in your browser. Your browser runs the scripts; the crawlers in question don't. These checks show what's in the raw HTML:

Open View Source, not the DevTools Elements panel. View Source shows the HTML the server sent. The Elements panel shows the page after scripts changed it. Search the source for a sentence from the page's main text.
Turn off JavaScript in your browser's site settings and reload the page. What's left is roughly what a crawler that doesn't render scripts can read.
Fetch the page from a terminal with a crawler's name as the user agent, as in the command below. This matters if your server or middleware sends crawlers different HTML from visitors: you want to see the crawler's version.
For Google, use the Rich Results Test or the URL Inspection Tool in Search Console. Google's own guidance is to use them to check if the rendered HTML has all content you expect.
Check one page of each template, not just the homepage: a product page, an article, a docs page, a category listing. Templates often fetch their data differently.

Raw HTML as a crawler that doesn't run scripts receives it

# Set PAGE_URL to one of your own page addresses first
curl -s -A "GPTBot" "$PAGE_URL" > crawler.html

# Count matches for a sentence that visitors see on the page
grep -c "a sentence copied from the visible page" crawler.html

If grep finds no match, the text is added in the browser and the crawlers behind ChatGPT and Claude won't see it. Also check the title tag and the meta description in crawler.html: on many single-page apps they stay generic until a script replaces them.

Ways to put content into the first HTML response

There are two broad fixes: change how the site renders, or give crawlers a rendered copy.

Server-side rendering: the server runs your framework for each request and sends finished HTML, which the browser then makes interactive. Every crawler and every visitor gets the content.
Static rendering: pages are built to HTML ahead of time. This suits marketing pages, documentation and articles that don't change per visitor.
Prerendering for crawlers, which Google calls dynamic rendering: your server detects crawlers and sends their requests to a rendering service that opens the page in a browser and returns the finished HTML. Visitors still get the client-rendered app. Google calls this a workaround and not a recommended solution, but it doesn't require rewriting the front end.

If you can move content pages to server-side or static rendering, do that. It fixes the problem for every bot and doesn't depend on detecting crawlers. Prerendering makes sense when a rewrite isn't practical, for example with a large Angular or Create React App codebase that works well for its visitors.

If you prerender for crawlers

Google describes the mechanism: "Dynamic rendering requires your web server to detect crawlers (for example, by checking the user agent)." In practice that's a small middleware in front of your app plus a rendering service. Before you rely on it, check a few things:

Serve the same content. Google says Googlebot generally doesn't consider dynamic rendering as cloaking when the content is similar. But using dynamic rendering to serve completely different content to users and crawlers can be considered cloaking.
Check the crawler list. Your middleware only prerenders for user agents it recognizes. Confirm that the AI crawlers you care about are on the list, and test it with the curl command above.
Cache rendered pages. Opening a browser for every crawler request is slow and costly. A stored copy can answer repeat requests.
Decide what happens when a render fails. A timeout or a blocked request should send the crawler your normal page, not an error.
Re-test after front-end releases. A new data-loading pattern can leave the rendered copy empty without anyone noticing.

One option is RenderSwap, a hosted prerender service made by the studio that publishes this blog. It works with prerender-node middleware you may already run. If you send crawler requests to Prerender.io's service URL today, you change two values, the service URL and the token, and the middleware stays. A Prerender.io token gets a 401 from RenderSwap, so the token has to change too. RenderSwap Pro is $19 a month for 30,000 fresh renders. Prerender.io's Starter plan is $59 a month for 30,000 renders, according to Prerender.io's pricing notice of 14 September 2026. When a render fails or a month goes past the cap, the crawler gets your origin's own page and you pay nothing extra. RenderSwap is not affiliated with or endorsed by Prerender.io. A rendering service of any kind is still the workaround route. If server rendering is within reach, it's the cleaner fix.

What robots.txt and client-side code still do

Rendering and access are separate questions. Vercel found that the robots.txt file is effective for all measured crawlers. If you want the crawlers behind ChatGPT and Claude to read your pages, make sure robots.txt doesn't block them, then make sure the content is in the HTML. If you would rather keep them out, robots.txt rules for each crawler's user agent are how you do it, and rendering doesn't matter.

You don't need to remove client-side code either. Vercel notes that client-side rendering still works for enhancement features. A chat widget, a sort control or a cart counter can stay in the browser. What has to be in the server's HTML is the text and links a crawler should learn from.

Sources

Questions

Does Googlebot run JavaScript?

Yes. Googlebot puts pages in a render queue. Once Google's resources allow, a headless Chromium renders the page and executes the JavaScript. The wait can be a few seconds or longer. Google still says server-side or pre-rendering is a great idea, because not all bots can run JavaScript.

Do AI crawlers download my JavaScript files?

Yes. Vercel's data shows JavaScript files making up a share of their fetches (ChatGPT: 11.50%, Claude: 23.84%), but neither crawler executes them. The files cost you requests and the crawler still reads only the server's HTML.

Is serving prerendered HTML to crawlers cloaking?

Not when the content matches what visitors see. Google says Googlebot generally doesn't consider dynamic rendering as cloaking. Serving completely different content to users and crawlers can be considered cloaking.

Does Gemini see client-rendered content?

Vercel reports that Google's Gemini uses Googlebot's infrastructure, which enables full JavaScript rendering. The crawlers behind ChatGPT and Claude don't render JavaScript, so a page that only works with scripts can show up for one assistant and not another.