AI Crawler Rendering

Do GPTBot and Google-Extended Render JavaScript on Product Pages

GPTBot does not render JavaScript in published crawl data, and Google-Extended is a robots.txt token. What each operator documents, plus a price and stock test.

On this page

GPTBot has not been shown to render JavaScript, and Google-Extended cannot render anything, because it is a robots.txt token with no crawler behind it. For a store whose price and stock arrive through a client-side API call, those two facts carry most of the answer: Google's renderer can run the script that fills in the offer, while the crawlers feeding ChatGPT and Perplexity may receive an empty shell. Below is what each operator documents as of October 7, 2026, what the one large published crawl study we found measured, and a test you can run on a single product URL.

Does GPTBot run the JavaScript on a product page?

GPTBot does not render JavaScript in the one large published measurement we found, and the crawler page from OpenAI never says that it does. That page lists four agents (OAI-SearchBot, OAI-AdsBot, GPTBot and ChatGPT-User) and gives each a purpose and a user-agent string. We found no statement on it about JavaScript, rendering or script execution. Of GPTBot it says: "It is used to crawl content that may be used in training our generative AI foundation models."

The measurement comes from Vercel, which published a study with MERJ on December 17, 2024. Its primary data came from nextjs.org and the Vercel network, checked against two job boards, one built with Next.js and one on a custom framework. For scale, Vercel reported 569 million GPTBot fetches across its network in the month before publication, against 4.5 billion for Googlebot. The central finding reads: "The results consistently show that none of the major AI crawlers currently render JavaScript." The list under that sentence names OpenAI's OAI-SearchBot, ChatGPT-User and GPTBot, Anthropic's ClaudeBot, Meta-ExternalAgent, Bytespider and PerplexityBot. The same section reports two that do render: Gemini, which uses Googlebot's infrastructure, and AppleBot, through a browser-based crawler.

Two details in the study matter for product pages. The crawlers do download script files (on nextjs.org, JavaScript made up 11.50% of the requests from ChatGPT's crawler and 23.84% of Claude's), yet the authors conclude that "they don't execute them." The study also notes that content in the initial HTML response, "like JSON data or delayed React Server Components, may still be indexed since AI models can interpret non-HTML content." A price inside a JSON-LD block in the first response can therefore still be picked up, while a price that exists only after a browser-side request resolves never reaches a client that skips the script.

That study is almost two years old, and crawlers change. We found no newer first-party statement from OpenAI or from Perplexity, whose crawler page describes PerplexityBot and Perplexity-User and is equally silent on JavaScript. Treat the table as a starting assumption and the test further down as the evidence.

Agent or tokenPurpose its operator statesOperator statement on JavaScriptMERJ and Vercel, December 2024
GPTBotCrawls content that may be used to train OpenAI's foundation modelsWe found noneDoes not render
OAI-SearchBotSurfaces websites in ChatGPT's search featuresWe found noneDoes not render
ChatGPT-UserVisits pages for user actions in ChatGPT and Custom GPTsWe found noneDoes not render
PerplexityBotSurfaces and links websites in Perplexity search resultsWe found noneDoes not render
GooglebotCrawls for Google Search and its featuresA headless Chromium renders the page and executes the JavaScriptFull JavaScript rendering, shared with Gemini
Google-Extendedrobots.txt token for Gemini training and groundingNo HTTP user agent string of its ownNot measured

Is Google-Extended a crawler that can ignore robots.txt?

Google-Extended does not bypass robots.txt or render JS, because nothing by that name ever requests a page. The list of common crawlers from Google, last updated July 14, 2026, puts it this way: "Google-Extended doesn't have a separate HTTP request user agent string. Crawling is done with existing Google user agent strings; the robots.txt user-agent token is used in a control capacity."

The token is itself a robots.txt instruction. It tells Google whether content its crawlers fetch may be used to train future generations of Gemini models, and for grounding in Gemini Apps and in Grounding with Google Search on Vertex AI. The same page states that "Google-Extended does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search." Rendering belongs to whichever crawler does the fetching. For Google Search that is Googlebot, and Google's JavaScript guide says Googlebot reads the robots.txt file before it requests a URL and skips any URL marked as disallowed.

What Google's documentation says about robots.txt, rendering and dynamic rendering

Google's crawler documentation covers Google-Extended, robots.txt, JavaScript rendering and dynamic rendering for Google Search on separate pages, and each page answers a different question. Four of those answers bear on a store.

  • On rendering, the JavaScript SEO basics guide from Google, last updated March 4, 2026, says Googlebot queues pages with a 200 status code for rendering, and that "Once Google's resources allow, a headless Chromium renders the page and executes the JavaScript." It gives no fixed wait: the page "may stay on this queue for a few seconds, but it can take longer than that." The guide still recommends server-side or pre-rendering, in part because "not all bots can run JavaScript."
  • On dynamic rendering, which means serving crawlers a prerendered copy chosen by user agent, the page from Google calls it "a workaround and not a recommended solution, because it creates additional complexities and resource requirements." It recommends server-side rendering, static rendering or hydration.
  • On AI features in Search, the guide from Google says a page shown as a supporting link in AI Overviews or AI Mode must be indexed and eligible for a snippet in Google Search, with "no additional technical requirements." It points to Google-Extended only for "AI training and grounding in some of Google's other systems."
  • On product markup, the merchant listing documentation from Google, last updated September 8, 2026, recommends "putting Product structured data in the initial HTML for best results" and warns that dynamically-generated markup "can make Shopping crawls less frequent and less reliable, which can be an issue for fast-changing content like product availability and price."

The one operator in the table that documents full rendering still steers price and stock toward the initial HTML, and for agents with no documented renderer that first response is the only version the published evidence says they read.

What a single-page app shows the Google-Extended user agent

A Google Extended user agent never requests your SPA or its dynamic pages, so there is no log line to look for and no rendering behavior to plan around. What a single-page app does have is two kinds of machine visitor. Googlebot fetches the shell, queues it and renders it when resources allow, so a client-rendered offer can reach Google Search after a delay Google does not put a number on. The other agents in the table receive the same shell and, on the published evidence, stop there. For routing, Google's JavaScript guide says to use the History API in place of URL fragments so that Googlebot can parse and extract your URLs.

A test for whether price and stock reach these crawlers

Request the page the way a client without a JavaScript engine would, then search the response for the exact values a shopper sees. This is the product-page version of the crawler access check in our guide to getting cited by ChatGPT, Perplexity and AI Overviews. The example is hypothetical: an invented store, and a jacket that the browser shows at 129.00 and in stock. The user-agent string is the GPTBot example on OpenAI's page, which notes that the version number may change.

URL="https://store.example/products/trail-jacket"
UA="Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot"

# 1. Save the response as the server sends it, with no script execution
curl -s -A "$UA" "$URL" -o raw.html

# 2. Count the matches for the visible price and the stock status
grep -o -F "129.00" raw.html | wc -l
grep -o -i -E "instock|in stock" raw.html | wc -l

# 3. Confirm robots.txt lets each agent fetch product URLs at all
curl -s "https://store.example/robots.txt" | grep -i -A 3 -E "gptbot|oai-searchbot|perplexitybot"
CheckHypothetical result, client-rendered offerHypothetical result, offer in the initial HTML
Matches for 129.0002 (visible price and JSON-LD offer)
Matches for the stock status02
robots.txt rules for the three agentsNo disallow on product pathsNo disallow on product paths
Logged GPTBot requests to the price APINoneNone, and none needed

A zero in the first two rows means a non-rendering crawler gets a product name at best. Run the same request with the OAI-SearchBot and PerplexityBot strings from each operator's page, since a firewall or bot rule can treat them differently.

A request from your laptop only shows what the server returns to that user-agent string. It does not come from the operator's addresses, so finish in the server logs. Filter product URLs by each agent, match the source addresses against the published lists (OpenAI publishes https://openai.com/gptbot.json and https://openai.com/searchbot.json, Perplexity publishes https://www.perplexity.com/perplexitybot.json), and check the status code and response size. Then look for those agents calling the endpoint that supplies the price. A crawler that executed your JavaScript would have to call it, so if an agent fetched the product URL and never called that endpoint, your own logs are consistent with the study's finding. For Google, the JavaScript guide points to the Rich Results Test or the URL Inspection Tool, where you can look at the rendered HTML.

When the offer is missing, the fix Google recommends for its own crawler works for every client: render price and availability on the server, generate them statically, or hydrate a page whose first response already contains them, and put Product structured data in that same response. That is rendering work, the kind our headless web development service covers. Server rendering and markup make the offer readable, and they do not guarantee a citation, a recommendation or a rich result.

Readable pages are a precondition for generative engine optimization, and once the raw HTML passes, the question shifts from access to outcomes: whether assistants mention your products, which is what AI share of voice measures. Further resources if you want them: the AI visibility checker, the AI search playbook and our AI search optimization service. Start with the curl command, though. It takes a minute, and it replaces an assumption drawn from a 2024 study with a fact about your own product page.

Sources

  • OpenAI, Overview of OpenAI Crawlers
  • Vercel, The rise of the AI crawler (with MERJ, December 17, 2024)
  • Perplexity, Perplexity Crawlers
  • Google, Google's common crawlers
  • Google, Understand JavaScript SEO Basics
  • Google, Dynamic Rendering as a workaround
  • Google, AI Features and Your Website
  • Google, How To Add Merchant Listing Structured Data

Frequently asked questions

Does GPTBot execute JavaScript when it crawls a page?
OpenAI's crawler page says nothing about JavaScript either way. The study Vercel published with MERJ on December 17, 2024 found that the OpenAI, Anthropic, Meta, ByteDance and Perplexity crawlers it listed did not render JavaScript, with GPTBot, OAI-SearchBot and ChatGPT-User among them, while Gemini (through Googlebot) and AppleBot did. The ChatGPT and Claude crawlers fetched script files without executing them. We found no newer first-party statement, so test your own pages with a raw HTML request.
Is Google-Extended a crawler with its own user agent?
Google documents Google-Extended as a robots.txt product token that has no separate HTTP request user agent string. Crawling is done with existing Google user agent strings. The token controls whether content Google crawls may be used to train future Gemini models and for grounding in Gemini Apps and Vertex AI, so you will not see a Google-Extended user agent in server logs.
Does blocking Google-Extended remove product pages from Google Search or AI Overviews?
Google states that Google-Extended does not impact a site's inclusion in Google Search and is not used as a ranking signal. Its AI features guide names robots.txt rules for Googlebot as the crawl control for Search, with nosnippet, data-nosnippet, max-snippet and noindex to limit what is shown. Google-Extended is the control it points to for AI training and grounding in some of its other systems.
Can ChatGPT read a price that loads through JavaScript after the page opens?
In the 2024 MERJ and Vercel data, OpenAI's agents did not execute JavaScript, so a price requested by the browser after load would not reach them. The same study notes that content in the initial HTML response, JSON data included, may still be indexed. The safer design puts price and availability in the first response, both as visible text and as Product markup.
Is dynamic rendering a good way to show AI crawlers a JavaScript product page?
Google calls dynamic rendering a workaround and recommends server-side rendering, static rendering or hydration, because the workaround adds complexity and resource requirements. Sending prerendered HTML to crawler user agents would put the offer in front of bots that skip JavaScript, but it creates a second version of every product page that must stay identical on price and stock.

Free tools for this topic

FREE TOOLAI Brand Visibility MonitorDoes ChatGPT recommend you — or your competitor?CALCULATORROAS & Break-Even CalculatorKnow the ROAS you actually need before you scale.PLAYBOOKThe AI Search PlaybookGet cited by ChatGPT, Perplexity and Google AI Overviews.

Keep reading

GuidesHow to Get Cited by ChatGPT, Perplexity & AI OverviewsRead →GlossaryWhat Is AI Share of Voice? Measuring Brand Visibility in LLMsRead →GlossaryWhat Is GEO? Generative Engine Optimization, ExplainedRead →
CATALIST NEWSLETTER

Monthly dose of growth marketing.

Get marketing tips, narratives, guides, and playbooks delivered to your inbox.

Protected by reCAPTCHA — Google's Privacy Policy and Terms of Service apply.