JavaScript SEO: how Google crawls, renders and indexes JavaScript

How does Google crawl, render and index JavaScript?

Google describes three phases for JavaScript web apps: crawling, rendering and indexing. Googlebot fetches a URL after it checks robots.txt. Every page that returns a 200 status code is queued for rendering, and once Google’s resources allow, a headless Chromium renders the page and runs the JavaScript. Google then uses the rendered HTML to index the page and parses it again for links to crawl.

Older guides talk about “two waves of indexing”. Google’s current documentation describes the three phases above and a render queue instead. A page may stay in that queue for a few seconds, but it can take longer. Client-side content can therefore be indexed, just later than content that is already in the HTML.

Four rules follow from how that pipeline works:

• Blocked files are not rendered. If robots.txt blocks a script or the page, Google does not run that JavaScript.
• noindex can end the process early. When Google sees noindex in the HTML it may skip rendering, so a script that removes the tag later may never run.
• Only real links are discovered. Google can only find links that are <a> elements with an href attribute.
• HTML metadata wins. JavaScript can set the title, description and canonical, but HTML is preferred, and a canonical set by script must match the one in the HTML.

Which rendering strategy is best for JavaScript SEO?

The rendering strategy decides what the first response contains. Google copes with all of them, but many other crawlers do not. The table follows the definitions in web.dev’s guide to rendering on the web.

Google says dynamic rendering was a workaround and not a long-term solution and recommends server-side rendering, static rendering or hydration. Hydration adds client scripts to server-rendered HTML. It keeps the HTML complete for crawlers, but web.dev notes it can have a significant negative impact on Total Blocking Time and Interaction to Next Paint. For a framework-specific walkthrough, see our guide to Next.js SEO.

The choice matters beyond Google. Vercel’s analysis from December 2024 found that none of the major AI crawlers it examined rendered JavaScript, among them OpenAI’s, Anthropic’s and Perplexity’s, while Googlebot and Applebot did. Crawler behaviour changes, so treat that as a dated snapshot and check your own server logs. The safe rule is to ship the content, links and metadata that matter in the initial HTML.

What are the most common JavaScript SEO problems?

Most failures come from a handful of patterns. Each row names the symptom and the fix.

How do you test what Google actually sees?

Test in three layers: the HTML your server sends, the DOM after rendering, and Google’s own view. A mismatch between the first two is the most common finding.

1. 1

   Look at the delivered HTML

   Fetch the page without a browser and count the headings and links that are already there. If the counts are zero, everything depends on rendering.

   curl -s https://example.com/pricing -o raw.html
   grep -o "<h1" raw.html | wc -l
   grep -o "<a [^>]*href=" raw.html | wc -l

2. 2

   Render it in a real browser

   Save this small Playwright script as renderPage.mjs. It prints the full HTML after the scripts have run, including the doctype.

   import { chromium } from 'playwright'
   
   const browser = await chromium.launch()
   const page = await browser.newPage()
   await page.goto(process.argv[2], { waitUntil: 'networkidle' })
   process.stdout.write(await page.content())
   await browser.close()

3. 3

   Compare the two

   Run the same counts on the rendered file. Large differences in headings, links or words show what a non-rendering crawler misses.

   node renderPage.mjs https://example.com/pricing > rendered.html
   grep -o "<h1" rendered.html | wc -l
   grep -o "<a [^>]*href=" rendered.html | wc -l

4. 4

   Ask Google

   The URL Inspection tool shows loaded resources, JavaScript console output and exceptions, and the rendered DOM, as described in Google’s debugging guide. Run a live test and open “View tested page” to see how Google renders the URL. The Rich Results Test can also confirm that the rendered HTML contains your content.

Read the comparison like an auditor. These differences matter most:

• Headings and text: a missing H1 or a large gap in word count means the main content depends on rendering.
• Links: internal links that appear only in the rendered file will not be followed by a crawler that does not render.
• Title, description and canonical: values that change after rendering tell different crawlers different stories.
• Robots directives: a noindex or nofollow that appears or disappears after rendering is a bug that belongs in the HTML.

How does Serpel’s crawler render JavaScript pages?

A JavaScript SEO audit with Serpel’s site audit starts from the HTML the server delivers, then decides per page whether to render it. The project setting has three modes: “Automatic for empty pages” (the default), “Always render” and “Never render”. In automatic mode a page counts as a likely JavaScript shell when its delivered HTML has fewer than 50 words and shows a sign of a framework: a root container such as #__next, #__nuxt, #root or #app, Angular’s <app-root>, a <noscript> message about JavaScript, or several scripts with no links, no H1 and almost no text.

Pages that qualify are rendered in headless Chromium, using a mobile or desktop viewport that matches the crawl device. The renderer waits for the network to go idle and falls back to the load event if that times out. A crawl renders at most a fixed number of pages and reports when it reaches the limit. After rendering, the title, headings, links and the other checks run on the rendered page, and the crawl compares it with the raw HTML.

• Content only via JavaScript: the rendered page has at least 50 words and the raw HTML has less than half as many.
• Links only via JavaScript: at least 3 internal links exist only after rendering, and the raw HTML holds fewer than half of the rendered links.
• Metadata changed by JavaScript: title, description or H1 differ, or the canonical or robots directive differs between the two versions.
• Render problems: the page could not be rendered, or the browser console reported JavaScript errors.
• Rendering recommended: the page looks like an empty shell but was not rendered, because rendering is off, unavailable, over the limit or failed. Serpel reports this instead of false “missing title” or “missing H1” errors.

serpel projects update <project-id> --render-mode always
serpel crawl start --project <project-id> --wait
serpel audit issues --project <project-id> --category rendering

What should you do first?

1. Fetch the raw HTML of your main templates and check that the H1, text and navigation links are present.
2. Replace click handlers that navigate with real <a href> links.
3. Return real status codes: 404 or 410 for missing content, 301 for moved content.
4. Server-render or prerender everything you want indexed.
5. Make sure robots.txt does not block your scripts and stylesheets.
6. Crawl the site with rendering on and review the pages where raw and rendered HTML differ.