# Serpel > Serpel is an SEO tool for developers: rank tracking, site audits and AI visibility for ChatGPT and Google AI Overviews, in a dashboard, CLI, API and MCP server. Serpel is an SEO tool for developers. It tracks Google rankings, audits every page of a website, reads Search Console and analytics data, and checks whether ChatGPT and Google AI Overviews cite you. Use it from the dashboard, the CLI, the REST API or your coding agent over MCP. --- # Serpel: SEO that understands your codebase URL: https://serpel.app/ Serpel is an SEO tool for developers. It tracks Google rankings, audits every page of a website, reads Search Console and analytics data, and checks whether ChatGPT and Google AI Overviews cite you. Use it from the dashboard, the CLI, the REST API or your coding agent over MCP. ## Frequently asked questions ### What is a credit? Credits are the only currency in Serpel. One credit is worth €0.01, including VAT. Actions that call live data, like a rank check, a ChatGPT answer check, a keyword search or a crawl, book a fixed number of credits from a public price list. Looking at your data in the dashboard, the CLI or the API is free. ### Do credits expire? Credits you buy and your 100 start credits never expire. The monthly allowance of a Starter or Pro plan renews with every billing period, and the unused part of it expires at the end of that period. ### Which search engines and AI assistants does Serpel check? Rankings come from live Google results on desktop or mobile, down to position 100, nationwide or for a city or postal code. AI visibility covers ChatGPT with web search and Google’s AI Overviews. Search performance data comes from Google Search Console and Bing Webmaster Tools. ### Which countries and languages are supported? 43 countries and 22 languages, including the United States, the United Kingdom, Canada, Australia, India, most of Europe, Latin America and key markets in Asia, the Middle East and Africa. Each project has its own country and language, and every tracked keyword can have its own location. ### Does the crawler render JavaScript? Yes, when it is needed. If your server returns an empty app shell, Serpel renders the page in a headless browser and audits what visitors actually see, up to 200 pages per crawl. You can also render every page or turn rendering off per project. The crawler respects robots.txt and crawl-delay, keeps its request rate low and identifies itself with its own user agent. ### Can I use Serpel from the terminal or in CI? Yes. The CLI covers projects, rankings, audits, AI checks, reports and exports. Every command prints JSON with --json, exit codes are documented, and API tokens can be limited to 18 scopes and to specific projects. ### Does Serpel work with Claude Code, Cursor and other coding agents? Yes. Connect any agent with remote MCP support to serpel.app/mcp and sign in once. It can read rankings, search data, crawl issues and recommendations, and paid tools only run within the budget you approve. Agents that can run a terminal command can also use the Serpel CLI. Ready-made plugins for Claude Code, Cursor, Codex, GitHub Copilot and the Gemini CLI are on the roadmap. ### How does Serpel handle my data? Google Search Console is connected read-only through Google’s own sign-in. Every stored access key is encrypted with AES-256-GCM, API tokens are stored only as hashes, and each customer’s data is strictly separated. --- # Serpel pricing URL: https://serpel.app/pricing Start without a subscription and pay per credit, or choose a plan with a monthly credit allowance. One credit is worth €0.01, all prices are in euro and include VAT. Reading your data in the dashboard, CLI, API or over MCP is free. ## Plans | Plan | Price | Credits | Projects | Tracked keywords per project | AI prompts per project | Pages per crawl | | --- | --- | --- | --- | --- | --- | --- | | **Free** | €0 | 100 credits to start | 1 | 50 | 10 | 250 | | **Starter** | €19 per month | 2,000 credits every month | 5 | 200 | 25 | 1000 | | **Pro** | €49 per month | 6,000 credits every month | 20 | 500 | 50 | 1000 | **Free:** Start with one project and your start credits, then top up whenever you need more. No subscription needed. **Starter:** For one business or a handful of client sites that you check every few days. **Pro:** For agencies and teams who want daily rankings across many sites. ## Credit prices | Action | Unit | Credits | | --- | --- | --- | | Scheduled monitoring, up to top 50 | per keyword | 2 | | Scheduled monitoring, top 100 | per keyword | 3 | | Rank check right now, up to top 50 | per keyword | 5 | | Rank check right now, top 100 | per keyword | 9 | | ChatGPT answer check | per question | 2 | | Google AI Overview check | per question | 2 | | Keyword research | per search | 8 + 4 per 50 results | | Keyword research by domain | per domain | 4 + 2 per 50 results | | Site crawl | per 20 pages | 1 | --- # Everything you need to rank. In search and in AI answers. URL: https://serpel.app/features Serpel tracks your Google rankings, audits every page and checks whether ChatGPT and Google AI Overviews cite your website. Each feature works in the dashboard, the CLI, the REST API and over MCP. - [AI visibility](https://serpel.app/features/ai-visibility): Serpel is an AI visibility tool: it checks whether ChatGPT and Google AI Overviews cite your website for the questions your customers ask. - [AI Overview tracking](https://serpel.app/features/ai-overview-tracking): Serpel is an AI Overview tracker: see for which Google searches an AI Overview appears and whether it cites your website, checked with every rank check. - [Rank tracking](https://serpel.app/features/rank-tracking): Serpel is a keyword rank tracker for Google: live results on desktop and mobile, nationwide or local, with history, reports and an API. - [Mobile rank tracking](https://serpel.app/features/mobile-rank-tracking): Serpel is a mobile rank tracker: track Google rankings on mobile, desktop or both, set the device per project and keyword, and add locations and depth. - [Site audit](https://serpel.app/features/site-audit): Serpel runs a technical SEO audit with 94 checks, JavaScript rendering and Core Web Vitals, and explains every issue and its fix. - [Search data](https://serpel.app/features/search-data): See Google Search Console and Bing Webmaster Tools in one place: clicks, impressions, CTR and position per query and page, with history and an API. --- # An AI visibility tool for ChatGPT and Google AI Overviews. URL: https://serpel.app/features/ai-visibility Updated: 2026-10-10 An AI visibility tool shows whether AI assistants name or link your website when people ask about your topic. Serpel asks ChatGPT with web search and checks Google AI Overviews for the questions your customers ask, then records whether each answer cites you, mentions you or leaves you out. Results build up as a history in the dashboard, the CLI, the API and your coding agent. ## What Serpel tracks for AI visibility - **Prompts you choose:** Add the questions your customers ask an assistant. Each prompt keeps its own country and language and is checked on every run. - **ChatGPT with web search:** Every check asks ChatGPT with web search and records the answer, its sources and whether your domain is one of them. - **Google AI Overviews:** With AI Overview tracking on, the same prompt is searched on Google, so you see whether an Overview appears and whether it links you. - **Citations and sources:** See which domains an assistant cites most often, so you know who it trusts instead of you. - **AI crawler access:** Each crawl tests your robots.txt against 13 AI crawlers and looks for an llms.txt file. - **History and reports:** Follow citations per prompt over time. Every monitoring report lists new and lost citations and goes to email, Slack, Discord or a webhook. ## What is an AI visibility tool? An AI visibility tool measures whether AI assistants show your brand or link your website in their answers. A classic rank tracker reports a position on a results page. An assistant writes one answer with a handful of sources, so the useful questions change: were you cited, were you mentioned, and who was cited instead? You will also see these tools called AI visibility trackers, AI rank trackers or ChatGPT rank trackers, although a written answer has no position, only sources. Serpel covers 2 surfaces: ChatGPT with web search and Google AI Overviews. It doesn’t query other assistants such as Perplexity, Gemini or Claude, but it does check whether the crawlers of other assistants can reach your site. ## How does Serpel check ChatGPT and Google AI Overviews? | Surface | What Serpel does | What you see | | --- | --- | --- | | ChatGPT with web search | Asks your prompt in the country and language of the prompt. | Cited, mentioned or not mentioned, the cited URL, every source and an answer excerpt. | | Google AI Overviews | Searches the same prompt on Google, on your project’s device. | Whether an Overview appears, whether it links to you or mentions you, and its sources. | A **citation** means a source in the answer links to a page on your domain. A **mention** means your name or domain appears in the text. Cited answers count as mentioned too, and a failed check is recorded as failed, never as a miss. AI answers change from one request to the next, so a single result is a sample. Serpel stores every check, which lets you compare citations over time and ignore one-off outliers. The Google side has its own page: [AI Overview tracking](https://serpel.app/features/ai-overview-tracking). ## Which AI crawlers can reach your website? Citations need access. Each Serpel crawl tests your robots.txt against 13 AI crawlers, plus Googlebot and Bingbot, and checks whether an llms.txt file exists. A blocked AI search crawler raises a warning in the site audit, because the site can’t appear as a source there. | Purpose | Crawlers checked | How Serpel treats a block | | --- | --- | --- | | AI search | `OAI-SearchBot`, `Claude-SearchBot`, `PerplexityBot`, `MistralAI-Index` | Raises a warning in the site audit. | | User request | `ChatGPT-User`, `Claude-User`, `Perplexity-User` | Listed in the access report, not flagged. | | Training | `GPTBot`, `ClaudeBot`, `Google-Extended`, `Applebot-Extended`, `CCBot`, `Meta-ExternalAgent` | Listed in the access report, not flagged. You can block these separately. | Without an account, the free [robots.txt checker](https://serpel.app/tools/robots-txt-checker) runs the same crawler check for any domain. ## What does AI visibility tracking cost? | Check | Credits per prompt | Price per prompt | | --- | --- | --- | | ChatGPT with web search | 2 | €0.02 | | Google AI Overview | 2 | €0.02 | | Both together | 4 | €0.04 | Checking 25 prompts once a week on both surfaces uses about 429 credits a month, or €4.29. Prompts per project: 10 on Free, 25 on Starter and 50 on Pro. Reading results costs nothing, and all plans include the dashboard, CLI and API. See [pricing](https://serpel.app/pricing) for plans and credit packs. ## Where do you see the results? - **Dashboard:** an AI visibility page with citations per prompt, a ChatGPT result chart and the most cited sources. - **Reports:** new and lost citations in every monitoring report. - **CLI:** `serpel ai prompts`, `serpel ai history` and `serpel ai status` print the same data, or JSON with `--json`. - **API:** `GET /api/v1/projects/{id}/ai-visibility` returns the summary. The [REST API](https://serpel.app/developers/api) also lists prompts and their history. - **Coding agents:** `get_project_overview` over MCP includes AI visibility next to rankings and audit results. ## Start tracking AI visibility 1. **Add the questions your customers ask** Write prompts the way people phrase them to an assistant. Serpel uses the market of your project unless you choose another. ```bash serpel ai add --project --prompt "Which SEO tool has a CLI?" ``` 2. **Run a check** Check all prompts now, or let monitoring run them on your schedule. ```bash serpel ai run --project --wait ``` 3. **Read the results** One summary shows citations, mentions, the most cited sources, AI crawler access and llms.txt. ```bash serpel ai status --project ``` ## Frequently asked questions ### How do I check whether ChatGPT cites my website? Add the questions your customers ask as prompts in a Serpel project and run a check. Serpel asks ChatGPT with web search and reports whether a source links to your domain, whether your name is mentioned or whether you are missing. It repeats the check on every monitoring run, so you see changes over time. ### Does Serpel track Perplexity, Gemini or Claude? No. Serpel checks ChatGPT with web search and Google AI Overviews. The crawler check still covers the crawlers of other assistants, such as PerplexityBot and Claude-SearchBot, so you can see whether they are allowed to fetch your pages. ### How many prompts can I track? The limit depends on your plan: 10 on Free, 25 on Starter and 50 on Pro per project. Each check of one prompt uses 4 credits when Google AI Overview tracking is on, or 2 credits for ChatGPT alone. ### Why do AI answers differ between checks? Assistants can word an answer differently and pick different sources on each request. One check is a sample, not a verdict. Serpel keeps the history of every prompt, so you can judge the trend instead of a single result. --- # An AI Overview tracker for your Google keywords. Cited or not. URL: https://serpel.app/features/ai-overview-tracking Updated: 2026-10-10 An AI Overview tracker shows for which searches Google displays an AI Overview and whether it links to your website. Serpel records this with every rank check of a tracked keyword and with every AI visibility check of a prompt, so you see changes over time. AI Overview tracking is on by default and is included in the price of a keyword rank check. ## What AI Overview tracking records - **Appears or not:** Every check records whether Google showed an AI Overview for the search, so a missing Overview is data and not a gap. - **Cited or not:** Serpel records whether the Overview links a page of your domain and lists the sources it shows. - **Change over time:** Reports show new and lost citations from run to run, next to your position changes. ## What is an AI Overview tracker? An AI Overview is the AI-generated summary that Google can show for a search, with links to the sources it used. An AI Overview tracker watches a fixed set of searches and records, over time, whether an Overview appears and whether your website is one of its sources. Not every search triggers an Overview, so “no AI Overview” is a normal result. Serpel stores it as a result, which lets you tell a missing Overview apart from a check that has not run yet. ## How does Serpel track AI Overviews? Serpel tracks AI Overviews in two places: on the keywords you rank-track and on the prompts you track for AI visibility. | Aspect | Tracked keywords | AI visibility prompts | | --- | --- | --- | | When it runs | With every rank check of a keyword | With every AI visibility check of a prompt | | What is searched | The keyword, in its own country, language, device and location | The question, in its own market and on your project’s device | | What is recorded | Whether an Overview appears, whether it links your domain, and its sources | The same, plus whether your name or domain is mentioned and an answer excerpt | | Cost | Included in the rank-check price | 2 credits per prompt (€0.02) | ## What does the AI Overview column tell you? | Status | Meaning | | --- | --- | | Cited | Google showed an AI Overview and it links to a page on your domain. | | Not cited | There was an AI Overview, but it doesn’t link to your domain. | | No AI Overview | Google showed none for this search. | | No data | The keyword has not been checked yet, the check failed or tracking is switched off. | The status sits next to the position in the rankings table and in `serpel rankings list`. `serpel rankings history` shows the Overview status of every check next to the position. ## How do you switch AI Overview tracking on or off? The switch is a project setting called “Also check Google AI Overview”. It’s on for new projects. Turn it off in the project settings, with `serpel projects update --ai-overview off`, or through the API with `PATCH /api/v1/projects/{id}` and the field `aiOverviewTracking`. A keyword check up to the top 50 costs 2 credits when it runs on a schedule and 5 credits when you start it by hand, with or without the AI Overview. Only prompt checks add a separate charge. See [keyword rank tracking](https://serpel.app/features/rank-tracking) for the full price list. ## How do you read AI Overview data in reports and the API? - **Reports:** every monitoring report counts citations in AI Overviews next to ranking changes and ChatGPT citations. - **Rankings API:** each check carries an `aiOverview` object with `present`, `cited`, `url` and `sources`. Read it with `GET /api/v1/projects/{id}/rankings`. - **AI visibility API:** `GET /api/v1/projects/{id}/ai-visibility` summarises Overviews per keyword and per prompt. See the [rank tracking API](https://serpel.app/developers/rank-tracking-api). ## AI Overview tracker or AI visibility tool? Use AI Overview tracking to watch searches you already rank for. Use [AI visibility tracking](https://serpel.app/features/ai-visibility) to watch the questions your customers ask across ChatGPT and Google. Both run inside the same project and share one monitoring schedule. ## Frequently asked questions ### Does Google show an AI Overview for every search? No. Many searches have none. Serpel records “no AI Overview” as a result, so you can tell a missing Overview apart from a check that has not run yet or one that failed. ### Does AI Overview tracking cost extra credits? For tracked keywords it doesn’t: it’s included in the rank-check price. For AI visibility prompts, the Google AI Overview check adds 2 credits per prompt on top of the ChatGPT check. ### What counts as a citation in an AI Overview? An Overview cites you when it links to a page on your domain. For prompts, Serpel also records a mention when your name or domain appears in the text without a link. ### Can I track AI Overviews on mobile? Yes. Keyword checks use each keyword’s own device, so track the same keyword on desktop and mobile to compare them. Prompt checks use the device set in the project. --- # A keyword rank tracker for Google, desktop and mobile. URL: https://serpel.app/features/rank-tracking Updated: 2026-10-10 A keyword rank tracker checks where your website appears in Google for the keywords you care about and keeps the history. Serpel runs live Google searches on desktop or mobile, nationwide or for a single city, down to position 100. Checks run on a schedule you choose and end with a report. ## What the keyword rank tracker covers - **Desktop and mobile:** Each keyword is tracked on one device. Track the same keyword on both to compare the two results. - **Local rankings:** Check nationwide or from a city, postal code or region. The same keyword can be tracked for several places. - **Depth up to 100:** A keyword outside the checked depth reads “not in top N”. Serpel never guesses a position. - **Scheduled monitoring:** Check manually, daily, every 3 days or weekly at the hour you pick, and get a report after each run. - **Competitors:** Add up to 10 competitors and compare positions keyword by keyword from the stored results, at no extra cost. - **AI Overview column:** See whether Google shows an AI Overview for a keyword and whether it cites your website. ## What does a keyword rank tracker do? A keyword rank tracker records where your pages appear in Google for a list of keywords, every time it checks. Over weeks, the history shows which pages gain, which slip and which keywords you lose entirely. A SERP tracker goes one step further and keeps the results page itself: Serpel stores the organic results of the latest check, so you can see who ranks ahead of you. ## How does Serpel check a ranking? Each check is a live Google search for the keyword in its own country, language, device and location. Serpel finds the first result that belongs to your domain and stores the position, the URL and the time. If your domain isn’t within the checked depth, the keyword reads “not in top N” and the position stays empty. - The position, the ranking URL and the change since the previous check. - Whether Google showed an [AI Overview](https://serpel.app/features/ai-overview-tracking) and whether it cites you. - Search volume and CPC, added automatically for new keywords and refreshed after 30 days. This is a small separate credit charge that you can switch off per project. - The organic results down to the checked depth, with titles for the top 10. ## Which devices, locations and depths can you track? | Setting | Options | Where you set it | | --- | --- | --- | | Device | Desktop or mobile | Per keyword, with a default per project | | Location | Nationwide, or a city, postal code or region | Per keyword, with a default per project | | Market | 43 countries and 22 languages | Per project, and per keyword if needed | | Depth | Top 10, 20, 30, 50 or 100 | Per project | | Schedule | Manual, daily, every 3 days or weekly | Per project, at the hour you choose | Daily checks need the Pro plan. The page on [mobile rank tracking](https://serpel.app/features/mobile-rank-tracking) explains how the device and location settings work together. ## What does keyword rank tracking cost? | Check | Credits per keyword | Price per keyword | | --- | --- | --- | | Scheduled check, up to top 50 | 2 | €0.02 | | Scheduled check, top 100 | 3 | €0.03 | | Check started by hand, up to top 50 | 5 | €0.05 | | Check started by hand, top 100 | 9 | €0.09 | Scheduled runs cost less than checks you start by hand, because Serpel queues them with the data provider instead of waiting for a live result. This is what 100 keywords at depth 50 use in a month: | Schedule | Checks a month | Credits a month | Price a month | | --- | --- | --- | --- | | Weekly | 4.3 | 858 | €8.58 | | Every 3 days | 10 | 2,000 | €20.00 | | Daily | 30 | 6,000 | €60.00 | ## How many keywords can you track? | Plan | Projects | Keywords per project | Scheduled checks | | --- | --- | --- | --- | | Free | 1 | 50 | Weekly and every 3 days | | Starter | 5 | 200 | Weekly and every 3 days | | Pro | 20 | 500 | Daily, every 3 days and weekly | Each combination of keyword, country, language, device and location takes one slot, so tracking a keyword on desktop and on mobile uses two. ## How do you get rankings out of Serpel? - **Dashboard:** a rankings table with position, change, ranking URL, search volume and the history of every keyword. - **Reports:** a report after every run, by email to up to 10 recipients or to Slack, Discord and a webhook. - **CLI and exports:** `serpel rankings list`, `serpel rankings history` and `serpel export --type rankings`. - **API:** the [rank tracking API](https://serpel.app/developers/rank-tracking-api) returns positions as JSON. - **Coding agents:** `get_keyword_rankings` over MCP. ## Start tracking in three steps 1. **Add keywords** Add one or more keywords. They take the country, language, device and location of the project unless you set them. ```bash serpel rankings add --project --keyword "keyword rank tracker" --keyword "serp tracker" ``` 2. **Run a check** Start a check and wait for it to finish. The first check gives every keyword its first result. ```bash serpel rankings run --project --wait ``` 3. **Schedule monitoring** Let Serpel check on a schedule. Each run ends with a report that compares positions with the last one. ```bash serpel projects update --schedule weekly --schedule-hour 6 ``` ## Frequently asked questions ### What is the difference between a rank tracker and Search Console? Search Console reports the average position of queries that already earned impressions. A rank tracker checks the keywords you choose on a fixed schedule, including ones where you don’t rank yet. Serpel shows both, and you can connect [Search Console](https://serpel.app/features/search-data) to the same project. ### How often does Serpel check rankings? You choose: manually, daily, every 3 days or weekly, at the hour you pick. Daily checks need the Pro plan. You can also start a check at any time. ### Can I track rankings for a city? Yes. Search for a location by name or postal code and set it for a keyword or as the project default. The same keyword can be tracked for several places, and each place uses its own slot. ### Why can my own Google results differ from Serpel’s? Positions come from live Google searches for the keyword, device and location you set. Personalisation, your real location and the time of the search can still make your own results differ. --- # A mobile rank tracker for Google’s mobile results. URL: https://serpel.app/features/mobile-rank-tracking Updated: 2026-10-10 A mobile rank tracker records where your pages rank in Google’s mobile results, which can differ from the desktop ones. Serpel stores a device with every keyword, so you can track a keyword on mobile, on desktop or on both. Each project has a default device, and you add a location and a depth on top. ## How mobile tracking works in Serpel - **A device per keyword:** Every keyword is stored with its own device and keeps its own position history. - **Locations:** Track nationwide or from a city, postal code or region, on mobile just as on desktop. - **Depth:** Check the top 10, 20, 30, 50 or 100 results. The depth is a project setting and applies to both devices. ## Why do you need a mobile rank tracker? Results can differ between desktop and mobile for the same keyword, so a tracker that checks one device can miss changes that your other visitors see. Serpel therefore treats the device as part of the keyword. The same keyword on mobile and on desktop is tracked as two entries, each with its own history, URL and position. Mobile rank tracking is not a separate product in Serpel. It is the [keyword rank tracker](https://serpel.app/features/rank-tracking) with the device set to mobile, so schedules, reports, competitors, the API and the CLI all work the same way. ## How does Serpel set the device for a project? Every project has a default device, desktop or mobile. A new project starts on desktop. New keywords take the default unless you choose a device when you add them. Each keyword keeps the device it was added with, so changing the project default later doesn’t change keywords you already track. | Situation | Device that is used | | --- | --- | | You add a keyword and choose a device | The device you chose | | You add a keyword without a device | The project’s default device | | You change the project default later | Existing keywords keep their device, new keywords take the new default | | Serpel checks a prompt on Google | The project’s default device | That last row is why the default matters beyond rankings: [AI Overview](https://serpel.app/features/ai-overview-tracking) results for your prompts are searched on the project’s device. ## How do you track a keyword on mobile and on desktop? Set the device with `--device` when you add a keyword, or with the field `device` in the API. To track one keyword on both devices, add it twice. ```bash serpel projects update --device mobile serpel rankings add --project --keyword "mobile rank tracker" --device mobile serpel rankings add --project --keyword "mobile rank tracker" --device desktop ``` In the REST API, `POST /api/v1/projects/{id}/rankings` accepts `device` next to `country`, `language` and `locationCode`, and `PATCH /api/v1/projects/{id}` sets the project default `rankDevice`. The [rank tracking API](https://serpel.app/developers/rank-tracking-api) page lists every endpoint. ## How do locations and depth work with a mobile rank tracker? | Setting | Options | Where the default comes from | | --- | --- | --- | | Device | Desktop or mobile | The project’s default device | | Location | Nationwide, or a city, postal code or region | The project’s default location | | Country and language | 43 countries and 22 languages | The project’s market | | Depth | Top 10, 20, 30, 50 or 100 | A project setting that applies to every keyword | Search for a place by name or postal code, then set it for a keyword or as the project default. Serpel prints the location it picked, so you can confirm it before the first check. ```bash serpel rankings locations Bochum serpel rankings add --project --keyword "eyebrow studio" --device mobile --location 44789 ``` A keyword outside the checked depth reads “not in top N” instead of a guessed position. The same keyword can be tracked for several places, and each place uses its own slot. ## What does mobile rank tracking cost? A mobile check costs the same as a desktop check. A scheduled check up to the top 50 uses 2 credits per keyword (€0.02), and a check you start by hand uses 5. Tracking one keyword on both devices uses two slots and two checks. All prices are on the [pricing](https://serpel.app/pricing) page. ## Frequently asked questions ### Is mobile rank tracking separate from desktop? Yes. Each keyword carries a device, and each device has its own position history. Track a keyword on both devices to compare them side by side. ### How do I change the device of an existing keyword? A keyword’s device is fixed when you add it. To switch, add the keyword again with the other device and remove the old entry if you no longer need its history. Removing an entry deletes its history. ### Why does my mobile position differ from what I see on my phone? Positions come from live Google searches for the keyword, device and location you set. Personalisation, your real location and the time of the search can still make your own results differ. ### Does the site audit check the mobile version of my site? By default, the crawler fetches pages as a mobile browser and identifies itself as SerpelBot. It also measures Core Web Vitals on mobile and on desktop, and it flags a missing viewport tag. --- # A technical SEO audit that renders JavaScript. URL: https://serpel.app/features/site-audit Updated: 2026-10-10 A technical SEO audit crawls your website like a search engine and reports what stops pages from being found, indexed or shown well. Serpel runs 94 checks in 18 categories, renders JavaScript when a page needs it and measures Core Web Vitals. Every issue says why it matters and how to fix it. ## What the site audit covers - **An issue catalogue:** 11 error, 43 warning and 40 notice checks. Each one comes with the reason it matters and a fix. - **JavaScript rendering:** Pages that arrive almost empty are rendered in a headless browser and audited as visitors see them. Choose automatic, always or never per project. - **Core Web Vitals:** LCP, INP and CLS for the start page and key pages on mobile and desktop, with lab and field values. - **Crawl comparison:** Compare two crawls to see new, fixed and changed issues, and the score before and after. - **AI search checks:** Your robots.txt is tested against 13 AI crawlers, and Serpel looks for an llms.txt file. - **A polite crawler:** SerpelBot respects robots.txt and crawl-delay, keeps its request rate low and identifies itself with its own user agent. ## What does a technical SEO audit check? A technical SEO audit looks at the parts of a site that search engines and AI crawlers depend on: whether pages respond, whether they can be indexed, whether titles and headings are in place, how pages link to each other and how fast they load. Serpel’s audit is a crawl of your own domain that starts at your start URL and follows internal links up to the page limit you set, within the maximum of your plan. People also call this a website SEO checker. The difference to a one-page checker is scope: Serpel covers the whole site and reports issues by type, with the number of affected URLs. | Group | Checks | What it finds | | --- | --- | --- | | Availability and redirects | 10 | Client and server errors, unreachable URLs, soft 404s, redirect loops and redirect chains. | | Links | 10 | Broken internal and external links, orphan pages and pages buried deep in the site. | | Titles, descriptions and headings | 12 | Missing, duplicate, too long or too short titles and descriptions, and missing or repeated H1s. | | Indexing and crawling | 16 | noindex, canonical conflicts, robots.txt and sitemap problems, and blocked search crawlers. | | Content and images | 7 | Duplicate and thin content, a missing page language, missing alt text and image dimensions. | | HTTPS, mobile and speed | 9 | Missing HTTPS, mixed content, a missing viewport tag, slow responses and oversized pages. | | Structured data, social and international | 13 | Invalid structured data, missing Open Graph and Twitter cards, and hreflang errors. | | JavaScript rendering | 8 | Content, links or meta tags that only exist after JavaScript runs, and pages that fail to render. | | Core Web Vitals | 7 | LCP, INP and CLS values that need improvement or are poor. | | AI search | 2 | Blocked AI search crawlers and a missing llms.txt. | 15 of the checks are heuristics, such as title length or thin content. They are marked “heuristic” in the results, so you can tell a rule of thumb from a confirmed error. ## How does Serpel audit JavaScript websites? Serpel first fetches the HTML your server delivers. If a page arrives almost empty but shows traces of a JavaScript framework, it renders the page in a headless browser and audits the result. It compares the delivered HTML with the rendered page, so it can flag content, links, titles, canonicals and robots directives that only exist after JavaScript runs. - **Automatic for empty pages:** the default. Only pages delivered almost empty are rendered. - **Always render:** every page is rendered. - **Never render:** only the delivered HTML is audited. Rendering is capped per crawl, and a page that can’t be rendered is reported as such, so the audit never claims more than it measured. Change the mode in the project settings or with `serpel projects update --render-mode always`. ## How does Serpel measure Core Web Vitals? At the end of a crawl, Serpel measures the start page and key pages on mobile and desktop with Google PageSpeed Insights and rates LCP, INP and CLS against Google’s thresholds. | Metric | Good | Poor | | --- | --- | --- | | Largest Contentful Paint (LCP) | Up to 2,500 ms | Above 4,000 ms | | Interaction to Next Paint (INP) | Up to 200 ms | Above 500 ms | | Cumulative Layout Shift (CLS) | Up to 0.1 | Above 0.25 | You get lab values from a test run and, where Google has enough data, field values from real users over the last 28 days. INP exists only as a field value. ## How big a crawl can you run, and what does it cost? | Plan | Maximum pages per crawl | | --- | --- | | Free | 250 | | Starter | 1,000 | | Pro | 1,000 | New projects crawl up to 100 pages by default, and you can raise the limit up to your plan’s maximum, for example with `serpel projects update --max-pages 500`. A crawl costs 1 credit per 20 pages, so 100 pages cost 5 credits and 1,000 pages cost 50 credits, or €0.50. Reading the results is free, and crawls can also run weekly on a schedule. See [pricing](https://serpel.app/pricing) for plans and credit packs. ## What happens after the audit? - **Score and issues:** a score from 0 to 100, plus issues grouped by severity with the affected URLs, the reason and the fix. - **Tasks:** create a task from any issue with `serpel tasks create --project --issue title_missing`. The next crawl rechecks it. - **Comparison:** `serpel crawl compare` lists new, fixed and changed issues between two crawls. - **Exports and API:** download issues as CSV or JSON, or read them through the [REST API](https://serpel.app/developers/api). - **Search data:** read the audit next to [Search Console and Bing data](https://serpel.app/features/search-data) in the same project. ## Run your first audit 1. **Create a project** Add your domain. Serpel uses it as the start URL for the crawl. ```bash serpel projects create --domain example.com --country US --language en ``` 2. **Start a crawl** The crawl runs in the background. With `--wait`, the command ends with a summary of pages and issues. ```bash serpel crawl start --project --wait ``` 3. **Read the errors first** List the issues of the latest crawl, starting with errors. ```bash serpel audit issues --project --severity error ``` 4. **Fix, crawl again and compare** After your fixes, run another crawl and compare it with the previous one. ```bash serpel crawl compare --project ``` ## Frequently asked questions ### Is Serpel a website SEO checker? Yes. Create a project for your domain and start a crawl. You get a score from 0 to 100, issues grouped by severity and a fix for each. The crawl covers the whole site up to the page limit you set, within your plan’s maximum, not a single page. ### Does the audit work for single-page apps and server-rendered sites? Yes. Pages delivered almost empty are rendered in a headless browser, and Serpel reports differences between the delivered HTML and the rendered page. Server-rendered pages are audited from their HTML. ### How is the audit score calculated? The score starts at 100 and loses points for each issue type, weighted by severity and by the share of HTML pages affected. Site-wide issues count in full. Compare two crawls to see how a fix changed the score. ### Does the crawler respect robots.txt? Yes. SerpelBot follows robots.txt and crawl-delay, keeps its request rate low and identifies itself with its own user agent. The crawler is described on the [Serpel crawler page](https://serpel.app/bot). --- # Search Console and Bing Webmaster Tools in one place URL: https://serpel.app/features/search-data Updated: 2026-10-10 Serpel shows Google Search Console and Bing Webmaster Tools in one place: a single page per project with clicks, impressions, click-through rate and average position, and a switch between Google and Bing. You connect each account once, Serpel matches properties to your projects by domain, and reading the data costs no credits. The same numbers are available in the CLI, the REST API and your coding agent over MCP. ## What the search data view gives you - **Queries and pages:** Clicks, impressions, click-through rate and average position for every search query and every page. - **Period comparison:** Choose 7, 28 or 90 days. Totals are compared with the previous period of the same length, and a daily chart shows the trend. - **Connect once:** Google connects with a read-only sign-in and Bing with an API key. Serpel matches properties to projects by domain. - **A daily history for Google:** Serpel syncs Google data every day and keeps 16 months of history, so reports read from storage. - **Quick wins:** Queries on positions 4 to 20 with impressions show where a small improvement pays off first. - **CLI, API and MCP:** Read the same data with `serpel search queries`, a REST endpoint or the `get_search_performance` tool. ## How do you see Search Console and Bing Webmaster Tools in one place? Create a project for your domain and connect the accounts that hold its data. Google connects with Google’s own sign-in and gives Serpel read-only access. Bing connects with an API key from Bing Webmaster Tools. Account connections are matched to projects by the domain of the property, so a new project finds its data without further setup. Each project has one connection per provider, and the Search data page switches between Google and Bing. Stored access keys are encrypted with AES-256-GCM. | Aspect | Google Search Console | Bing Webmaster Tools | | --- | --- | --- | | How you connect | Google’s own sign-in, read-only | An API key from Bing Webmaster Tools | | What you choose | The matching property | The matching website | | History in Serpel | Synced daily, kept for 16 months | None, always read live | | Daily chart | Every day, with empty days filled in | Only days that have data | ## What data does Serpel show? - **Clicks, impressions, click-through rate and average position** as totals, per search query and per page. - **7, 28 or 90 days**, compared with the previous period of the same length, plus a daily chart. - **A tracked marker:** queries that you already track as keywords are marked, and you can add any query to rank tracking from the list. - **CSV export** of the query list. - **Analytics next to it:** connect PostHog, Plausible or Google Analytics 4 to see visitors and conversions per landing page. ## How fresh is the data, and how long is it kept? Both providers deliver data with a delay of two to three days, and Serpel caches results for six hours. The reporting period therefore ends two days before today. For Google, Serpel also keeps its own daily history. The first sync loads the last 90 days, and every daily sync adds the most recent 3 days. Serpel stores the 5,000 rows with the most impressions per day and keeps them for 16 months. When the stored history covers a period without gaps, reports read from it. Otherwise they ask Google live. Bing data is always read live. ## How do you get Search Console data through an API? Serpel serves the same data through its REST API, so you can read Search Console and Bing numbers without building your own OAuth flow. One endpoint returns clicks, impressions, CTR and position for either provider. ```bash curl "https://serpel.app/api/v1/projects/$PROJECT_ID/search-performance?provider=google&dimension=query&days=28&limit=25" \ -H "Authorization: Bearer $SERPEL_TOKEN" ``` The parameter `provider` is required and is `google` or `bing`. `dimension` is `query` or `page`, `days` is 7, 28 or 90, and `limit` goes up to 500. The token needs the `projects:read` scope. The CLI offers `serpel search queries` with the same options, and `--json` prints the raw response. The [REST API](https://serpel.app/developers/api) page lists the other endpoints. ## What can you do with search data in Serpel? - **Find quick wins:** queries on positions 4 to 20 with impressions, available as `find_keyword_opportunities` over MCP. - **Get recommendations:** Serpel combines search data, rankings, crawl results, analytics and your code into prioritised opportunities, such as pages with a low click-through rate for their position. - **Compare with rank tracking:** search data shows queries you don’t track yet, while [rank tracking](https://serpel.app/features/rank-tracking) checks the keywords you chose. - **Ask your agent:** `get_search_performance` returns the same totals and top rows to a coding agent over [MCP](https://serpel.app/developers/mcp). ## Frequently asked questions ### Can I connect Google Search Console and Bing Webmaster Tools to the same project? Yes. Each project can have one Google property and one Bing website. A switch on the Search data page changes between them, and both show the same measures. ### Is Serpel’s access to Search Console read-only? Yes. Google access uses the read-only Search Console scope, and Serpel only reads data. Every stored access key is encrypted with AES-256-GCM, and you can disconnect a provider at any time. ### Does reading search data use credits? No. Syncing and reading Search Console and Bing data is free. Credits are only used for live data such as rank checks, AI visibility checks, keyword research and crawls. ### Why is the latest data two or three days old? Google and Bing deliver search performance data with a delay of two to three days. Serpel ends the reporting period two days before today for that reason, and it caches results for six hours. --- # SEO in your terminal. And in your coding agent. URL: https://serpel.app/developers Serpel gives developers three ways in: a REST API, a CLI with JSON output and an MCP server for coding agents. All three use the same projects, data and credits as the dashboard. - [Rank tracking API](https://serpel.app/developers/rank-tracking-api): Serpel’s rank tracking API adds keywords, runs Google rank checks and returns positions as JSON, for desktop and mobile, with locations and history. - [SEO API](https://serpel.app/developers/api): Serpel’s SEO API is a REST API for rank tracking, site audits, AI visibility and search data, with bearer tokens, JSON responses and credit-based pricing. - [MCP server](https://serpel.app/developers/mcp): Serpel’s SEO MCP server gives Claude Code, Cursor and other agents your rankings, audits and search data, with OAuth and a budget for paid tools. - [CLI](https://serpel.app/developers/cli): The Serpel SEO CLI runs rank checks, site audits and AI visibility checks from your terminal, with JSON output, documented exit codes and CI support. --- # A rank tracking API for Google positions as JSON. URL: https://serpel.app/developers/rank-tracking-api Updated: 2026-10-10 A rank tracking API lets your code add keywords, start rank checks and read Google positions without a dashboard. Serpel’s REST API returns positions for desktop and mobile as JSON, with history, ranking URL and AI Overview status, and it uses the same projects and credits as the app. Authenticate with a bearer token that carries the `rankings:read` and `rankings:write` scopes. ## What the rank tracking API covers - **REST and JSON:** Resource-style endpoints under `/api/v1` with a consistent `data`, `pagination` and `error` format. - **Device and location per keyword:** Set country, language, device and a Google location code for every keyword you add. - **Asynchronous checks:** Start a check and follow it as a job, or let scheduled monitoring run it for you. - **History and result pages:** Read the position history of a keyword and the stored organic results of its last check. - **Scoped tokens:** Limit a token to rank tracking, to chosen projects and to an expiry date. - **Estimates before you spend:** Ask `POST /billing/estimate` for the credit cost of a check before you start it. ## What is a rank tracking API? A rank tracking API returns search positions as data. Instead of opening a dashboard, your script, CI job or reporting tool asks for the positions of your keywords and gets them as JSON. Serpel’s API is a REST API under `/api/v1`. The dashboard and the CLI call the same endpoints with the same permission checks. Whether you call it a rank tracker API, an SEO rank API or a Google rank tracking API, the flow is the same: add keywords, trigger checks, read positions. The page on [keyword rank tracking](https://serpel.app/features/rank-tracking) explains what each check records. ## Which endpoints cover rank tracking? | Endpoint | What it does | Scope | | --- | --- | --- | | `POST /projects/{id}/rankings` | Adds keywords with country, language, device and location. | `rankings:write` | | `POST /projects/{id}/rankings/run` | Starts a check for all or selected keywords and returns a job. | `rankings:write` | | `GET /jobs/{id}` | Reports the job status: queued, running, succeeded, failed or cancelled. | `rankings:read` | | `GET /projects/{id}/rankings` | Lists tracked keywords with latest position, change and ranking URL. | `rankings:read` | | `GET /projects/{id}/rankings/summary` | Totals: tracked, top 3, top 10, not found and average position. | `rankings:read` | | `GET /rank-keywords/{id}/history` | Returns the position history of one keyword. | `rankings:read` | | `GET /rank-keywords/{id}/serp` | Returns the stored organic results of the last check. | `rankings:read` | | `GET /locations` | Finds location codes for cities, postal codes and regions. | `rankings:read` or `projects:read` | | `GET /projects/{id}/competitors/comparison` | Compares your positions with competitors, keyword by keyword. | `rankings:read` | All paths start with `/api/v1`. The [REST API](https://serpel.app/developers/api) page covers the other areas of the API. ## How do rank checks run? Checks are asynchronous. `POST /projects/{id}/rankings/run` answers with `202` and a job, and the response field `alreadyRunning` tells you when a check for the project is already in progress. Poll `GET /jobs/{id}` until the status is `succeeded`, `failed` or `cancelled`, then read the keyword list again. Scheduled monitoring uses the same jobs. Once a project has a schedule, its keywords are checked daily, every 3 days or weekly without any API call, and every run ends with a report. ## How do you read a position? | Field | Meaning | | --- | --- | | `latest.position` | The position of your domain. It is `null` when the domain was not found within the checked depth. | | `latest.depth` | How many results the check looked at. | | `latest.url` | The URL of your page that ranks. | | `change` | The difference to the previous check. A positive number means the keyword moved up. | | `latest.aiOverview` | Whether Google showed an AI Overview and whether it cites your domain. | | `device` and `locationName` | The device and the location the keyword is checked for. | | `searchVolume` and `cpc` | Estimates added for the keyword, `null` until they are available. | ## How does authentication work? Send an API token as a bearer token. Create one in the dashboard under Settings, API tokens, or with `serpel tokens create`. Serpel has 18 scopes. Rank tracking needs `rankings:read` to read and `rankings:write` to add keywords and start checks. A token can be limited to specific projects and given an expiry date. Tokens begin with `vsk_` and are shown only once. ## What does the rank tracking API cost? Reading positions is free. A check books credits per keyword from the public price list. `POST /billing/estimate` returns the cost of an operation without booking it, and a request that your balance can’t cover fails with `402` and `insufficient_credits`. | Check | Credits per keyword | Price per keyword | | --- | --- | --- | | Started with `POST /projects/{id}/rankings/run`, up to top 50 | 5 | €0.05 | | Started with `POST /projects/{id}/rankings/run`, top 100 | 9 | €0.09 | | Scheduled run, up to top 50 | 2 | €0.02 | | Scheduled run, top 100 | 3 | €0.03 | Plans and credit packs are on the [pricing](https://serpel.app/pricing) page. The page on [mobile rank tracking](https://serpel.app/features/mobile-rank-tracking) shows how the device setting works. ## How does the API report errors and limits? - Errors use one format: `{ "error": { "code", "message", "details" } }`. - `400 validation_error` means invalid input. `401 unauthorized` means a missing or invalid token. - `402 insufficient_credits` carries `details.available` and `details.required`. - `403 forbidden` means a missing scope. `403 plan_limit` means a plan limit, such as the keyword limit per project, was reached. - `429 rate_limited` carries a `Retry-After` header and `details.retryAfterSeconds`. Calls that cost credits are limited more tightly than reads. ## Your first rank check through the API 1. **Create a token** Create a token with the two rank tracking scopes, limited to one project. The secret is shown once, so store it as a secret in your CI system. ```bash serpel tokens create --name rank-api --scope rankings:read --scope rankings:write --project ``` 2. **Poll the job** After the `rankings/run` request, poll the job until its status is `succeeded`. ```bash curl "https://serpel.app/api/v1/jobs/$JOB_ID" \ -H "Authorization: Bearer $SERPEL_TOKEN" ``` 3. **Read the summary** The summary returns totals for the whole project in one call. ```bash curl "https://serpel.app/api/v1/projects/$PROJECT_ID/rankings/summary" \ -H "Authorization: Bearer $SERPEL_TOKEN" ``` ## Frequently asked questions ### Is there a free rank tracking API? Serpel’s API is part of every plan, including the free one, and reading data costs nothing. Checks cost credits at the public price list, and new accounts start with 100 credits. ### Does the API return live or stored positions? The list endpoints return the result of the latest check that Serpel stored. To get fresh positions, start a check with `POST /projects/{id}/rankings/run` and read the list again when the job has succeeded. Each check also reports whether it came from a cache in the field `cached`. ### Can I get rankings for a city through the API? Yes. Use `GET /locations?country=US&q=austin` to find a location code and pass it as `locationCode` when you add keywords. Without a code, the project’s default location applies, or nationwide results if the project has none. ### Can I track mobile and desktop through the API? Yes. Pass `device` as `desktop` or `mobile` when you add a keyword. Each keyword keeps its own device, so add the keyword twice to track both. --- # An SEO API for rankings, audits and search data. URL: https://serpel.app/developers/api Updated: 2026-10-10 An SEO API gives your code access to rankings, audit results and search data without a dashboard. Serpel’s REST API covers rank tracking, site audits, AI visibility, keyword research, search data and reports under `/api/v1`, with JSON responses and bearer-token authentication. It is the same API that the dashboard and the CLI use. ## What the SEO API gives you - **One API, every area:** Projects, rankings, audits, AI visibility, search data, keyword research, reports and billing sit behind the same base path. - **Scoped tokens:** 18 scopes, optional project limits and expiry dates let you give a script only what it needs. - **Jobs for long work:** Crawls, rank checks, AI checks and monitoring runs return a job that you can poll. - **Predictable responses:** A `data` envelope, `pagination` on lists and one error format across every endpoint. - **Costs you can see:** Estimate an operation before you run it and read the credit ledger afterwards. - **A machine-readable index:** `GET /api/v1` lists every endpoint by area and needs no authentication. ## What can you do with the Serpel SEO API? The API follows the structure of the product. Each area has endpoints to read data and, where it makes sense, to start work. The dashboard and the CLI call the same endpoints with the same permission checks. | Area | What you can do | Main scopes | | --- | --- | --- | | Projects | Create projects, change settings such as device, depth, schedule and JavaScript rendering, and manage members. | `projects:read`, `projects:write` | | Rank tracking | Add keywords, run checks, read positions, history, stored results, locations and competitors. | `rankings:read`, `rankings:write` | | Site audit | Start crawls, read pages, issues, Core Web Vitals and crawl comparisons. | `audit:read`, `audit:write` | | AI visibility | Add prompts, run ChatGPT and AI Overview checks, read citations and sources. | `rankings:read`, `rankings:write` | | Search data | Connect Google and Bing, read clicks, impressions, CTR and position. | `projects:read`, `projects:write` | | Keyword research | Research keywords, manage keyword lists, import and export. | `keywords:read`, `keywords:write` | | Recommendations | Read prioritised opportunities and change their status. | `recommendations:read`, `recommendations:write` | | Codebase | Upload and read route snapshots of a Next.js project. | `codebase:read`, `codebase:write` | | Reports and monitoring | Run monitoring, read reports, manage recipients and webhooks. | `rankings:write`, `projects:read`, `projects:write` | | Billing | Read the balance and ledger, estimate costs and manage agent spending. | `billing:read`, `billing:write` | ## How do you run a site audit through the API? `POST /projects/{id}/crawls` starts a crawl and answers with `202`, a job and the flag `alreadyRunning`. Poll `GET /jobs/{id}` until the job has succeeded, then read `GET /projects/{id}/issues`. It returns the score, the counts by severity and one group per issue type with `why`, `fix` and the number of affected URLs. Add `severity`, `category` or `q` to filter. For detail, `GET /crawls/{id}/pages` lists the crawled URLs and `GET /crawls/{id}/pages/{pageId}` returns links, issues and signals for rendering, hreflang, structured data and soft 404s. `GET /crawls/{id}/web-vitals` returns Core Web Vitals per URL and device, and `GET /projects/{id}/crawls/compare` shows new, fixed and changed issues between two crawls. The [site audit](https://serpel.app/features/site-audit) page explains the 94 checks. ## How do authentication, responses and errors work? Send an API token as `Authorization: Bearer `. Create tokens in the dashboard under Settings, API tokens, or with `serpel tokens create`. A token carries only the scopes you choose, can be limited to specific projects and can expire. Tokens begin with `vsk_` and are shown once. OAuth access tokens issued for the MCP server are rejected by the REST API. Responses use a `data` envelope. List responses add `pagination` with `limit`, `offset` and `total`, and accept `limit`, `offset` and, on most lists, `q` for search. Errors share one format with `code`, `message` and `details`. | HTTP status | Error code | | --- | --- | | 400 | `validation_error`, `bad_request` | | 401 | `unauthorized` | | 402 | `insufficient_credits` | | 403 | `forbidden`, `plan_limit` | | 404 | `not_found` | | 409 | `conflict` | | 429 | `rate_limited`, with a `Retry-After` header | | 502 and 503 | `provider_error`, `provider_not_configured` | Paid synchronous calls, such as keyword research, accept an `Idempotency-Key` header. A retry with the same key neither calls the data provider nor books credits again. ## What does the SEO API cost? Reading data is free. Calls that fetch live data book credits from a public price list. One credit is worth €0.01 including VAT, and new accounts start with 100 credits. | Operation | Unit | Credits | | --- | --- | --- | | Scheduled monitoring, up to top 50 | per keyword | 2 | | Scheduled monitoring, top 100 | per keyword | 3 | | Rank check right now, up to top 50 | per keyword | 5 | | Rank check right now, top 100 | per keyword | 9 | | ChatGPT answer check | per question | 2 | | Google AI Overview check | per question | 2 | | Keyword research | per search | 8 + 4 per 50 results | | Keyword research by domain | per domain | 4 + 2 per 50 results | | Site crawl | per 20 pages | 1 | `POST /billing/estimate` returns the cost of an operation without booking it, and `GET /billing/ledger` lists every booking. A request that your balance can’t cover fails with `402`. Plans and credit packs are on the [pricing](https://serpel.app/pricing) page. ## Where do you find the endpoint list? `GET /api/v1` returns an index of all endpoints grouped by area, and `GET /api/health` reports whether the service and its database are up. Neither needs authentication. For rankings in particular, see the [rank tracking API](https://serpel.app/developers/rank-tracking-api). ## Frequently asked questions ### Can I use the Serpel SEO API on the free plan? Yes. API access is part of every plan, including the free one. You only pay credits for calls that fetch live data, such as rank checks, AI checks, keyword research and crawls. ### Which API calls start long-running work? Crawls, rank checks, AI visibility checks and monitoring runs return `202` and a job. Poll `GET /jobs/{id}` for the status. Keyword research and keyword list refreshes run synchronously and accept an `Idempotency-Key`. ### Can an AI agent use the API? Yes, with a scoped token. Agents that speak MCP can also use the [MCP server](https://serpel.app/developers/mcp), which signs in with OAuth and exposes read-only tools, so no token has to be copied into a configuration file. ### How does Serpel handle rate limits? Requests above the limit get `429` with `rate_limited`, a `Retry-After` header and `details.retryAfterSeconds`. Calls that cost credits are limited more tightly than reads, so build retries around the header. --- # An SEO MCP server for Claude Code, Cursor and Codex. URL: https://serpel.app/developers/mcp Updated: 2026-10-10 An SEO MCP server lets a coding agent read your rankings, search data and audit results as tools instead of copying numbers into a prompt. Serpel’s remote MCP server at `serpel.app/mcp` offers 13 tools and signs you in with OAuth in the browser. Of these, 12 are free and read-only, and the one paid tool runs only within the budget you approve. ## What the Serpel MCP server gives your agent - **OAuth sign-in:** The agent opens your browser once. You choose the workspace and approve access. No API key goes into a configuration file. - **Read-only tools:** 12 of 13 tools only read data. Each checks scope, project role and workspace, like the dashboard. - **A budget for paid tools:** `research_keywords` is the only tool that spends credits, and only within the limits you set. - **Jobs followed by ID:** Crawls and rank checks start in the dashboard, CLI or API. The agent follows them with `get_job_status`. - **Code-aware findings:** `get_codebase_overview` connects SEO findings to the routes you uploaded with `serpel scan`. - **Works with your agent:** Claude Code, Claude Desktop, ChatGPT, Codex, Cursor, VS Code with GitHub Copilot and the Gemini CLI use one URL. ## What is an SEO MCP server? The Model Context Protocol (MCP) lets AI apps call tools on a server. An SEO MCP server exposes SEO data as tools, so a coding agent can answer “why did this keyword drop?” by reading rankings, crawl issues and search data itself, and then change the code. Serpel’s server is remote, so there is nothing to install: your agent connects to a URL. The server uses Streamable HTTP and answers each request on its own, with JSON. Lists are capped at 25 rows, so agents get compact answers instead of raw data. For Claude Code SEO work, that is the whole integration: one command adds the server. ## How do you connect Claude Code, Cursor and other agents? Every client uses the same URL. The first tool call opens a browser window where you sign in, choose the workspace and approve access. The [Claude Code setup guide](https://serpel.app/docs/mcp/claude-code) walks through it step by step. | Client | How to connect | | --- | --- | | Claude Code | Run `claude mcp add --transport http serpel https://serpel.app/mcp`. | | Claude Desktop and claude.ai | Add a custom connector with the URL under Connectors. | | Cursor | Add the URL to `.cursor/mcp.json`, as shown above, and sign in when the browser opens. | | Codex | Add `url = "https://serpel.app/mcp"` under `[mcp_servers.serpel]` in `~/.codex/config.toml`. | | ChatGPT | In developer mode, add a connector with the URL under Apps and Connectors. | | VS Code with GitHub Copilot | Add a server of type `http` with the URL to `.vscode/mcp.json`. | | Gemini CLI | Set `httpUrl` to the URL under `mcpServers` in `~/.gemini/settings.json`. | ## Which tools does the Serpel MCP server offer? The server offers 13 tools. Each has a title, a description, an input schema and annotations, and the 12 free tools are marked read-only. | Tool | What it does | Cost | | --- | --- | --- | | `list_projects` | Lists the projects you can access, with IDs, domains, markets and your role. Call it first: every other tool needs a project ID. | Free | | `get_project_overview` | Summarises one project: audit score, rankings, AI visibility, tasks, top issues and next steps. | Free | | `get_keyword_rankings` | Returns tracked keywords with the latest Google position, the previous position and the change. | Free | | `get_search_performance` | Returns clicks, impressions, CTR and position from Search Console or Bing, compared with the previous period. | Free | | `find_keyword_opportunities` | Finds search queries on positions 4 to 20 with impressions and matches them against your tracked keywords. | Free | | `get_crawl_issues` | Lists the issues of the latest crawl by type, severity and affected pages, with the score and its change. | Free | | `get_ai_visibility` | Shows how often ChatGPT cites or mentions your website and how often Google AI Overviews cite it, with the latest result per prompt. | Free | | `get_seo_recommendations` | Lists prioritised recommendations with impact, estimated clicks, effort, page, code route and next action. | Free | | `get_codebase_overview` | Summarises the latest `serpel scan` snapshot: routes without metadata and differences between code and the live site. | Free | | `get_conversion_performance` | Shows sessions, conversions and conversion rate per landing page from your analytics. | Free | | `analyze_competitors` | Compares the project with saved competitors from stored results, including keywords where a competitor leads. | Free | | `get_job_status` | Returns the status, progress and result of a background job by its ID. | Free | | `research_keywords` | Looks up keyword ideas or metrics: search volume, CPC, keyword difficulty and intent. Returns at most 25 keywords. | Credits | Long-running work such as crawls and rank checks never starts inside a tool call. Start it in the dashboard, the [CLI](https://serpel.app/developers/cli) or the [REST API](https://serpel.app/developers/api), and let the agent follow it with `get_job_status`. ## What can you ask an agent with Serpel connected? - “Which keywords moved since the last rank check?” The agent calls `get_keyword_rankings`. - “Which queries rank on positions 4 to 20?” It calls `find_keyword_opportunities`. - “What should I fix first?” It combines `get_crawl_issues` and `get_seo_recommendations`. - “Which routes in my code lack a title or description?” It calls `get_codebase_overview`. ## How do budget rules protect your credits? Only `research_keywords` spends credits. It runs only when every condition holds: 1. The connection includes the `keywords:write` permission. OAuth sign-in grants it. 2. The workspace has an active Starter or Pro plan. 3. The workspace owner has switched on “Agents can spend credits” in the Billing settings, or ran `serpel agents allow`. 4. The call costs at most 50 credits, and today’s agent spending plus the call stays under the daily limit. The default limit is 500 credits, and you can set it up to 100,000. The day follows UTC. 5. The balance covers the call. Credits come from your monthly plan allowance first, then from credits you bought. A repeated call with the same `requestId` is charged once. If a rule blocks the call, the tool returns a specific error such as `plan_required`, `agent_spend_disabled`, `daily_cap_reached` or `insufficient_credits`. Plans and credit prices are on the [pricing](https://serpel.app/pricing) page. ## Is it safe to give an agent access? - OAuth access tokens work only at the MCP server. The REST API rejects them. - Access tokens last 1 hour. Refresh tokens last 30 days and rotate on every use. - Tools respect token scopes, your project role and your workspace. A project you can’t access returns `not_found`. - You can disconnect an app at any time under Settings, API, Connected apps. Disconnecting revokes its access. ## Connect in three steps 1. **Add the server** Point your agent at the Serpel URL. For Claude Code, one command does it. ```bash claude mcp add --transport http serpel https://serpel.app/mcp ``` 2. **Sign in** The first tool call opens the browser. Choose the workspace, check the permissions and approve. 3. **Ask a question** Start with “List my Serpel projects”, or ask why a keyword dropped. The agent calls `list_projects` first, because every other tool needs a project ID. ## Frequently asked questions ### Does Serpel’s MCP server work with Claude Code? Yes. Run `claude mcp add --transport http serpel https://serpel.app/mcp`, then sign in when the browser opens. Cursor, Codex, VS Code with GitHub Copilot, ChatGPT and the Gemini CLI connect to the same URL. ### Can the agent start a crawl or a rank check? No. The MCP server doesn’t start long-running jobs. Start crawls and rank checks in the dashboard, the CLI or the API. The agent can follow progress with `get_job_status` and read the results as soon as the job has succeeded. ### Can an agent spend my credits? Only through `research_keywords`, and only if you have a Starter or Pro plan, have switched on “Agents can spend credits” and stay within the per-call and daily limits. Every other tool is free. ### Do I need an API key for the MCP server? No. Sign in with OAuth in the browser. A client without OAuth support can use an API token with the `mcp:read` permission instead, plus the read permissions for the data you want to use, and send it as a bearer token. --- # An SEO CLI for rankings, audits and CI. URL: https://serpel.app/developers/cli Updated: 2026-10-10 An SEO CLI lets you run rank checks, site audits and AI visibility checks from the terminal and from CI, and read the results as JSON. The Serpel CLI is a client of the REST API with device login, documented exit codes and a `--json` flag on every command. Install it from npm with `npm install -g @serpel/cli`. ## What the Serpel CLI is built for - **Almost the whole API:** Projects, crawls, rankings, AI visibility, search data, keywords, tasks, reports and exports, from one command. - **JSON you can pipe:** With `--json`, stdout carries only JSON. Notes, warnings and progress go to stderr. - **Jobs you can wait for:** Crawls, rank checks, AI checks and monitoring runs print a job ID, or wait for the result with `--wait` and `--timeout`. - **Device login:** Sign in through the browser without copying a token. In CI, set `SERPEL_TOKEN` instead. - **Token safety:** The token goes to the system keychain when one exists, never appears in output and is never forwarded through redirects. - **Codebase scan:** `serpel scan` reads route metadata from a Next.js project and never uploads source code. ## What can an SEO CLI do? An SEO CLI turns SEO work into commands that you can script, schedule and run in CI. With the Serpel CLI you create projects, research keywords, start crawls, track rankings, check AI visibility, read search data, manage tasks and export results, all from a terminal. It calls the same REST API as the dashboard, so permissions, credits and data are identical. | Command | What it does | | --- | --- | | `serpel projects` | Create, show and update projects and manage members. | | `serpel crawl` and `serpel audit` | Start crawls, compare them, list issues and read Core Web Vitals. | | `serpel rankings` | Add keywords, run checks, read positions, history and stored results. | | `serpel ai` | Add prompts, run ChatGPT and AI Overview checks and show citations. | | `serpel search` | Connect Google and Bing and read clicks, impressions and positions. | | `serpel keywords` | Research keywords and manage keyword lists. | | `serpel monitoring` and `serpel reports` | Run monitoring, manage recipients and read reports. | | `serpel tasks` and `serpel recommendations` | Manage tasks and act on recommendations. | | `serpel scan` | Upload route metadata from a Next.js project. | | `serpel export` | Export issues, keywords, rankings or tasks as CSV or JSON. | | `serpel tokens`, `serpel billing` and `serpel agents` | Manage tokens, check balance and estimates, and control agent spending. | ## How do you install the Serpel CLI? ```bash npm install -g @serpel/cli serpel auth login ``` The CLI is published on npm as [`@serpel/cli`](https://www.npmjs.com/package/@serpel/cli) under the MIT license and installs the `serpel` command. It needs Node.js 20.9 or newer. To try it without installing, run `npx @serpel/cli --help`. Technically, the CLI is built with esbuild into a single self-contained file. It bundles its dependencies, apart from an optional native module for the system keychain, so it needs no other Serpel packages on the machine that runs it. The [CLI reference](https://serpel.app/docs/cli) documents every command and option. ## How does sign-in work in the terminal and in CI? `serpel auth login` uses device login. The CLI shows a code and opens the browser, and you approve access in the dashboard. No token is copied. The token is stored in the system keychain when one is available, otherwise in a file that only your account can read. It never appears in output, not even with `--verbose`. In CI, set `SERPEL_TOKEN` and, if needed, `SERPEL_API_URL`. The variable takes precedence over any stored token and is never saved. Create a token with only the scopes the job needs, and limit it to one project. ## How do you use the SEO CLI in CI? Start a crawl, wait for it, and fail the build when the audit finds errors. `--wait` ends the command when the job is done, and `--timeout` limits the wait. The `--json` output of `serpel audit issues` carries the counts by severity. ```bash serpel crawl start --project "$PROJECT_ID" --wait --timeout 900 errors=$(serpel audit issues --project "$PROJECT_ID" --json | jq '.counts.error') test "$errors" -eq 0 ``` | Exit code | Meaning | | --- | --- | | 0 | Success. | | 2 | A usage error, such as an unknown option, or a validation error from the API. | | 3 | Not signed in, or the token is invalid. | | 4 | A missing permission or no access to the project. | | 7 | A provider problem or a rate limit. | | 8 | A job failed or was cancelled while waiting with `--wait`. | | 10 | The `--timeout` for `--wait` ran out. The job keeps running. | ## Which options work with every command? | Option | What it does | | --- | --- | | `--json` | Prints only machine-readable JSON on stdout. Errors use the same format. | | `--api-url ` | Sets the API base URL for one call. The default is the Serpel service. | | `--no-color` | Turns colours off. The `NO_COLOR` variable does the same. | | `--verbose` | Logs method, URL, status and duration of every request to stderr, without the token. | | `--wait` and `--timeout ` | Wait for a job to finish, on the commands that start jobs. | ## What does the CLI cost, and can an agent use it? Reading data is free. Commands that fetch live data book credits from the same public price list as the API, and new accounts start with 100 credits. Check a price first with `serpel billing estimate --operation rankCheck --keywords 100 --depth 50`, and read your balance with `serpel billing balance`. An agent that can run terminal commands can call the CLI with `--json` and read the exit codes. Agents that support MCP can use the [MCP server](https://serpel.app/developers/mcp) instead, which needs no install. ## Get started in four steps 1. **Get the CLI** Install the bundled file for Node.js 20.9 or newer, then check that it runs. ```bash serpel --version ``` 2. **Sign in** Approve access in the browser. The CLI stores the token for you. ```bash serpel auth login ``` 3. **Create a project** Add your domain with the market you want to track. ```bash serpel projects create --domain example.com --country US --language en ``` 4. **Run your first audit** Start a crawl and wait for the summary of pages and issues. ```bash serpel crawl start --project --wait ``` ## Frequently asked questions ### Is there an SEO CLI that runs in CI? Yes. The Serpel CLI works in CI: set `SERPEL_TOKEN`, run commands with `--json` and use the exit codes to fail a build. A command that waits with `--wait` ends with exit code 8 when the job fails and 10 when the wait times out. ### How do I install the Serpel CLI? Run `npm install -g @serpel/cli`, then `serpel auth login`. The package is on npm under the MIT license and needs Node.js 20.9 or newer. To try it once without installing, run `npx @serpel/cli --help`. ### Where does the CLI store my token? In the system keychain when one is available, otherwise in a file that only your account can read. `SERPEL_TOKEN` takes precedence and is never stored. The token never appears in output, not even with `--verbose`. ### Does the CLI send my source code to Serpel? No. `serpel scan` reads route metadata such as titles, descriptions and links from a Next.js project. It never reads `.env` files or anything listed in `.gitignore`, shows you what it will send first and asks for confirmation. --- # Serpel developer documentation URL: https://serpel.app/docs Updated: 2026-10-10 Serpel has three developer interfaces that share one account, one set of permissions and one credit balance: the `serpel` command line, a REST API and a remote MCP server for coding agents. This documentation shows how to sign in, what each interface can do and how to set it up. All three work on the same projects as the dashboard. ## What can you do with the Serpel CLI, API and MCP server? Every interface reads and changes the same rankings, crawls, keyword lists and tasks as the dashboard. Pick the one that fits where you work. | Interface | Use it to | Sign in with | Documentation | | --- | --- | --- | --- | | CLI | Run crawls, rank checks and exports from a terminal, a script or a CI job. | Browser approval (`serpel auth login`) or an API token in `SERPEL_TOKEN`. | [CLI reference](https://serpel.app/docs/cli) | | REST API | Build your own integrations that read or change the data the dashboard shows. | An API token in the `Authorization` header. | [REST API](https://serpel.app/docs/api) | | MCP server | Let Claude Code, Cursor or Codex read your SEO data while they work on your code. | OAuth in the browser, or an API token with the `mcp:read` scope. | [Claude Code](https://serpel.app/docs/mcp/claude-code), [Cursor](https://serpel.app/docs/mcp/cursor), [Codex](https://serpel.app/docs/mcp/codex) | ## How do you authenticate with Serpel? Serpel knows two kinds of credentials. API tokens serve the CLI and the REST API. OAuth access tokens serve MCP clients and work nowhere else. ### API tokens An API token starts with `vsk_` and is shown exactly once when you create it. Each token has a name, a list of permissions called scopes, an optional restriction to single projects and an optional expiry of 1 to 365 days. Create tokens in the dashboard under Settings, API tokens, with `serpel tokens create` or with `POST https://serpel.app/api/v1/tokens`. Send the token as `Authorization: Bearer vsk_…`. ### Browser approval for the CLI `serpel auth login` starts a device login. The CLI prints a code and opens the dashboard, you check the code and choose the permissions, and the CLI stores the token it receives. Nothing to copy and paste. The [CLI reference](https://serpel.app/docs/cli) explains the flow and the other ways to sign in. ### OAuth for MCP clients MCP clients such as Claude Code, Cursor and Codex sign in with OAuth 2.1 and PKCE. The client registers itself, you approve it on a consent page in the browser and choose the workspace. Access tokens last 1 hour and refresh tokens 30 days. The REST API rejects OAuth tokens with HTTP 403. You can disconnect any app under Settings, API, Connected apps. > **Give tokens the least access they need:** A token only works with the scopes you chose, and a token restricted to projects cannot see any other project (the API answers with 404 for those). Create one token per script or CI job, with only the scopes it needs. The 18 available scopes are listed in the [REST API reference](https://serpel.app/docs/api). ## What are the base URLs? | Interface | Address | Notes | | --- | --- | --- | | REST API | `https://serpel.app/api/v1` | JSON over HTTPS. `GET /api/v1` returns an index of all endpoints without authentication. | | MCP server | `https://serpel.app/mcp` | Streamable HTTP, POST only, JSON responses. | | Health check | `https://serpel.app/api/health` | Returns `ok` and the database status without authentication. | | CLI | `https://serpel.app` | The default API address of the CLI. Override it with `--api-url` or `SERPEL_API_URL`. | ```bash curl https://serpel.app/api/v1/projects \ -H "Authorization: Bearer $SERPEL_TOKEN" ``` ## How do credits work for CLI, API and MCP calls? Reading data is free. Operations that query a data provider use credits, at the same prices as in the dashboard. New accounts start with 100 welcome credits. | Operation | Unit | Credits | | --- | --- | --- | | Scheduled monitoring, up to top 50 | per keyword | 2 | | Scheduled monitoring, top 100 | per keyword | 3 | | Rank check right now, up to top 50 | per keyword | 5 | | Rank check right now, top 100 | per keyword | 9 | | ChatGPT answer check | per question | 2 | | Google AI Overview check | per question | 2 | | Keyword research | per search | 8 + 4 per 50 results | | Keyword research by domain | per domain | 4 + 2 per 50 results | | Site crawl | per 20 pages | 1 | Check a price before you run an operation with `serpel billing estimate` or `POST /billing/estimate`. An estimate books nothing. If the balance is too low, the API answers with HTTP 402 and the error code `insufficient_credits`, and the CLI exits with code 6. Coding agents get an extra safeguard. The MCP tool `research_keywords` is the only tool that uses credits, and it only runs when you allow agents to spend credits and stay within a daily limit. The [Claude Code guide](https://serpel.app/docs/mcp/claude-code) lists the rules. ## Where should you start? - [CLI reference](https://serpel.app/docs/cli): install the CLI, sign in, run every command group and use it in CI. - [REST API](https://serpel.app/docs/api): authentication, scopes, errors, rate limits, the endpoint list and curl examples. - [Claude Code](https://serpel.app/docs/mcp/claude-code): one command to connect, the tool list and the agent spending rules. - [Cursor](https://serpel.app/docs/mcp/cursor): the `mcp.json` entry for a remote server with OAuth. - [Codex](https://serpel.app/docs/mcp/codex): `codex mcp add` or `config.toml`, then `codex mcp login`. --- # Serpel CLI reference URL: https://serpel.app/docs/cli Updated: 2026-10-10 The `serpel` command line runs crawls, rank checks, keyword research and exports through the same REST API as the dashboard. This page covers installation, sign-in, every command group, JSON output, exit codes and a CI example. Run any command with `--help` for its full list of flags. ## How do you install the Serpel CLI? The CLI is published on npm as [`@serpel/cli`](https://www.npmjs.com/package/@serpel/cli) under the MIT license. It installs one command, `serpel`, ships as a single bundled file and needs Node.js 20.9 or newer. 1. **Install it globally.** npm puts the `serpel` command on your path. pnpm, Yarn and Bun work the same way. ```bash npm install -g @serpel/cli ``` 2. **Or run it without installing.** `npx` downloads the latest version for a single run, which is handy in CI or on a shared machine. ```bash npx @serpel/cli --help ``` 3. **Check the installation.** The command prints the CLI version. ```bash serpel --version serpel --help ``` The package contains `dist/index.js` with its libraries bundled, plus their license notices. The one optional dependency is `@napi-rs/keyring`, which lets the CLI keep your token in the system keyring. Without it, the CLI falls back to a file that only you can read. To update, run `npm install -g @serpel/cli@latest`. To remove it, run `npm uninstall -g @serpel/cli`. ## How do you sign in to the Serpel CLI? ### Browser sign-in (device login) This is the default and needs no token to copy. ```bash serpel auth login ``` 1. The CLI asks Serpel for a sign-in code and prints a confirmation address and a verification code such as `ABCD-EFGH`. The code is valid for ten minutes. The CLI opens the address in your default browser. With `--no-browser`, it only prints the address, which suits servers without a desktop. 2. In the dashboard, check that the code matches, choose the permissions the token gets and approve the sign-in. 3. The CLI waits for your approval, stores the token and prints your name, email address, permissions and project restriction. Ctrl+C cancels the wait. If the code expires or you deny the sign-in, the command exits with code 3. ### Sign in with an API token Create a token in the dashboard under Settings, API tokens, or with `serpel tokens create`, and hand it to the CLI. The CLI checks it with `GET /me` first and stores nothing if the token is invalid. Prefer `--token-stdin` over `--token`, because an argument ends up in your shell history. ```bash echo "$SERPEL_TOKEN" | serpel auth login --token-stdin serpel auth login --token-stdin < token.txt ``` ### Use SERPEL_TOKEN in scripts and CI In CI you do not need `serpel auth login`. Set `SERPEL_TOKEN` and the CLI uses it for every command. The variable beats any stored token and is never written to disk. ```bash export SERPEL_TOKEN=vsk_your_token serpel projects list --json ``` ### Check your session or sign out ```bash serpel auth status serpel auth logout ``` `auth logout` revokes the stored token on the server and then deletes it locally. If the server cannot be reached, the CLI still deletes the local copy and warns that the token stays valid until you revoke it in the dashboard or it expires. A token from `SERPEL_TOKEN` is not touched. ## How is the Serpel CLI configured? The CLI talks to `https://serpel.app` unless you tell it otherwise. The first source that is set wins: the `--api-url ` flag, the `SERPEL_API_URL` variable, the config file (`serpel config set api-url `), then the default. Only `https://` addresses are accepted, except `http://` for `localhost`, `127.0.0.1` and `[::1]`. A trailing `/api/v1` is removed, and addresses with credentials, query parameters or anchors are rejected. Most people never change the address. Set it only if you use another Serpel instance. ```bash serpel config get serpel config get api-url serpel config set api-url https://example.com serpel config unset api-url ``` ### Environment variables | Variable | Meaning | | --- | --- | | `SERPEL_API_URL` | API address, see above. | | `SERPEL_TOKEN` | API token. Takes precedence over a stored token and is never stored. | | `SERPEL_CONFIG_DIR` | Use another directory for the config file and the credentials file. | | `SERPEL_TOKEN_STORAGE` | `auto` (default: keyring, otherwise file), `keyring` or `file`. | | `SERPEL_HTTP_TIMEOUT` | Time limit per request in seconds. Default 30. Keyword research and list refreshes get at least 120 seconds. | | `SERPEL_POLL_INTERVAL_MS` | Check interval for `--wait` in milliseconds. Default 2000, minimum 50. | | `NO_COLOR`, `FORCE_COLOR` | Turn colours off, or force them when output is not a terminal. | ### Where does the CLI keep your token? - In the system keyring when one is available: Windows Credential Manager, macOS Keychain or the Secret Service on Linux. The entry uses the service name `serpel` and the API address as the account. - Otherwise in `credentials.json` in the config directory, with permissions that only your account can read, and a warning. Under Windows, the permissions of your user profile protect the file. - The token never appears in output, not even with `--verbose` or `--json`, and the CLI does not follow redirects, so the token cannot be sent to another address. | System | Config file | | --- | --- | | Windows | `%APPDATA%\serpel\config.json` | | Linux and macOS | `$XDG_CONFIG_HOME/serpel/config.json`, otherwise `~/.config/serpel/config.json` | ## Which global flags does the CLI have? These flags work with every command and can come before or after it. | Flag | Meaning | | --- | --- | | `--api-url ` | API address for this call. | | `--json` | Machine-readable JSON on standard output only. | | `--no-color` | Turn colours off. | | `--verbose` | Log method, URL, status and duration of each request to standard error, without headers, bodies or the token. | | `-h`, `--help` | Show help for the command. | | `-V`, `--version` | Show the CLI version. | ## Which commands does the Serpel CLI have? Every command accepts `--help` and prints its flags. Arguments such as `` stand for IDs you get from other commands, for example `serpel projects list`. Commands that delete or disconnect something send nothing until you add `--yes`, and exit with code 2 without it. ### Account and configuration | Command | What it does | | --- | --- | | `serpel auth login` | Signs in. By default it opens a browser approval (device login). `--no-browser` only prints the address, `--token-stdin` or `--token ` stores an existing API token after checking it. | | `serpel auth status` | Shows the signed-in user, permissions, project restriction, API address and where the token is stored. | | `serpel auth logout` | Revokes the stored token on the server and deletes it locally. | | `serpel auth profile` | Changes the name (`--name`) or email address (`--email`) of your own account. Needs a token with all permissions and no project restriction. | | `serpel config get [api-url]` | Shows the configuration, or a single value. | | `serpel config set api-url ` | Stores the API address in the config file. | | `serpel config unset api-url` | Removes the stored API address. | | `serpel tokens list` | Lists your API tokens with status, permissions and last use. | | `serpel tokens create --name --scope ` | Creates an API token. Repeat `--scope` for several permissions. `--project ` (repeatable) restricts the token to projects and `--expires-in-days ` (1 to 365) sets an expiry. The secret is printed once. | | `serpel tokens revoke --yes` | Revokes a token immediately. | ### Projects | Command | What it does | | --- | --- | | `serpel projects list` | Lists your projects. Flags: `--query` (`-q`), `--limit`, `--page`, `--offset`. | | `serpel projects create --domain ` | Creates a project. Optional: `--name`, `--country` (API default DE) and `--language` (API default de). Exit code 6 with the existing project ID if you already have a project for the domain. | | `serpel projects show ` | Shows the project overview: last crawl, issues, rankings, AI visibility, tasks and next steps. | | `serpel projects update ` | Changes the settings you pass, listed below. | | `serpel projects delete --yes` | Deletes a project and cancels its running jobs. Owners only. | | `serpel projects members list --project ` | Lists the members and their roles. | | `serpel projects members add --project --email
` | Adds an existing user. `--role editor` or `viewer` (API default viewer). Owners only. | | `serpel projects members remove --project --yes` | Removes a member. Owners only. | `serpel projects update` takes these settings. Pass only the ones you want to change. | Flag | Setting | | --- | --- | | `--name`, `--country`, `--language` | Display name and market. | | `--location ` | Default location for new keywords: a city, postal code, location code or `nationwide`. | | `--max-pages ` | Maximum pages per crawl, 1 to 1,000. | | `--rank-depth ` | Result depth of rank checks: 10, 20, 30, 50, 100. | | `--device desktop` or `--device mobile` | Device used for rank checks. | | `--schedule `, `--schedule-hour <0-23>` | Automatic rank check: `manual`, `daily`, `every_3_days` or `weekly`, and the hour it starts. | | `--crawl-schedule ` | Automatic crawl: `manual` or `weekly`. | | `--render-mode ` | JavaScript rendering during crawls: `auto` (only nearly empty pages), `always` or `never`. | | `--ai-overview on` or `off` | Track the Google AI Overview during rank checks and AI visibility checks. | | `--keyword-metrics on` or `off` | Add search volume and CPC automatically before rank checks. | | `--slack-webhook`, `--discord-webhook`, `--webhook` | Where monitoring reports are sent. `--no-slack-webhook`, `--no-discord-webhook` and `--no-webhook` remove a target. `--webhook` takes your own https address and receives JSON. | | `--slack-channel`, `--discord-channel` | Channel name shown in Serpel (up to 80 characters). `--no-slack-channel` and `--no-discord-channel` remove it. | ### Crawls and audit | Command | What it does | | --- | --- | | `serpel crawl start --project ` | Starts a crawl as a background job and prints the job ID. Flags: `--max-pages ` (1 to 1,000), `--wait`, `--timeout `. If a crawl is already running, you get that one back. | | `serpel crawl list --project ` | Lists crawls with status, pages and audit score. | | `serpel crawl show ` | Shows a crawl in detail: status codes, rendering, link check, Core Web Vitals status, robots.txt, sitemaps and llms.txt. | | `serpel crawl pages --crawl ` | Lists the crawled pages. `--status` filters by `ok`, `redirect`, `client_error`, `server_error`, `failed` or `blocked`, `--query` searches URL and title. | | `serpel crawl page --crawl ` | Shows one page: metadata, issues, rendering, hreflang, structured data and links. | | `serpel crawl compare --project ` | Compares two completed crawls (`--crawl`, `--base`): score, new, resolved and changed issues. Defaults to the latest crawl and the one before it. | | `serpel audit issues --project ` | Lists the issues of the latest completed crawl (or `--crawl`) grouped by type. Filters: `--severity error`, `warning` or `notice`, `--category`, `--query`. | | `serpel audit issue --project ` | Shows one issue with what was found, why it matters, how to fix it and the affected URLs. | | `serpel audit web-vitals ` | Shows LCP, INP and CLS per URL and device, as field and lab values. | The `--category` filter accepts `availability`, `redirects`, `meta`, `headings`, `indexing`, `crawling`, `links`, `images`, `security`, `content`, `performance`, `mobile`, `structured_data`, `ai_search`, `rendering`, `international`, `web_vitals`, `social`. ### Jobs | Command | What it does | | --- | --- | | `serpel jobs status ` | Shows the status of a job. Add `--wait` (and `--timeout `) to follow it to the end. | | `serpel jobs list` | Lists jobs. Filters: `--project`, `--status` (`queued`, `running`, `succeeded`, `failed`, `cancelled`) and `--type` (`crawl`, `rank_check`, `ai_check`, `monitoring`, `search_sync`, `analytics_sync`, `codebase_snapshot`, `recommendation_generate`). | | `serpel jobs cancel ` | Cancels a queued or running job. | | `serpel jobs retry ` | Restarts a failed or cancelled job. Other jobs end with exit code 6. | | `serpel jobs activity --project ` | Shows the project history: jobs, emailed reports and keyword research with cost and duration. `--days` (1 to 90, default 7). | ### Keyword research and keyword lists | Command | What it does | | --- | --- | | `serpel keywords research ""` | Researches keyword ideas and uses credits. Choose exactly one mode: a keyword, `--list "a,b,c"`, `--file ` (one keyword per line, `-` reads standard input) or `--domain `. Up to 700 keywords per request. Flags: `--country`, `--language`, `--limit` (1 to 500, API default 100), `--project`, `--refresh` (bypasses the cache and queries the provider again, which incurs costs), `--save-to `, `--idempotency-key `. | | `serpel keywords history` | Lists saved research runs, newest first (`--project`, `--limit`). | | `serpel keywords research-show ` | Shows a saved result again without a new query and without cost. | | `serpel keywords lists` | Lists keyword lists (`--project`, `--query`, `--limit`, `--page`, `--offset`). | | `serpel keywords list-show ` | Shows a list with its description and keyword count. | | `serpel keywords list-create --project --name ` | Creates a list. Optional `--description`. | | `serpel keywords list-update ` | Renames a list or changes the description (`--name`, `--description`, `--no-description`). `list-rename` is an alias. | | `serpel keywords list-delete --yes` | Deletes a list with all its keywords. | | `serpel keywords items ` | Shows the keywords of a list. `--sort` takes `keyword`, `volume`, `difficulty`, `cpc` or `created`. | | `serpel keywords items-add --keyword ` | Adds keywords. Repeat `--keyword`, or add `--research-run ` to take the metrics from a saved run. A list holds up to 5,000 keywords. | | `serpel keywords items-remove --yes` | Removes one keyword. The item ID is the ID column of `keywords items`. | | `serpel keywords import --file ` | Imports a CSV file of at most 2 MB (`-` reads standard input). Optional `--country` and `--language`. | | `serpel keywords refresh ` | Fetches the metrics of all keywords in a list again. Uses credits. Optional `--idempotency-key`. | ### Rankings and competitors | Command | What it does | | --- | --- | | `serpel rankings add --project --keyword ` | Adds keywords to rank tracking. Repeat `--keyword`. Optional `--device`, `--country`, `--language`, `--location`. | | `serpel rankings run --project ` | Starts a rank check for all or selected keywords (`--keyword-id`, repeatable) as a background job. Uses credits. Supports `--wait` and `--timeout`. | | `serpel rankings list --project ` | Lists tracked keywords with position, change, URL, AI Overview status and check time. | | `serpel rankings history ` | Shows the position history of a keyword (`--limit` 1 to 365, default 90). | | `serpel rankings serp ` | Shows the stored Google results of the last check (`--limit` 1 to 100, default 10). | | `serpel rankings summary --project ` | Shows top 3, top 10, average position and changes for all tracked keywords. | | `serpel rankings locations ` | Searches locations for local rank tracking (`--country`, `--limit`). | | `serpel rankings remove --yes` | Removes a keyword and its history. | | `serpel competitors list --project ` | Lists competitors with visibility, top 10 keywords and estimated clicks. | | `serpel competitors compare --project ` | Compares your position with each competitor, keyword by keyword. | | `serpel competitors suggest --project ` | Suggests domains that often appear in the top 20 for your keywords. | | `serpel competitors add --project --domain ` | Adds a competitor, up to 10 per project. | | `serpel competitors remove --project --domain ` | Removes a competitor by domain or ID. | ### AI visibility | Command | What it does | | --- | --- | | `serpel ai add --project --prompt ` | Adds prompts the way users ask them of an AI. Repeat `--prompt`. `--platform` currently accepts `chat_gpt` only. Optional `--country` and `--language`. | | `serpel ai prompts --project ` | Lists prompts with the latest result: cited, mentioned, Google AI Overview and check time. | | `serpel ai run --project ` | Checks all or selected prompts (`--prompt-id`, repeatable) as a background job. Uses credits. Supports `--wait` and `--timeout`. | | `serpel ai history ` | Shows the history of a prompt for ChatGPT and the Google AI Overview (`--limit` 1 to 365, default 60). | | `serpel ai status --project ` | Shows citations, mentions, the most frequent sources, AI crawler access in robots.txt and llms.txt. | | `serpel ai remove --yes` | Removes a prompt with all its results. | ### Monitoring and reports | Command | What it does | | --- | --- | | `serpel monitoring run --project ` | Checks rankings and AI visibility now, compares with the last report and creates a new report. Uses credits. Supports `--wait` and `--timeout`. | | `serpel monitoring webhook-test --project ` | Sends a test message to the configured webhooks. `--channel` limits it to `slack`, `discord` or `custom`. | | `serpel monitoring recipients list --project ` | Lists the email recipients of the reports. | | `serpel monitoring recipients add --project --email
` | Adds a recipient, up to 10 per project. | | `serpel monitoring recipients remove --project --email
` | Removes a recipient by address or ID. | | `serpel monitoring email-test --project ` | Sends the latest report as a test email to all recipients, or to one with `--email`. | | `serpel reports list` | Lists reports, newest first (`--project`, `--unread`, `--limit`, `--page`, `--offset`). | | `serpel reports show ` | Shows a report with rankings, AI visibility and website audit. Does not mark it as read. | | `serpel reports read ` | Marks a report as read. | | `serpel reports read-all` | Marks all reports as read, or those of `--project`. | | `serpel reports unread` | Shows the number of unread reports. | | `serpel reports send ` | Sends a report again to all active recipients. | | `serpel reports preview ` | Shows the email of a report, or saves it as HTML with `--output ` and opens it with `--open`. | ### Search data, web analytics and integrations API keys for Bing, PostHog and Plausible are read from a file with `--api-key-file `, or from standard input with `-`. They are never passed as an argument and never printed. Google connections open a browser approval; `--no-open` only prints the address. | Command | What it does | | --- | --- | | `serpel search status --project ` | Shows the Google Search Console and Bing Webmaster Tools connections of a project. | | `serpel search connect-google --project ` | Creates an authorisation address (valid for ten minutes) and opens it. Confirm it with the account that created the token. | | `serpel search connect-bing --project --api-key-file ` | Connects Bing Webmaster Tools. Optional `--site-url`. | | `serpel search site --project --provider --site-url ` | Chooses the property of a connection. `--provider` is `google` or `bing`. | | `serpel search queries --project ` | Shows clicks, impressions, CTR and position. Flags: `--provider google` or `bing`, `--dimension query` or `page`, `--days 7`, `28` or `90`, `--limit` (1 to 500, default 50). | | `serpel search disconnect --project --provider --yes` | Deletes the stored credentials of a connection. | | `serpel analytics status --project ` | Shows which of PostHog, Plausible and Google Analytics 4 are connected. | | `serpel analytics report --project ` | Shows visitors, sessions, pageviews, bounce rate, top pages, entry pages, sources and devices. Flags: `--provider`, `--days 7`, `28` or `90`, `--daily`. | | `serpel analytics connect-posthog --project --posthog-project --api-key-file ` | Connects PostHog. `--region eu` or `us`, or `--host ` for your own instance. | | `serpel analytics connect-plausible --project --site-id --api-key-file ` | Connects Plausible. Optional `--host ` for your own instance. | | `serpel analytics connect-google --project ` | Connects Google Analytics 4 in the browser. | | `serpel analytics property --project --property-id ` | Chooses the Google Analytics property. | | `serpel analytics properties --project ` | Lists the Google Analytics properties with their domains. `--refresh` reloads them from Google. | | `serpel analytics disconnect --project --provider --yes` | Deletes the stored credentials of a connection. | | `serpel integrations status` | Shows account integrations and how they map to your projects (`--details` per project). | | `serpel integrations connect-google` | Connects Google Search Console and Google Analytics for all projects. | | `serpel integrations connect-bing --api-key-file ` | Connects Bing Webmaster Tools for all projects. | | `serpel integrations connect-posthog --api-key-file ` | Connects PostHog for all projects (`--region`, `--host`). | | `serpel integrations connect-plausible --api-key-file ` | Connects Plausible for all projects (`--host`). | | `serpel integrations sync --provider ` | Maps projects again, for example after new properties. `--provider` is `google`, `bing`, `posthog` or `plausible`. | | `serpel integrations disconnect --provider --yes` | Disconnects an integration. Connected projects lose its data. | ### Tasks and recommendations | Command | What it does | | --- | --- | | `serpel tasks list --project ` | Lists tasks. Filters: `--status open`, `in_progress` or `done`, `--priority high`, `medium` or `low`, `--query`. | | `serpel tasks create --project --title ` | Creates a task. Optional `--priority`, `--description` and `--url` (repeatable). | | `serpel tasks create --project <id> --issue <code>` | Creates a task from an audit issue and carries over the affected URLs. `--crawl` picks the crawl, default is the last completed one. | | `serpel tasks update <taskId>` | Changes `--status`, `--priority`, `--title` or `--description`. | | `serpel tasks show <taskId>` | Shows a task with its URLs and the result of the recheck after the latest crawl. | | `serpel tasks delete <taskId> --yes` | Deletes a task. | | `serpel recommendations list --project <id>` | Lists recommendations (alias `serpel recs`). `--status` takes `open`, `accepted`, `dismissed`, `done`, `stale` or `active`, `--rule` filters by rule, `--query` searches. | | `serpel recommendations show <id>` | Shows a recommendation with evidence, data sources and the next action. | | `serpel recommendations accept <id>` | Accepts a recommendation. `dismiss`, `done` and `reopen` work the same way. | ### Codebase scan `serpel scan [directory]` reads a Next.js project, builds a snapshot of its routes and metadata and uploads it to a project. Serpel then matches routes to crawled pages and search data and flags differences between code and live site. The snapshot contains no source code. The upload is free and needs the `codebase:write` scope. | Flag | Meaning | | --- | --- | | `[directory]` | The folder with the `package.json` of the Next.js app. Defaults to the current folder. In a monorepo, pass the subfolder, for example `apps/web`. | | `--project <id>` | The project the snapshot belongs to. Required unless you use `--dry-run`. | | `--dry-run` | Sends nothing and needs no sign-in. The snapshot goes to standard output, or to `--out`. | | `--out <file>` | Writes the snapshot as JSON to a file. | | `--include-text` | Also sends a text excerpt of at most 500 characters per page, only for routes with build output. | | `--yes` | Skips the confirmation before sending. Without a terminal and without `--yes`, the command exits with code 2. | | `--wait`, `--timeout <seconds>` | Waits until Serpel has processed the snapshot. | The CLI never reads or sends `.env` files, key files, `node_modules`, `.git` or anything your `.gitignore` excludes, and it removes values that look like credentials. Limits: 2,000 routes, 5,000 sitemap URLs, 2 MB of JSON and 20 uploads per project and day. Answering anything but yes to the confirmation prompt ends the command with exit code 130. ```bash serpel scan --dry-run serpel scan apps/web --dry-run --out snapshot.json serpel scan --project "$PROJECT_ID" --wait serpel scan apps/web --project "$PROJECT_ID" --yes --include-text ``` ### Export `serpel export --project <id> --type <type>` exports `issues`, `keywords`, `rankings` or `tasks`. `--format` is `csv` (default) or `json`. Without `--output <file>` (`-o`), the file content goes to standard output so you can redirect it. `--crawl <id>` applies to issues (default: the last completed crawl) and `--list <id>` to keywords. ```bash serpel export --project "$PROJECT_ID" --type issues --format csv --output issues.csv serpel export --project "$PROJECT_ID" --type rankings --format json > rankings.json ``` ### Billing, agents and providers | Command | What it does | | --- | --- | | `serpel billing balance` | Shows available, total and reserved credits, usage today and in the last 30 days, and a forecast of how long the balance lasts. | | `serpel billing estimate --operation <operation>` | Estimates the cost of an operation and books nothing. Operations: `rankCheck`, `aiCheck`, `keywordResearch`, `keywordListRefresh`, `keywordMetrics`, `crawl`, `monitoring`. Inputs: `--keywords`, `--depth`, `--prompts`, `--no-ai-overview`, `--scheduled`, `--mode`, `--limit`, `--tasks`, `--max-pages`. | | `serpel billing history` | Lists credit transactions (`--limit`, `--page`, `--offset`). | | `serpel billing plan` | Shows your plan, limits and subscription. | | `serpel billing top-up` | Opens Stripe Checkout for a credit top-up. `--package credits500`, `credits1000`, `credits2500` or `credits5000`, or `--amount <cents>`. `--no-open` only prints the address. | | `serpel billing subscribe --plan <plan>` | Opens Stripe Checkout for `starter` or `pro`. | | `serpel billing change-plan --plan <plan>` | Changes the plan. Upgrades apply immediately, downgrades at the end of the billing period. | | `serpel billing cancel`, `serpel billing resume` | Cancels the subscription at the end of the billing period, or withdraws the cancellation. | | `serpel billing portal` | Opens the Stripe customer portal for payment details and invoices. | | `serpel agents status` | Shows whether agents may spend credits, the daily limit and today’s usage. | | `serpel agents allow` | Allows agents to spend credits through the MCP server. Needs the Starter or Pro plan. `--daily-limit <credits>` sets the limit at the same time. | | `serpel agents deny` | Stops agents from spending credits. | | `serpel agents daily-limit <credits>` | Sets the daily limit for agents. `--reset` restores the default. | | `serpel providers status` | Shows the data providers, whether they are configured, daily budget and usage, and when each was last used. | Top-ups, plan changes and the agent settings need the workspace owner role. ## Which permissions do the commands need? A token can only run commands whose scope it has. Without the scope, the API answers with HTTP 403 and the CLI exits with code 4. The [REST API reference](https://serpel.app/docs/api) lists every scope. | Commands | Scopes | | --- | --- | | `projects` (read), `reports` (list, show, read, preview), `search`, `analytics` and `integrations` (status and reports), `monitoring recipients list` | `projects:read` | | `projects` (create, update, delete, members), connect and disconnect commands, `monitoring webhook-test`, `monitoring recipients add` and `remove`, `monitoring email-test`, `reports send` | `projects:write` | | `crawl` (list, show, pages, page, compare), `audit` | `audit:read` | | `crawl start` | `audit:write` | | `rankings` (list, history, serp, summary, locations), `competitors` (list, compare, suggest), `ai` (prompts, history, status) | `rankings:read` | | `rankings` (add, run, remove), `competitors` (add, remove), `ai` (add, run, remove), `monitoring run` | `rankings:write` | | `keywords` (history, research-show, lists, list-show, items) | `keywords:read` | | `keywords` (research, list-create, list-update, list-delete, items-add, items-remove, import, refresh) | `keywords:write` | | `jobs` (status, list) | `audit:read` for crawls, `rankings:read` for rank checks, AI checks and monitoring runs | | `jobs` (cancel, retry) | `audit:write` for crawls, `rankings:write` for the other jobs | | `tasks` (list, show) | `tasks:read` | | `tasks` (create, update, delete) | `tasks:write` | | `recommendations` (list, show) | `recommendations:read` | | `recommendations` (accept, dismiss, done, reopen) | `recommendations:write` | | `scan` | `codebase:write` | | `export` | `export:read` | | `billing` (balance, estimate, history, plan), `agents status` | `billing:read` | | `billing` (top-up, subscribe, change-plan, cancel, resume, portal), `agents` (allow, deny, daily-limit) | `billing:write` | | `providers status` | Any valid token. | | `tokens`, `auth profile` | A token with all permissions. A token restricted to projects can only create tokens for its own projects. | ## How do you wait for background jobs? Crawls, rank checks, AI visibility checks and monitoring runs run in a worker. `crawl start`, `rankings run`, `ai run` and `monitoring run` print the job ID and return. With `--wait`, the CLI checks the job every two seconds until it succeeds, fails or is cancelled, and then prints a summary. `--timeout <seconds>` limits the wait and only works with `--wait`. `jobs status <jobId> --wait` follows a job you started earlier. - If the job fails or is cancelled, the CLI exits with code 8 and prints the last error. - If `--timeout` is reached, the CLI exits with code 10. The job keeps running on the server. - Ctrl+C stops waiting and exits with code 130. The job keeps running. - Short network errors while waiting are retried up to three times. - `SERPEL_POLL_INTERVAL_MS` changes the check interval. ```bash serpel crawl start --project "$PROJECT_ID" --wait --timeout 900 serpel jobs status "$JOB_ID" --wait ``` ## How does pagination work? List commands accept `--limit <n>` (1 to 200, default 50) and either `--page <n>` (from 1) or `--offset <n>`. When more entries exist, the CLI prints the range and the flags for the next page on standard error. With `--json`, the total is in `pagination.total`. ## How do you use JSON output in scripts? With `--json`, the CLI writes only JSON to standard output. Hints, warnings and progress go to standard error, and colours are off. Lists come as `{ "data": [...], "pagination": { "limit", "offset", "total" } }`. Single objects and actions return the object itself. Errors are printed to standard output as an object and set the exit code. ```json { "error": { "code": "not_logged_in", "message": "Not signed in to https://serpel.app." } } ``` API errors keep the code of the API, such as `validation_error` or `unauthorized`. Local errors use their own codes: `usage_error`, `not_logged_in`, `network_error`, `timeout`, `job_failed`, `job_cancelled`, `wait_timeout`, `interrupted` and `webhook_failed`. Some errors add a `details` field. ```bash serpel projects list --json | jq -r '.data[] | "\(.id) \(.domain)"' PROJECT_ID=$(serpel projects list --json | jq -r '.data[] | select(.domain == "example.com") | .id') JOB_ID=$(serpel crawl start --project "$PROJECT_ID" --json | jq -r .job.id) ``` ## What do the exit codes mean? | Code | Meaning | | --- | --- | | 0 | Success. | | 1 | Unexpected error, HTTP 500 `internal_error`, an invalid API response, or a webhook test message that was not delivered (`webhook_failed`). | | 2 | Usage error such as an unknown flag, a missing argument or an invalid value, or `validation_error` and `bad_request` from the API. Also a destructive command without `--yes`. | | 3 | Not signed in, HTTP 401, or a device login that expired or was denied. | | 4 | HTTP 403: a missing scope, no access to the project, or token management with a restricted token. | | 5 | HTTP 404: not found. | | 6 | HTTP 409 conflict, such as a project that already exists or `jobs retry` on a running job. Also HTTP 402 `insufficient_credits`. | | 7 | Provider problem: HTTP 429 (`rate_limited`, `budget_exceeded`), 502 (`provider_error`) or 503 (`provider_not_configured`). | | 8 | A job failed or was cancelled while waiting with `--wait`. | | 9 | The API is not reachable: connection error, DNS, TLS, request timeout or an unexpected redirect. | | 10 | The `--wait --timeout` limit was reached. The job keeps running. | | 130 | Interrupted with Ctrl+C, or any answer but yes to the `scan` confirmation. | ## How do you run the CLI in GitHub Actions? Create a token with only the scopes the job needs, restrict it to one project and store it as the repository secret `SERPEL_TOKEN`. The CLI reads that variable on every run and never stores it. ```bash serpel tokens create --name "GitHub Actions" --scope projects:read --scope audit:read --scope audit:write --project "$PROJECT_ID" --expires-in-days 90 ``` The workflow below crawls the site every Monday, waits for the crawl and fails when the latest crawl contains errors. The install step pulls `@serpel/cli` from npm; pin a version such as `@serpel/cli@0.1.0` if you want reproducible builds. ```yaml name: Serpel crawl on: schedule: - cron: "0 6 * * 1" workflow_dispatch: jobs: crawl: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 22 - name: Install the Serpel CLI run: npm install -g @serpel/cli - name: Crawl the site and fail on errors env: SERPEL_TOKEN: ${{ secrets.SERPEL_TOKEN }} SERPEL_PROJECT_ID: ${{ vars.SERPEL_PROJECT_ID }} run: | serpel crawl start --project "$SERPEL_PROJECT_ID" --wait --timeout 900 serpel audit issues --project "$SERPEL_PROJECT_ID" --json > issues.json jq -e '.counts.error == 0' issues.json ``` > **Crawls use credits:** Every crawl is billed per 20 pages. Run `serpel billing estimate --operation crawl --max-pages 500` before you schedule one, and set `--max-pages` to match your site. ## How do you troubleshoot the CLI? - `--verbose` prints the method, URL, status and duration of every request to standard error, without headers, bodies or the token. - `serpel config get` shows which API address applies and where it comes from. - “Endpoint not found (HTTP 404)” or “The API response is not valid JSON” usually means a wrong API address. Run `serpel config get` and check `--api-url` and `SERPEL_API_URL`. - If the system keyring is not available, for example on Linux without a running Secret Service, the CLI stores the token in `credentials.json` and prints a warning. - Exit code 3 means the token is missing, invalid, expired or revoked. Sign in again or create a new token. --- # Serpel REST API reference URL: https://serpel.app/docs/api Updated: 2026-10-10 The Serpel REST API under `https://serpel.app/api/v1` offers the same features as the dashboard and the CLI, as JSON over HTTPS. You authenticate with an API token that carries permissions called scopes. This page lists the scopes, errors, rate limits, credit rules and every endpoint, and ends with curl examples. ## What is the base URL of the Serpel API? All endpoints live under `https://serpel.app/api/v1`. Paths in this reference are relative to it, so `/projects` means `https://serpel.app/api/v1/projects`. Requests and responses use JSON. Send `Content-Type: application/json` with a body. `GET https://serpel.app/api/v1` returns a machine-readable index of the endpoints without authentication, and `GET https://serpel.app/api/health` returns the service status. ## How do you authenticate with the API? Send an API token as a bearer token. Tokens start with `vsk_`. Create one in the dashboard under Settings, API tokens, with `serpel tokens create` or with `POST /tokens`. ```http GET /api/v1/projects HTTP/1.1 Host: serpel.app Authorization: Bearer vsk_your_token ``` - A token can only call endpoints whose scope it has. Without it, the API answers with 403 `forbidden`. - The project role still applies: viewers read, editors change, owners manage members and delete the project. - The dashboard uses a session cookie instead of a token. Cookie requests that change data must come from the Serpel origin. - OAuth access tokens issued to MCP clients only work at the MCP server. The REST API rejects them with 403 `forbidden`. ## Which scopes can an API token have? Serpel has 18 scopes. Choose the smallest set that your integration needs. | Scope | Label in the dashboard | What it allows | | --- | --- | --- | | `projects:read` | Read projects | Read projects, overviews, members, reports, search data, web analytics, conversions and integrations. | | `projects:write` | Create and edit projects | Create, change and delete projects and members, connect search data, web analytics and integrations, and manage report recipients and webhooks. | | `keywords:read` | Read keyword lists | Read keyword lists, their keywords and saved research runs. | | `keywords:write` | Research keywords and edit lists | Run keyword research (uses credits), create and change keyword lists, import keywords and refresh metrics. | | `audit:read` | Read crawls and issues | Read crawls, pages, issues, comparisons and Core Web Vitals. | | `audit:write` | Start crawls | Start crawls. | | `rankings:read` | Read rankings | Read tracked keywords, rank history, stored search results, competitors, AI prompts and AI visibility. | | `rankings:write` | Track keywords and start rank checks | Track and remove keywords, start rank checks, AI checks and monitoring runs, and manage competitors and AI prompts. | | `tasks:read` | Read tasks | Read tasks. | | `tasks:write` | Create and edit tasks | Create, change and delete tasks. | | `export:read` | Export data | Export issues, keywords, rankings and tasks. | | `billing:read` | Read balance and usage | Read balance, usage, the credit ledger, the plan and the agent spending settings, and estimate costs. | | `billing:write` | Top up balance | Start checkouts, change or cancel the subscription and change the agent spending settings. Workspace owners only. | | `codebase:read` | Read codebase snapshots | Read codebase snapshots. | | `codebase:write` | Upload codebase snapshots | Upload codebase snapshots. | | `recommendations:read` | Read recommendations | Read recommendations. | | `recommendations:write` | Accept, dismiss and complete recommendations | Accept, dismiss, complete and reopen recommendations. | | `mcp:read` | Use the MCP server (read-only) | Use the MCP server, read-only. The REST API does not need this scope. | ### Restrict a token to projects A token can be limited to a list of projects with `projectIds`. Such a token sees only those projects. A project outside the list answers with 404 `not_found`, as if it did not exist, and lists leave it out. Tokens can also expire after 1 to 365 days. ```bash curl -X POST https://serpel.app/api/v1/tokens \ -H "Authorization: Bearer $SERPEL_TOKEN" \ -H "Content-Type: application/json" \ -d '{"name": "CI crawl", "scopes": ["projects:read", "audit:read", "audit:write"], "projectIds": ["0b6f0f0e-6d1a-4c8e-9a53-4f9d2d7b8a10"], "expiresInDays": 90}' ``` In the response, `token` holds the metadata and `secret` holds the token itself. The secret is shown exactly once. Creating tokens needs a token with all permissions. A token restricted to projects can only create tokens for its own projects. ## What do responses and pagination look like? ```json { "data": { } } { "data": [ ], "pagination": { "limit": 50, "offset": 0, "total": 123 } } ``` Single objects come in `data`. Lists add `pagination`. Lists accept `limit` (1 to 200, default 50), `offset` (from 0) and usually `q` to search. Every response carries an `X-Request-ID` header. You can send your own `X-Request-ID` (up to 128 letters, digits, `.`, `_`, `:` and `-`) and quote the value when you contact support. ## How does the API report errors? ```json { "error": { "code": "validation_error", "message": "Enter a domain.", "details": [{ "path": "domain", "message": "Enter a domain." }] } } ``` | HTTP | Code | Meaning | | --- | --- | --- | | 400 | `validation_error` | The input is invalid. `details` lists `path` and `message` for each problem. | | 400 | `bad_request` | The request is malformed, for example invalid JSON or a body over 2.5 MB. | | 400 | `authorization_pending`, `slow_down`, `expired_token`, `access_denied` | Answers of the device login at `POST /auth/device/token`. | | 401 | `unauthorized` | The token is missing, invalid, expired or revoked. | | 402 | `insufficient_credits` | Not enough credits. `details` has `balance`, `available` and `required`. | | 403 | `forbidden` | A scope or role is missing, an OAuth token was used on the REST API, or the workspace is blocked from paid operations. | | 403 | `plan_limit` | The plan’s limits are exceeded: projects, tracked keywords, AI prompts or daily monitoring. `details` has `plan` and `limits`. | | 404 | `not_found` | The item does not exist or the token cannot see it. | | 409 | `conflict` | The state does not allow it, for example a project that already exists (`details.existingProjectId`). | | 429 | `rate_limited` | Too many requests. The `Retry-After` header and `details.retryAfterSeconds` tell you how long to wait. | | 429 | `budget_exceeded` | The workspace’s daily credit limit or a provider budget is reached. | | 500 | `internal_error` | An error on the server. Retry later and quote the `X-Request-ID`. | | 502 | `provider_error` | A data provider failed or could not be reached. | | 503 | `provider_not_configured` | The data provider for this feature is not set up. | ## What are the API rate limits? | Limit | Applies to | Window | | --- | --- | --- | | 600 requests | Each API token, for all endpoints. | One minute | | 30 operations | Each workspace, for crawls, rank checks, AI checks, monitoring runs, keyword research and list refreshes. | One minute | | 20 requests | Each IP address, for all free tool endpoints under `/tools/`, which need no token. | 10 minutes | | 200 requests | Each IP address, for all free tool endpoints under `/tools/`. | One day | When you exceed a limit, the API answers with HTTP 429 and `rate_limited`. Wait for the seconds in the `Retry-After` header, then continue. The MCP server has its own limit, described in the [Claude Code guide](https://serpel.app/docs/mcp/claude-code). ## How do idempotency keys work? `POST /keywords/research` and `POST /keyword-lists/{id}/refresh` use credits and answer synchronously. Add an `Idempotency-Key` header with up to 200 letters, digits and the characters `.`, `_`, `:` and `-` to make a retry safe. A repeated request with the same key does not call the data provider again and does not book credits again. Without the header, the server generates a random key, which protects nothing on a retry. The CLI creates a key for you and accepts `--idempotency-key`. ## How do credits work in the API? Reading data is free. These operations use credits, at the prices of the dashboard: | Operation | Unit | Credits | | --- | --- | --- | | Scheduled monitoring, up to top 50 | per keyword | 2 | | Scheduled monitoring, top 100 | per keyword | 3 | | Rank check right now, up to top 50 | per keyword | 5 | | Rank check right now, top 100 | per keyword | 9 | | ChatGPT answer check | per question | 2 | | Google AI Overview check | per question | 2 | | Keyword research | per search | 8 + 4 per 50 results | | Keyword research by domain | per domain | 4 + 2 per 50 results | | Site crawl | per 20 pages | 1 | Estimate a cost first with `POST /billing/estimate`. It needs `billing:read`, books nothing and returns `minCredits`, `maxCredits` and a `breakdown`. Supported operations are `rankCheck`, `aiCheck`, `keywordResearch`, `keywordListRefresh`, `keywordMetrics`, `crawl` and `monitoring`. ```bash curl -X POST https://serpel.app/api/v1/billing/estimate \ -H "Authorization: Bearer $SERPEL_TOKEN" \ -H "Content-Type: application/json" \ -d '{"operation": "rankCheck", "params": {"keywordCount": 20, "depth": 50}}' ``` If the balance is too low for an operation, the API answers with HTTP 402. ```json { "error": { "code": "insufficient_credits", "message": "The available credits balance isn’t sufficient for this operation.", "details": { "balance": 3, "available": 3, "required": 12 } } } ``` ## Which endpoints does the REST API have? The groups below follow the index that `GET /api/v1` returns. Path parameters appear in braces. Methods in one row share the path. The scope line above each table names the scopes the endpoints need. ### Authentication These endpoints serve the sign-up and sign-in screens of the dashboard and need no token. The Google endpoints are browser redirects, not JSON calls. | Endpoint | Methods | What it does | | --- | --- | --- | | `/auth/signup` | POST | Create an account with a personal workspace and send a confirmation email | | `/auth/verify-email` | POST | Verify the email address with a one-time token valid for 24 hours | | `/auth/password-reset/request` | POST | Request a password reset without revealing whether an account exists | | `/auth/password-reset/confirm` | POST | Set a new password with a one-time token valid for one hour | | `/auth/google/start` | GET | Start signing in or signing up with Google as a browser redirect (query intent login, signup or reauthenticate, optional next path); creates a one-time state, PKCE verifier and nonce bound to a short-lived cookie | | `/auth/google/callback` | GET | Return address of Google: checks state, cookie binding, code and ID token, then signs in, links the account by verified email address or asks for the terms; redirects to the dashboard or back to the sign-in page with an error code | | `/auth/google/complete` | POST | Create the account after the first Google sign-in once the terms are accepted ({ acceptTerms: true }, cookie from the callback) | | `/cancellation-requests` | POST | Cancel a subscription without signing in (§ 312k BGB), store the receipt with a timestamp and confirm it by email | ### Free tools These endpoints need no token. They share one limit per IP address instead: 20 requests per 10 minutes and 200 per day. GET results are cached for a short time: 1 minute for the schema validator and the redirect checker, 5 minutes for the sitemap checker and 10 minutes for the robots.txt check. | Endpoint | Methods | What it does | | --- | --- | --- | | `/tools/robots-check` | GET | Check without signing in which AI crawlers a domain’s robots.txt allows on the homepage, plus its llms.txt and sitemaps (domain query parameter) | | `/tools/schema-validate` | GET, POST | Validate JSON-LD, Microdata and RDFa without signing in. GET takes a public page URL (url query parameter) and reads its static HTML. POST takes a JSON body with a code field and checks pasted JSON-LD or HTML up to 512 KB. Returns the detected items by type with properties, errors and warnings with property paths, and Google rich result eligibility | | `/tools/redirect-check` | GET | Trace the redirect chain of a URL without signing in (url query parameter, optional userAgent: browser, googlebot, googlebotSmartphone, bingbot or gptbot). Returns every hop with status code, Location, response time and server header for up to 10 redirects, loop and downgrade detection, a meta refresh on the final page and recommendations | | `/tools/sitemap-check` | GET | Validate the XML sitemaps of a domain or a sitemap URL without signing in (url query parameter). Discovers sitemaps through robots.txt and the default locations, checks status, content type, gzip, XML, namespace, limits and entries of up to 10 child sitemaps, and samples the status of up to 20 listed URLs | ### Account `GET /me` works with any valid token. Changing the profile and using `/tokens` need a token with all permissions. `/account/export`, `/account/delete`, `/account/password/set` and `/account/identities/google` need a dashboard session, not an API token. | Endpoint | Methods | What it does | | --- | --- | --- | | `/me` | GET, PATCH | Signed-in user and token permissions; change name and email address | | `/account/export` | GET | Download account data, workspace, projects, integrations and export links as JSON | | `/account/delete` | POST | Delete the account with session and password, or, without a password, after a confirmation with Google from the last 10 minutes; permanent cleanup after 30 days | | `/account/security` | GET | Sign-in methods of the account: password set or not, linked Google account, and whether a recent confirmation with Google is valid | | `/account/password/set` | POST | Set the first password of an account that signed up with Google; needs a confirmation with Google from the last 10 minutes | | `/account/identities/google` | DELETE | Unlink the Google account; only possible when the account has a password | | `/tokens` | GET, POST | List and create API tokens | | `/tokens/{id}` | DELETE | Revoke a token | | `/providers` | GET | Data providers, daily budget, usage and last use | | `/favicons/{domain}` | GET | Logo of a domain for the interface, fetched server-side and cached for 7 days | ### Billing Reading needs `billing:read`. Checkouts and changes need `billing:write` and the workspace owner role. | Endpoint | Methods | What it does | | --- | --- | --- | | `/billing/summary` | GET | Read balance, daily limit and usage of the last 30 days | | `/billing/estimate` | POST | Estimate the credit cost of an operation without booking it | | `/billing/ledger` | GET | Read the entries of the workspace credit ledger | | `/billing/checkout-sessions` | POST | Create a Stripe Checkout for a credit pack or a custom amount | | `/billing/subscription` | GET, PATCH | Read plan, limits and subscription or change the plan | | `/billing/subscription-sessions` | POST | Create a Stripe Checkout for a Starter or Pro subscription | | `/billing/subscription/cancel` | POST | Cancel the subscription at the end of the billing period | | `/billing/subscription/resume` | POST | Undo the cancellation of the subscription | | `/billing/portal-sessions` | POST | Open the Stripe customer portal for payment details and invoices | | `/billing/agent-spending` | GET, PATCH | Read or change the “Agents can spend credits” setting with daily limit and today’s usage | | `/billing/stripe-webhook` | POST | Process signed Stripe events idempotently into payments and the credit ledger | ### MCP and connected apps These endpoints need a dashboard session, or a token with all permissions for the two read and disconnect endpoints. | Endpoint | Methods | What it does | | --- | --- | --- | | `/oauth/consent` | POST | Decide on an app’s OAuth request (session only, with CSRF token) | | `/oauth/apps` | GET | Apps connected via OAuth with workspace, permissions and last use | | `/oauth/apps/{id}` | DELETE | Disconnect a connected app and revoke all its tokens | ### Projects `projects:read` to read, `projects:write` to create and change. Changing a project needs the editor role, deleting it and managing members the owner role. | Endpoint | Methods | What it does | | --- | --- | --- | | `/projects` | GET, POST | List and create projects | | `/projects/{id}` | GET, PATCH, DELETE | Read a project, change settings (including JavaScript rendering, Slack, Discord and custom webhook, channel name, search volume), delete it | | `/projects/{id}/overview` | GET | Overview with score, rankings, AI visibility and next steps | | `/projects/{id}/members` | GET, POST | List and add members | | `/projects/{id}/export` | GET | Export issues, keywords, rankings or tasks as CSV or JSON | ### Audit `audit:read` to read, `audit:write` to start crawls. | Endpoint | Methods | What it does | | --- | --- | --- | | `/projects/{id}/crawls` | GET, POST | List and start crawls | | `/projects/{id}/crawls/compare` | GET | Compare two crawls: new, fixed and changed issues | | `/projects/{id}/issues` | GET | Issues of the crawl with score and comparison | | `/projects/{id}/issues/{code}` | GET | Affected URLs of an issue | | `/crawls/{id}` | GET | Read a crawl, with statistics on rendering, link checks and Core Web Vitals | | `/crawls/{id}/pages` | GET | Checked URLs of a crawl, with HTML size and rendering flags | | `/crawls/{id}/pages/{pageId}` | GET | A page with links, issues and signals (rendering, hreflang, structured data, soft 404) | | `/crawls/{id}/web-vitals` | GET | Core Web Vitals of the crawl from PageSpeed Insights: lab and field values per URL and device | ### Keywords and rankings `keywords:read` and `keywords:write` for research and lists, `rankings:read` and `rankings:write` for tracked keywords and competitors. Research and list refreshes accept an `Idempotency-Key` header. | Endpoint | Methods | What it does | | --- | --- | --- | | `/keywords/research` | GET, POST | Start keyword research with idempotency and credits; read the history with cost and duration | | `/keyword-lists` | GET, POST | Manage keyword lists | | `/keyword-lists/{id}/refresh` | POST | Refresh list metrics idempotently, billed in credits | | `/projects/{id}/rankings` | GET, POST | List and add tracked keywords | | `/projects/{id}/rankings/run` | POST | Start a ranking check right away | | `/rank-keywords/{id}/history` | GET | Position history of a keyword | | `/rank-keywords/{id}/serp` | GET | Stored search results of the last check | | `/locations` | GET | Search locations for local rank tracking | | `/projects/{id}/competitors` | GET, POST | List and add competitors | | `/projects/{id}/competitors/overview` | GET | Visibility of you and your competitors | | `/projects/{id}/competitors/suggestions` | GET | Competitor suggestions from the search results | | `/projects/{id}/competitors/comparison` | GET | Positions of you and your competitors, keyword by keyword | ### AI visibility and monitoring `rankings:read` and `rankings:write` for AI visibility and monitoring runs, `projects:read` and `projects:write` for reports, recipients and webhooks. | Endpoint | Methods | What it does | | --- | --- | --- | | `/projects/{id}/ai-visibility` | GET | Citations in ChatGPT and Google AI Overviews, access for AI crawlers | | `/projects/{id}/ai-prompts` | GET, POST | List and create prompts for ChatGPT, each with the result of Google AI Overviews | | `/projects/{id}/ai-prompts/run` | POST | Check prompts right away, also in Google AI Overviews when the switch is on | | `/projects/{id}/monitoring/run` | POST | Start monitoring with a report right away | | `/reports` | GET | Reports, optionally only unread ones, with status and duration of the email delivery | | `/projects/{id}/report-recipients` | GET, POST | Email recipients of the reports and delivery status, with sandbox mode for Amazon SES | | `/projects/{id}/report-recipients/test` | POST | Send a test email to the recipients | | `/reports/{id}/email` | POST | Send a report by email again | | `/reports/{id}/email-preview` | GET | Email of a report as HTML | | `/projects/{id}/webhook/test` | POST | Send a test message to Slack, Discord and your own URL, or to a single target (channel) | ### Account integrations `projects:read` to read, `projects:write` to connect, map and disconnect. | Endpoint | Methods | What it does | | --- | --- | --- | | `/integrations` | GET | Google, Bing, PostHog and Plausible with assignment per project | | `/integrations/google/authorize` | POST | Authorise Google for Search Console and Analytics | | `/integrations/{bing\|posthog\|plausible}` | POST, DELETE | Connect with an API key or disconnect | | `/integrations/{provider}/sync` | POST | Reassign projects | ### Search data `projects:read` to read, `projects:write` to connect and disconnect. | Endpoint | Methods | What it does | | --- | --- | --- | | `/projects/{id}/search-console` | GET | Connections to Google Search Console and Bing Webmaster Tools | | `/projects/{id}/search-console/google/authorize` | POST | Create the authorisation URL for Google | | `/projects/{id}/search-console/bing` | POST | Connect Bing Webmaster Tools with an API key | | `/projects/{id}/search-console/{provider}` | PATCH, DELETE | Choose a property or disconnect | | `/projects/{id}/search-performance` | GET | Clicks, impressions, CTR and position per query or page, from the stored history when it fully covers the period, otherwise live | ### Web analytics `projects:read` to read, `projects:write` to connect and change. | Endpoint | Methods | What it does | | --- | --- | --- | | `/projects/{id}/analytics` | GET | Connections to PostHog, Plausible and Google Analytics | | `/projects/{id}/analytics/{provider}` | POST | Connect PostHog or Plausible with an API key | | `/projects/{id}/analytics/google_analytics/authorize` | POST | Create the authorisation URL for Google Analytics | | `/projects/{id}/analytics/{provider}` | PATCH, DELETE | Choose a property or disconnect | | `/projects/{id}/analytics-report` | GET | Visitors, visits, bounce rate per day, top pages, entry pages, sources and devices | | `/projects/{id}/analytics/{provider}/conversion-events` | GET, PUT | Read the provider’s events and choose the events that count as a conversion (at most 10) | | `/projects/{id}/conversions` | GET | Sessions, conversions and conversion rate per entry page from the daily sync | ### Recommendations and codebase `recommendations:read` and `recommendations:write`, `codebase:read` and `codebase:write`. | Endpoint | Methods | What it does | | --- | --- | --- | | `/projects/{id}/recommendations` | GET | Recommendations (“opportunities”) with impact, confidence, effort, evidence and data sources, filterable by status and rule | | `/recommendations/{id}` | GET, PATCH | Read a recommendation, accept it, dismiss it, mark it as done or reopen it | | `/projects/{id}/codebase-snapshots` | GET, POST | List and upload codebase snapshots (at most 2 MB, 20 per day and project, free of charge) | | `/codebase-snapshots/{id}` | GET | Snapshot with routes, mapping to live pages, deviations and diff to the previous snapshot | ### Tasks and jobs `tasks:read` and `tasks:write` for tasks. Jobs follow the scope of their type: `audit:read` for crawls, `rankings:read` for the other jobs. | Endpoint | Methods | What it does | | --- | --- | --- | | `/projects/{id}/tasks` | GET, POST | List and create tasks | | `/tasks/{id}` | GET, PATCH, DELETE | Read a task, change its status, delete it | | `/jobs` | GET | Running and finished background jobs | | `/jobs/{id}/cancel` | POST | Cancel a job | | `/projects/{id}/activity` | GET | History of a project: jobs, reports by email and keyword research with duration and cost | ### More endpoints These endpoints exist as well and are used by the CLI, but the index does not list them. | Endpoint | Methods | What it does | Scope | | --- | --- | --- | --- | | `/auth/device` | POST | Start a device login, as `serpel auth login` does | No token | | `/auth/device/token` | POST | Exchange the device code for a token once the user approved it | No token | | `/auth/logout` | POST | Revoke the token that sends the request | Any valid token | | `/jobs/{id}` | GET | Read one job with status, progress, result and last error | `audit:read` for crawls, `rankings:read` for the other jobs | | `/jobs/{id}/retry` | POST | Restart a failed or cancelled job | `audit:write` for crawls, `rankings:write` for the other jobs | | `/projects/{id}/rankings/summary` | GET | Top 3, top 10, average position and changes of all tracked keywords | `rankings:read` | | `/rank-keywords/{id}` | DELETE | Remove a tracked keyword with its history | `rankings:write` | | `/projects/{id}/competitors/{competitorId}` | DELETE | Remove a competitor | `rankings:write` | | `/ai-prompts/{id}/history` | GET | Check history of a prompt for ChatGPT and Google AI Overviews | `rankings:read` | | `/ai-prompts/{id}` | DELETE | Remove a prompt with all its results | `rankings:write` | | `/keywords/research/{runId}` | GET | Read a saved research run without querying the provider | `keywords:read` | | `/keyword-lists/{id}` | GET, PATCH, DELETE | Read, rename or delete a keyword list | `keywords:read` to read, `keywords:write` to change | | `/keyword-lists/{id}/items` | GET, POST | List the keywords of a list or add keywords | `keywords:read` to read, `keywords:write` to add | | `/keyword-lists/{id}/items/{itemId}` | DELETE | Remove a keyword from a list | `keywords:write` | | `/keyword-lists/{id}/import` | POST | Import keywords from CSV text | `keywords:write` | | `/keyword-lists/{id}/export` | GET | Export a keyword list as a CSV or JSON file | `export:read` | | `/projects/{id}/members/{userId}` | DELETE | Remove a member from a project | `projects:write`, owner role | | `/projects/{id}/report-recipients/{recipientId}` | DELETE | Remove an email recipient of the reports | `projects:write` | | `/reports/{id}` | GET | Read one report | `projects:read` | | `/reports/{id}/read` | POST | Mark a report as read | `projects:read` | | `/reports/read-all` | POST | Mark all reports, or those of one project, as read | `projects:read` | | `/reports/unread-count` | GET | Number of unread reports | `projects:read` | | `/projects/{id}/favicon` | GET | Website icon of a project, fetched server-side | `projects:read` | ## What do complete API examples look like? The examples assume that `SERPEL_TOKEN` holds a token with the scopes named in each step. The responses are abridged to the fields you need next, and their values are illustrative. The real responses contain more fields. ### Create a project Needs `projects:write`. The API returns HTTP 201. If you already have a project for the domain, it answers with 409 `conflict` and the ID of the existing project in `details.existingProjectId`. ```bash curl -X POST https://serpel.app/api/v1/projects \ -H "Authorization: Bearer $SERPEL_TOKEN" \ -H "Content-Type: application/json" \ -d '{"domain": "example.com", "name": "Example", "country": "US", "language": "en"}' ``` ```json { "data": { "id": "0b6f0f0e-6d1a-4c8e-9a53-4f9d2d7b8a10", "name": "Example", "domain": "example.com", "country": "US", "language": "en", "role": "owner" } } ``` ### Start a crawl and read the issues Needs `audit:write` to start and `audit:read` to follow the job. The crawl runs in a worker, so the API answers with 202 and a job. If a crawl is already running, you get that crawl back with HTTP 200 and `alreadyRunning: true`. Poll `GET /jobs/{id}` until `status` is `succeeded`, `failed` or `cancelled`, then read the issues. `maxPages` can be 1 to 1000. ```bash curl -X POST https://serpel.app/api/v1/projects/$PROJECT_ID/crawls \ -H "Authorization: Bearer $SERPEL_TOKEN" \ -H "Content-Type: application/json" \ -d '{"maxPages": 200}' ``` ```json { "data": { "crawl": { "id": "7d2a4c1e-3b9f-4e0a-8c55-2f61b0a9d3e4", "status": "queued", "maxPages": 200, "pagesCrawled": 0 }, "job": { "id": "c41e8f27-5a6d-4b13-9d70-e8a2f4b6c915", "type": "crawl", "status": "queued" }, "alreadyRunning": false } } ``` ```bash curl https://serpel.app/api/v1/jobs/$JOB_ID \ -H "Authorization: Bearer $SERPEL_TOKEN" curl https://serpel.app/api/v1/projects/$PROJECT_ID/issues \ -H "Authorization: Bearer $SERPEL_TOKEN" ``` ### Research keywords Needs `keywords:write` and uses credits. A keyword research costs 8 credits plus 4 credits per block of 50 results, so 20 results cost 12 credits. The `Idempotency-Key` makes a retry safe. `credits.charged` and `credits.balance` show what the call cost and what is left. ```bash curl -X POST https://serpel.app/api/v1/keywords/research \ -H "Authorization: Bearer $SERPEL_TOKEN" \ -H "Content-Type: application/json" \ -H "Idempotency-Key: research-cold-brew-2026-10-10" \ -d '{"mode": "keyword", "keyword": "cold brew concentrate", "country": "US", "language": "en", "limit": 20}' ``` ```json { "data": { "runId": "5e9b7a30-1c4d-4f82-b6a1-0d3c8e2f7a46", "mode": "keyword", "query": "cold brew concentrate", "country": "US", "language": "en", "cached": false, "items": [ { "keyword": "cold brew concentrate", "relation": "seed", "searchVolume": 2900, "cpc": 1.45, "keywordDifficulty": 38, "intent": "commercial" } ], "credits": { "charged": 12, "balance": 488 } } } ``` ### Read rankings Needs `rankings:read`. Each tracked keyword has its `latest` check and a `change` against the previous one; a positive change means the keyword moved up. A `position` of `null` with status `ok` means the page was not found within the checked depth. ```bash curl "https://serpel.app/api/v1/projects/$PROJECT_ID/rankings?limit=2&q=cold%20brew" \ -H "Authorization: Bearer $SERPEL_TOKEN" ``` ```json { "data": [ { "id": "a3f1c9d2-6b8e-4d57-9c20-7e4b1a5d8f03", "keyword": "cold brew concentrate", "country": "US", "language": "en", "device": "desktop", "searchVolume": 2900, "latest": { "checkedAt": "2026-10-09T06:00:12.000Z", "depth": 50, "position": 11, "url": "https://example.com/cold-brew-concentrate", "status": "ok" }, "change": 3 } ], "pagination": { "limit": 2, "offset": 0, "total": 1 } } ``` > **Prefer the CLI for scripts that wait on jobs:** The [CLI](https://serpel.app/docs/cli) wraps these calls, polls jobs with `--wait` and maps errors to exit codes, so you write less glue code. --- # Connect Claude Code to Serpel over MCP URL: https://serpel.app/docs/mcp/claude-code Updated: 2026-10-10 Run one `claude mcp add` command to connect Claude Code to the Serpel MCP server at `https://serpel.app/mcp`. You sign in with your Serpel account in the browser, and Claude Code can then read your rankings, crawl issues and Search Console data. The server offers 13 tools, and only one of them uses credits. ## How do you add the Serpel MCP server to Claude Code? 1. **Add the server.** Run this command in your terminal. `serpel` is the name you give the server, and `--transport http` selects the remote HTTP transport. ```bash claude mcp add --transport http serpel https://serpel.app/mcp ``` 2. **Sign in.** Start Claude Code and run `/mcp`. Select `serpel` and follow the steps in your browser: sign in to Serpel, choose the workspace and allow access. If the browser does not open, copy the address that Claude Code shows. You can also start the sign-in from the shell with `claude mcp login serpel`. ```text /mcp ``` 3. **Check the connection.** `claude mcp list` shows the status of every server. Serpel should read as connected. Then ask Claude Code a question about your site. ```bash claude mcp list claude mcp get serpel ``` ### Choose the scope of the server By default, Claude Code stores the server for the current project and only for you. `--scope` changes that. | Scope | Flag | Where it applies | | --- | --- | --- | | Local | `--scope local` (default) | Only you, only in the current project. | | Project | `--scope project` | Everyone who uses the repository. Claude Code writes `.mcp.json` and asks each person to approve it. | | User | `--scope user` | Only you, in all projects. | With `--scope project`, the entry in `.mcp.json` looks like this. ```json { "mcpServers": { "serpel": { "type": "http", "url": "https://serpel.app/mcp" } } } ``` The file contains no secret. Every teammate signs in with their own Serpel account and sees only their own projects. ## How does the OAuth sign-in work? Claude Code follows the MCP authorisation specification: OAuth 2.1 with PKCE (S256). The server answers an unauthenticated request with HTTP 401 and a `WWW-Authenticate` header that points to the discovery documents. The client registers itself, opens `/oauth/authorize` in your browser and exchanges the code at `/oauth/token`. You do not need to create a client ID or a secret. On the consent page you sign in to Serpel if needed, choose the workspace and review the permissions. The access token gets these scopes: `mcp:read`, `projects:read`, `rankings:read`, `audit:read`, `keywords:read`, `keywords:write`, `recommendations:read`, `codebase:read`. It only works at the MCP server. The REST API rejects it. Access tokens last 1 hour. Refresh tokens last 30 days and rotate each time they are used, so you stay signed in without doing anything. | Address | Purpose | | --- | --- | | `https://serpel.app/.well-known/oauth-protected-resource/mcp` | Describes the MCP server as an OAuth resource. | | `https://serpel.app/.well-known/oauth-authorization-server` | Describes the authorisation server: endpoints, PKCE, supported grants. | | `https://serpel.app/oauth/register` | Dynamic client registration. | | `https://serpel.app/oauth/authorize` | The consent page in your browser. | | `https://serpel.app/oauth/token` | Exchanges a code or a refresh token for an access token. | | `https://serpel.app/oauth/revoke` | Revokes a token and the whole connection. | To disconnect an app, open Settings, API, Connected apps in the dashboard. Disconnecting revokes the access and refresh tokens at once. ## How do you connect with an API token instead? Use an API token when Claude Code cannot open a browser, for example on a server or in CI. Create the token in the dashboard under Settings, API tokens. It needs the `mcp:read` scope, plus the scopes of the tools you want to use: - `mcp:read`: required for every call. Without it, the server answers with HTTP 403. - `projects:read`: `list_projects`, `get_project_overview`, `get_search_performance`, `find_keyword_opportunities`, `get_conversion_performance`. - `rankings:read`: `get_keyword_rankings`, `get_ai_visibility`, `analyze_competitors`, and `get_job_status` for rank checks, AI checks and monitoring runs. - `audit:read`: `get_crawl_issues`, and `get_job_status` for crawls. - `recommendations:read`: `get_seo_recommendations`. - `codebase:read`: `get_codebase_overview`. - `keywords:write`: `research_keywords`. You can restrict the token to single projects. The tools then see only those projects. Put the token in an environment variable and reference it in `.mcp.json`. Claude Code expands `${SERPEL_TOKEN}` from your environment, so the token never lands in the file. ```json { "mcpServers": { "serpel": { "type": "http", "url": "https://serpel.app/mcp", "headers": { "Authorization": "Bearer ${SERPEL_TOKEN}" } } } } ``` You can also pass the header on the command line with `--header`. The shell expands the variable when you run the command, and Claude Code stores the resulting token in its own configuration file, so prefer the `.mcp.json` variant. ```bash claude mcp add --transport http serpel https://serpel.app/mcp --header "Authorization: Bearer $SERPEL_TOKEN" ``` ## Which tools does the Serpel MCP server provide? The server offers 13 tools. All of them check the scope, your project role and your workspace before they return data, so an agent only sees what you could see yourself. Lists hold at most 25 rows, and every result is compact JSON with a short summary sentence. | Tool | What it does | Needs scope | Cost | | --- | --- | --- | --- | | `list_projects` | Lists your projects with ID, domain, market and your role. Call it first: every other tool needs a project ID from this list. | `projects:read` | Free | | `get_project_overview` | Summarises one project: audit score and issue counts, ranking summary, AI visibility, task counts, the most important issues and next steps. Sections the token has no scope for are left out. | `projects:read` | Free | | `get_keyword_rankings` | Shows tracked keywords with the latest and previous Google position, the change and a ranking summary. Supports `query`, `limit` and `offset`. | `rankings:read` | Free | | `get_search_performance` | Shows clicks, impressions, click-through rate and average position from Google Search Console or Bing Webmaster Tools, compared with the previous period, plus the top queries or pages. Needs a search data connection in the dashboard. | `projects:read` | Free | | `find_keyword_opportunities` | Finds queries on positions 4 to 20 with impressions, sorted by impressions. These are the quickest ranking wins. Needs a Search Console connection. | `projects:read` | Free | | `get_crawl_issues` | Lists the issues of the latest completed crawl by type, sorted by severity and affected pages, with the audit score and the change since the previous crawl. | `audit:read` | Free | | `get_ai_visibility` | Shows how often ChatGPT cites or mentions your website for your tracked prompts and how often Google shows an AI Overview that cites it, with the latest result per prompt and the sources the answers cite most. Reads stored checks and starts no new check. Supports `limit`. | `rankings:read` | Free | | `get_seo_recommendations` | Lists prioritised recommendations with rule, impact, estimated monthly clicks, confidence, effort, keyword, page, code route, next action and evidence. By default only open and accepted ones. | `recommendations:read` | Free | | `get_codebase_overview` | Summarises the latest snapshot uploaded with `serpel scan`: routes without title or description, routes without a live page, live pages without a route and differences between code and live site. | `codebase:read` | Free | | `get_conversion_performance` | Shows sessions, conversions and conversion rate per landing page from PostHog, Plausible or Google Analytics 4, for the conversion events you chose in the dashboard. | `projects:read` | Free | | `analyze_competitors` | Compares the project with its saved competitors using the stored Google results: visibility per domain, how often each competitor is ahead and the keywords where one ranks better. Runs no new searches. | `rankings:read` | Free | | `get_job_status` | Checks a crawl, rank check, AI visibility check or monitoring run by job ID: status, progress, error and a summary of the result. | `audit:read` for crawls, `rankings:read` for the other jobs | Free | | `research_keywords` | Looks up keyword ideas or metrics: search volume, CPC, keyword difficulty and search intent. Modes: `keyword`, `list` and `domain`. Returns at most 25 results and saves the full run in the dashboard. | `keywords:write` | Uses credits | All tools except `research_keywords` are read-only and free. They never change your data or your website. Crawls, rank checks, AI visibility checks and monitoring runs do not start inside a tool call. You start them in the dashboard, with the [CLI](https://serpel.app/docs/cli) or through the [REST API](https://serpel.app/docs/api), and then follow them with `get_job_status`. Tools that need a data source, such as Search Console, tell the agent what to connect in the dashboard when it is missing. In apps that can show interactive cards, such as ChatGPT and Claude, `get_keyword_rankings`, `get_crawl_issues`, `find_keyword_opportunities` and `get_ai_visibility` also show their result as a card. Every other client gets the same data as text. ChatGPT does not offer `research_keywords`. ## How do agent spending and budgets work? `research_keywords` is the only tool that uses credits, and it runs only when every one of these conditions holds. Serpel checks them in this order: 1. The token has the `keywords:write` scope. OAuth sign-in grants it. 2. The workspace has an active Starter or Pro plan. 3. The workspace owner allowed agent spending: “Agents can spend credits” under Settings, Billing, Agents and AI apps, or `serpel agents allow`. 4. One call costs at most 50 credits, and today’s agent spending plus the call stays within the daily limit. The default limit is 500 credits per day. You can set it between 1 and 100,000 with `serpel agents daily-limit <credits>`. The day follows UTC and resets at 00:00 UTC. 5. The workspace has enough credits. Serpel uses the monthly plan allowance first and then credits you bought. A research for 25 keyword ideas costs 12 credits, and a research of the keywords a domain ranks for costs 6 credits. Fewer results cost fewer credits. Every result states the credits charged (`credits.charged`), your balance, the remaining monthly allowance and the remaining agent budget for the day (`dailyAgentBudget`). If the agent repeats a call after a network error, it passes the same `requestId`, and the research is charged once. | Error code | When it happens | Extra fields | | --- | --- | --- | | `forbidden` | The token has no `keywords:write` scope. | None | | `plan_required` | The workspace has no Starter or Pro plan. | `pricingUrl` | | `agent_spend_disabled` | The owner has not allowed agents to spend credits. | `settingsUrl` | | `validation_error` | The call could cost more than 50 credits, or the input is invalid. | None | | `daily_cap_reached` | The call would exceed the daily limit for agents. | `dailyCapCredits`, `spentTodayCredits`, `requiredCredits`, `resetsAt` | | `insufficient_credits` | The workspace does not have enough credits. | `requiredCredits`, `availableCredits`, `nextAllowanceDate`, `pricingUrl` | Tool results only link to the dashboard and to the [pricing page](https://serpel.app/pricing). They never link to a checkout or a top-up, so an agent cannot buy credits for you. ## Which prompts work well in Claude Code? Ask in plain language. Claude Code picks the tools and starts with `list_projects` to find the project ID. - “List my Serpel projects and show the audit score of the one for example.com.” - “Which tracked keywords lost the most positions since the last rank check? Check the latest crawl issues for the pages that dropped.” - “Find queries on positions 4 to 20 for my project and tell me which page in this repository I should improve first.” - “Show the open Serpel recommendations and fix the one with the highest impact in the code.” - “Compare my rankings with my competitors and list the keywords where a competitor is ahead.” - “Research keyword ideas for ‘cold brew concentrate’ in the US. Tell me what it costs before you run it.” For prompts that connect SEO findings to your source files, upload a snapshot of your routes first with `serpel scan --project <id>`. `get_codebase_overview` and `get_seo_recommendations` use it. The [CLI reference](https://serpel.app/docs/cli) explains the scan. ## Is there a Serpel plugin for Claude Code? Serpel has a plugin package with a skill that tells the agent which tool to use for which question. It is not published to any plugin marketplace yet, so the `claude mcp add` command above is the supported way to connect. This page will change when the plugin is published. ## How do you troubleshoot the connection? Run `/mcp` in Claude Code to see the status of the server, and `claude mcp list` in the shell. Removing the server with `claude mcp remove serpel` also deletes its stored OAuth tokens, so you can start the sign-in again from scratch. In `/mcp`, “Clear authentication” revokes the stored access. | What you see | What to do | | --- | --- | | HTTP 401, or the client reports that it needs authentication | The client has no valid token. Sign in again with OAuth, or check that your API token is correct, not expired and not revoked. | | HTTP 403 with `insufficient_scope` | The API token has no `mcp:read` scope. Create a token that includes it. | | A tool answers `forbidden` | The token or your project role lacks the scope the tool needs. The tool table above names the scope for each tool. | | A tool answers `not_found` | The project or job does not exist, or your token is restricted to other projects. Call `list_projects` and use an ID from that list. | | `get_search_performance` or `find_keyword_opportunities` ask for a connection | Connect Google Search Console or Bing Webmaster Tools for the project in the dashboard. The tool result contains the link. | | `analyze_competitors` returns no data | It reads stored results. Run a rank check and add competitors first. | | HTTP 429 or `rate_limited` | A token may send 120 requests per minute. Wait for the seconds in `Retry-After` or `retryAfterSeconds`. | | HTTP 405 on a GET request | Expected. The server is stateless and only accepts POST requests with JSON-RPC messages. MCP clients handle this themselves. | More about `claude mcp` and the configuration scopes is in the [Claude Code MCP documentation](https://code.claude.com/docs/en/mcp). --- # Connect Cursor to Serpel over MCP URL: https://serpel.app/docs/mcp/cursor Updated: 2026-10-10 Add one entry to `mcp.json` to connect Cursor to the Serpel MCP server at `https://serpel.app/mcp`, then sign in with your Serpel account through OAuth. Cursor’s agent can then read your rankings, crawl issues and search data while it edits your code. The server offers 13 tools, and only one of them uses credits. ## How do you add the Serpel MCP server to Cursor? 1. **Create the config entry.** Cursor reads `.cursor/mcp.json` in your project for project-specific servers and `~/.cursor/mcp.json` in your home directory for servers that every project can use. A remote server needs only a `url`. ```json { "mcpServers": { "serpel": { "url": "https://serpel.app/mcp" } } } ``` 2. **Sign in.** Cursor supports OAuth for remote servers that require it. Sign in to Serpel in the browser, choose the workspace and allow access. Serpel’s consent page names the target app when the redirect goes back to Cursor. 3. **Check the server.** Make sure Cursor lists `serpel` in its MCP server settings and shows its tools. Then ask the agent a question about your site. ## How does the OAuth sign-in work? Cursor follows the MCP authorisation specification: OAuth 2.1 with PKCE (S256). The server answers an unauthenticated request with HTTP 401 and a `WWW-Authenticate` header that points to the discovery documents. The client registers itself, opens `/oauth/authorize` in your browser and exchanges the code at `/oauth/token`. You do not need to create a client ID or a secret. On the consent page you sign in to Serpel if needed, choose the workspace and review the permissions. The access token gets these scopes: `mcp:read`, `projects:read`, `rankings:read`, `audit:read`, `keywords:read`, `keywords:write`, `recommendations:read`, `codebase:read`. It only works at the MCP server. The REST API rejects it. Access tokens last 1 hour. Refresh tokens last 30 days and rotate each time they are used, so you stay signed in without doing anything. | Address | Purpose | | --- | --- | | `https://serpel.app/.well-known/oauth-protected-resource/mcp` | Describes the MCP server as an OAuth resource. | | `https://serpel.app/.well-known/oauth-authorization-server` | Describes the authorisation server: endpoints, PKCE, supported grants. | | `https://serpel.app/oauth/register` | Dynamic client registration. | | `https://serpel.app/oauth/authorize` | The consent page in your browser. | | `https://serpel.app/oauth/token` | Exchanges a code or a refresh token for an access token. | | `https://serpel.app/oauth/revoke` | Revokes a token and the whole connection. | To disconnect an app, open Settings, API, Connected apps in the dashboard. Disconnecting revokes the access and refresh tokens at once. Serpel supports dynamic client registration, so you do not need the optional `auth` block with a client ID that Cursor offers for servers without it. Serpel accepts the native `cursor://` redirect address and loopback addresses on `localhost`, so the browser can hand you back to Cursor. ## How do you connect with an API token instead? Use an API token when Cursor cannot open a browser, for example on a server or in CI. Create the token in the dashboard under Settings, API tokens. It needs the `mcp:read` scope, plus the scopes of the tools you want to use: - `mcp:read`: required for every call. Without it, the server answers with HTTP 403. - `projects:read`: `list_projects`, `get_project_overview`, `get_search_performance`, `find_keyword_opportunities`, `get_conversion_performance`. - `rankings:read`: `get_keyword_rankings`, `get_ai_visibility`, `analyze_competitors`, and `get_job_status` for rank checks, AI checks and monitoring runs. - `audit:read`: `get_crawl_issues`, and `get_job_status` for crawls. - `recommendations:read`: `get_seo_recommendations`. - `codebase:read`: `get_codebase_overview`. - `keywords:write`: `research_keywords`. You can restrict the token to single projects. The tools then see only those projects. Reference the token through an environment variable. Cursor resolves `${env:NAME}` in the `url` and `headers` fields. Remote servers do not support `envFile`, so set `SERPEL_TOKEN` in your shell profile or system environment. ```json { "mcpServers": { "serpel": { "url": "https://serpel.app/mcp", "headers": { "Authorization": "Bearer ${env:SERPEL_TOKEN}" } } } } ``` ## Which tools does the Serpel MCP server provide? The server offers 13 tools. All of them check the scope, your project role and your workspace before they return data, so an agent only sees what you could see yourself. Lists hold at most 25 rows, and every result is compact JSON with a short summary sentence. | Tool | What it does | Needs scope | Cost | | --- | --- | --- | --- | | `list_projects` | Lists your projects with ID, domain, market and your role. Call it first: every other tool needs a project ID from this list. | `projects:read` | Free | | `get_project_overview` | Summarises one project: audit score and issue counts, ranking summary, AI visibility, task counts, the most important issues and next steps. Sections the token has no scope for are left out. | `projects:read` | Free | | `get_keyword_rankings` | Shows tracked keywords with the latest and previous Google position, the change and a ranking summary. Supports `query`, `limit` and `offset`. | `rankings:read` | Free | | `get_search_performance` | Shows clicks, impressions, click-through rate and average position from Google Search Console or Bing Webmaster Tools, compared with the previous period, plus the top queries or pages. Needs a search data connection in the dashboard. | `projects:read` | Free | | `find_keyword_opportunities` | Finds queries on positions 4 to 20 with impressions, sorted by impressions. These are the quickest ranking wins. Needs a Search Console connection. | `projects:read` | Free | | `get_crawl_issues` | Lists the issues of the latest completed crawl by type, sorted by severity and affected pages, with the audit score and the change since the previous crawl. | `audit:read` | Free | | `get_ai_visibility` | Shows how often ChatGPT cites or mentions your website for your tracked prompts and how often Google shows an AI Overview that cites it, with the latest result per prompt and the sources the answers cite most. Reads stored checks and starts no new check. Supports `limit`. | `rankings:read` | Free | | `get_seo_recommendations` | Lists prioritised recommendations with rule, impact, estimated monthly clicks, confidence, effort, keyword, page, code route, next action and evidence. By default only open and accepted ones. | `recommendations:read` | Free | | `get_codebase_overview` | Summarises the latest snapshot uploaded with `serpel scan`: routes without title or description, routes without a live page, live pages without a route and differences between code and live site. | `codebase:read` | Free | | `get_conversion_performance` | Shows sessions, conversions and conversion rate per landing page from PostHog, Plausible or Google Analytics 4, for the conversion events you chose in the dashboard. | `projects:read` | Free | | `analyze_competitors` | Compares the project with its saved competitors using the stored Google results: visibility per domain, how often each competitor is ahead and the keywords where one ranks better. Runs no new searches. | `rankings:read` | Free | | `get_job_status` | Checks a crawl, rank check, AI visibility check or monitoring run by job ID: status, progress, error and a summary of the result. | `audit:read` for crawls, `rankings:read` for the other jobs | Free | | `research_keywords` | Looks up keyword ideas or metrics: search volume, CPC, keyword difficulty and search intent. Modes: `keyword`, `list` and `domain`. Returns at most 25 results and saves the full run in the dashboard. | `keywords:write` | Uses credits | All tools except `research_keywords` are read-only and free. They never change your data or your website. Crawls, rank checks, AI visibility checks and monitoring runs do not start inside a tool call. You start them in the dashboard, with the [CLI](https://serpel.app/docs/cli) or through the [REST API](https://serpel.app/docs/api), and then follow them with `get_job_status`. Tools that need a data source, such as Search Console, tell the agent what to connect in the dashboard when it is missing. In apps that can show interactive cards, such as ChatGPT and Claude, `get_keyword_rankings`, `get_crawl_issues`, `find_keyword_opportunities` and `get_ai_visibility` also show their result as a card. Every other client gets the same data as text. ChatGPT does not offer `research_keywords`. ## How do agent spending and budgets work? `research_keywords` is the only tool that uses credits, and it runs only when every one of these conditions holds. Serpel checks them in this order: 1. The token has the `keywords:write` scope. OAuth sign-in grants it. 2. The workspace has an active Starter or Pro plan. 3. The workspace owner allowed agent spending: “Agents can spend credits” under Settings, Billing, Agents and AI apps, or `serpel agents allow`. 4. One call costs at most 50 credits, and today’s agent spending plus the call stays within the daily limit. The default limit is 500 credits per day. You can set it between 1 and 100,000 with `serpel agents daily-limit <credits>`. The day follows UTC and resets at 00:00 UTC. 5. The workspace has enough credits. Serpel uses the monthly plan allowance first and then credits you bought. A research for 25 keyword ideas costs 12 credits, and a research of the keywords a domain ranks for costs 6 credits. Fewer results cost fewer credits. Every result states the credits charged (`credits.charged`), your balance, the remaining monthly allowance and the remaining agent budget for the day (`dailyAgentBudget`). If the agent repeats a call after a network error, it passes the same `requestId`, and the research is charged once. | Error code | When it happens | Extra fields | | --- | --- | --- | | `forbidden` | The token has no `keywords:write` scope. | None | | `plan_required` | The workspace has no Starter or Pro plan. | `pricingUrl` | | `agent_spend_disabled` | The owner has not allowed agents to spend credits. | `settingsUrl` | | `validation_error` | The call could cost more than 50 credits, or the input is invalid. | None | | `daily_cap_reached` | The call would exceed the daily limit for agents. | `dailyCapCredits`, `spentTodayCredits`, `requiredCredits`, `resetsAt` | | `insufficient_credits` | The workspace does not have enough credits. | `requiredCredits`, `availableCredits`, `nextAllowanceDate`, `pricingUrl` | Tool results only link to the dashboard and to the [pricing page](https://serpel.app/pricing). They never link to a checkout or a top-up, so an agent cannot buy credits for you. ## Which prompts work well in Cursor? Ask in the agent chat, in plain language. The agent starts with `list_projects` to find the project ID. - “Use Serpel to list the crawl issues of my project and fix the pages with a missing title in this repository.” - “Which routes in this repository have no title or description? Use the Serpel codebase overview.” - “Show my keyword rankings and tell me which tracked keyword lost the most positions.” - “Find queries on positions 4 to 20 and suggest edits for the matching pages.” - “Compare me with my competitors and list the keywords where a competitor ranks better.” - “Has the crawl with this job ID finished?” The codebase overview needs a snapshot of your routes. Upload one with `serpel scan --project <id>`. The [CLI reference](https://serpel.app/docs/cli) explains the scan. ## Can you share the Serpel server with your team? Yes. Commit `.cursor/mcp.json` with the `url` entry only. The file holds no secret. With OAuth, every teammate signs in with their own Serpel account and sees only the projects that account can access. Do not commit a token. ## How do you troubleshoot the connection? | What you see | What to do | | --- | --- | | The browser sign-in does not finish | Open Settings, API, Connected apps in the Serpel dashboard, disconnect the Cursor entry if it is listed, and start the sign-in again. | | Cursor does not list the server | Check that the JSON is valid and that the file is `.cursor/mcp.json` in the project or `~/.cursor/mcp.json` in your home directory. | | HTTP 401, or the client reports that it needs authentication | The client has no valid token. Sign in again with OAuth, or check that your API token is correct, not expired and not revoked. | | HTTP 403 with `insufficient_scope` | The API token has no `mcp:read` scope. Create a token that includes it. | | A tool answers `forbidden` | The token or your project role lacks the scope the tool needs. The tool table above names the scope for each tool. | | A tool answers `not_found` | The project or job does not exist, or your token is restricted to other projects. Call `list_projects` and use an ID from that list. | | `get_search_performance` or `find_keyword_opportunities` ask for a connection | Connect Google Search Console or Bing Webmaster Tools for the project in the dashboard. The tool result contains the link. | | `analyze_competitors` returns no data | It reads stored results. Run a rank check and add competitors first. | | HTTP 429 or `rate_limited` | A token may send 120 requests per minute. Wait for the seconds in `Retry-After` or `retryAfterSeconds`. | | HTTP 405 on a GET request | Expected. The server is stateless and only accepts POST requests with JSON-RPC messages. MCP clients handle this themselves. | More about `mcp.json`, OAuth and config interpolation is in the [Cursor MCP documentation](https://cursor.com/docs/mcp). --- # Connect Codex to Serpel over MCP URL: https://serpel.app/docs/mcp/codex Updated: 2026-10-10 Run `codex mcp add` to register the Serpel MCP server at `https://serpel.app/mcp`, then run `codex mcp login serpel` to sign in with your Serpel account. Codex can then read your rankings, crawl issues and search data while it works on your code. The server offers 13 tools, and only one of them uses credits. ## How do you add the Serpel MCP server to Codex? 1. **Add the server.** `--url` registers a streamable HTTP server. Codex stores the entry in `~/.codex/config.toml`. ```bash codex mcp add serpel --url https://serpel.app/mcp ``` 2. **Sign in.** This starts the OAuth login. Sign in to Serpel in the browser, choose the workspace and allow access. ```bash codex mcp login serpel ``` 3. **Check the server.** `codex mcp list` shows the configured servers and `codex mcp get serpel` shows one entry. Then ask Codex a question about your site. ```bash codex mcp list codex mcp get serpel ``` ### Edit config.toml directly You can write the same entry by hand. A `url` key makes Codex treat the server as a streamable HTTP server. Run `codex mcp login serpel` afterwards. ```toml [mcp_servers.serpel] url = "https://serpel.app/mcp" ``` ### Sign out or remove the server ```bash codex mcp logout serpel codex mcp remove serpel ``` ## How does the OAuth sign-in work? Codex follows the MCP authorisation specification: OAuth 2.1 with PKCE (S256). The server answers an unauthenticated request with HTTP 401 and a `WWW-Authenticate` header that points to the discovery documents. The client registers itself, opens `/oauth/authorize` in your browser and exchanges the code at `/oauth/token`. You do not need to create a client ID or a secret. On the consent page you sign in to Serpel if needed, choose the workspace and review the permissions. The access token gets these scopes: `mcp:read`, `projects:read`, `rankings:read`, `audit:read`, `keywords:read`, `keywords:write`, `recommendations:read`, `codebase:read`. It only works at the MCP server. The REST API rejects it. Access tokens last 1 hour. Refresh tokens last 30 days and rotate each time they are used, so you stay signed in without doing anything. | Address | Purpose | | --- | --- | | `https://serpel.app/.well-known/oauth-protected-resource/mcp` | Describes the MCP server as an OAuth resource. | | `https://serpel.app/.well-known/oauth-authorization-server` | Describes the authorisation server: endpoints, PKCE, supported grants. | | `https://serpel.app/oauth/register` | Dynamic client registration. | | `https://serpel.app/oauth/authorize` | The consent page in your browser. | | `https://serpel.app/oauth/token` | Exchanges a code or a refresh token for an access token. | | `https://serpel.app/oauth/revoke` | Revokes a token and the whole connection. | To disconnect an app, open Settings, API, Connected apps in the dashboard. Disconnecting revokes the access and refresh tokens at once. `codex mcp login` chooses the client registration method itself (`auto`). Serpel supports both dynamic client registration and client ID metadata documents, so the default works. You can force one for a single login with `--oauth-client-registration cimd` or `--oauth-client-registration dcr`. ## How do you connect with a bearer token instead? Use an API token when Codex cannot open a browser, for example on a server or in CI. Create the token in the dashboard under Settings, API tokens. It needs the `mcp:read` scope, plus the scopes of the tools you want to use: - `mcp:read`: required for every call. Without it, the server answers with HTTP 403. - `projects:read`: `list_projects`, `get_project_overview`, `get_search_performance`, `find_keyword_opportunities`, `get_conversion_performance`. - `rankings:read`: `get_keyword_rankings`, `get_ai_visibility`, `analyze_competitors`, and `get_job_status` for rank checks, AI checks and monitoring runs. - `audit:read`: `get_crawl_issues`, and `get_job_status` for crawls. - `recommendations:read`: `get_seo_recommendations`. - `codebase:read`: `get_codebase_overview`. - `keywords:write`: `research_keywords`. You can restrict the token to single projects. The tools then see only those projects. The OAuth login happens in a browser. On a machine without one, give Codex the name of an environment variable that holds the token. Codex sends its value as the bearer token and never writes it to `config.toml`. ```bash export SERPEL_TOKEN=vsk_your_token codex mcp add serpel --url https://serpel.app/mcp --bearer-token-env-var SERPEL_TOKEN ``` ```toml [mcp_servers.serpel] url = "https://serpel.app/mcp" bearer_token_env_var = "SERPEL_TOKEN" ``` ## Which tools does the Serpel MCP server provide? The server offers 13 tools. All of them check the scope, your project role and your workspace before they return data, so an agent only sees what you could see yourself. Lists hold at most 25 rows, and every result is compact JSON with a short summary sentence. | Tool | What it does | Needs scope | Cost | | --- | --- | --- | --- | | `list_projects` | Lists your projects with ID, domain, market and your role. Call it first: every other tool needs a project ID from this list. | `projects:read` | Free | | `get_project_overview` | Summarises one project: audit score and issue counts, ranking summary, AI visibility, task counts, the most important issues and next steps. Sections the token has no scope for are left out. | `projects:read` | Free | | `get_keyword_rankings` | Shows tracked keywords with the latest and previous Google position, the change and a ranking summary. Supports `query`, `limit` and `offset`. | `rankings:read` | Free | | `get_search_performance` | Shows clicks, impressions, click-through rate and average position from Google Search Console or Bing Webmaster Tools, compared with the previous period, plus the top queries or pages. Needs a search data connection in the dashboard. | `projects:read` | Free | | `find_keyword_opportunities` | Finds queries on positions 4 to 20 with impressions, sorted by impressions. These are the quickest ranking wins. Needs a Search Console connection. | `projects:read` | Free | | `get_crawl_issues` | Lists the issues of the latest completed crawl by type, sorted by severity and affected pages, with the audit score and the change since the previous crawl. | `audit:read` | Free | | `get_ai_visibility` | Shows how often ChatGPT cites or mentions your website for your tracked prompts and how often Google shows an AI Overview that cites it, with the latest result per prompt and the sources the answers cite most. Reads stored checks and starts no new check. Supports `limit`. | `rankings:read` | Free | | `get_seo_recommendations` | Lists prioritised recommendations with rule, impact, estimated monthly clicks, confidence, effort, keyword, page, code route, next action and evidence. By default only open and accepted ones. | `recommendations:read` | Free | | `get_codebase_overview` | Summarises the latest snapshot uploaded with `serpel scan`: routes without title or description, routes without a live page, live pages without a route and differences between code and live site. | `codebase:read` | Free | | `get_conversion_performance` | Shows sessions, conversions and conversion rate per landing page from PostHog, Plausible or Google Analytics 4, for the conversion events you chose in the dashboard. | `projects:read` | Free | | `analyze_competitors` | Compares the project with its saved competitors using the stored Google results: visibility per domain, how often each competitor is ahead and the keywords where one ranks better. Runs no new searches. | `rankings:read` | Free | | `get_job_status` | Checks a crawl, rank check, AI visibility check or monitoring run by job ID: status, progress, error and a summary of the result. | `audit:read` for crawls, `rankings:read` for the other jobs | Free | | `research_keywords` | Looks up keyword ideas or metrics: search volume, CPC, keyword difficulty and search intent. Modes: `keyword`, `list` and `domain`. Returns at most 25 results and saves the full run in the dashboard. | `keywords:write` | Uses credits | All tools except `research_keywords` are read-only and free. They never change your data or your website. Crawls, rank checks, AI visibility checks and monitoring runs do not start inside a tool call. You start them in the dashboard, with the [CLI](https://serpel.app/docs/cli) or through the [REST API](https://serpel.app/docs/api), and then follow them with `get_job_status`. Tools that need a data source, such as Search Console, tell the agent what to connect in the dashboard when it is missing. In apps that can show interactive cards, such as ChatGPT and Claude, `get_keyword_rankings`, `get_crawl_issues`, `find_keyword_opportunities` and `get_ai_visibility` also show their result as a card. Every other client gets the same data as text. ChatGPT does not offer `research_keywords`. ## How do agent spending and budgets work? `research_keywords` is the only tool that uses credits, and it runs only when every one of these conditions holds. Serpel checks them in this order: 1. The token has the `keywords:write` scope. OAuth sign-in grants it. 2. The workspace has an active Starter or Pro plan. 3. The workspace owner allowed agent spending: “Agents can spend credits” under Settings, Billing, Agents and AI apps, or `serpel agents allow`. 4. One call costs at most 50 credits, and today’s agent spending plus the call stays within the daily limit. The default limit is 500 credits per day. You can set it between 1 and 100,000 with `serpel agents daily-limit <credits>`. The day follows UTC and resets at 00:00 UTC. 5. The workspace has enough credits. Serpel uses the monthly plan allowance first and then credits you bought. A research for 25 keyword ideas costs 12 credits, and a research of the keywords a domain ranks for costs 6 credits. Fewer results cost fewer credits. Every result states the credits charged (`credits.charged`), your balance, the remaining monthly allowance and the remaining agent budget for the day (`dailyAgentBudget`). If the agent repeats a call after a network error, it passes the same `requestId`, and the research is charged once. | Error code | When it happens | Extra fields | | --- | --- | --- | | `forbidden` | The token has no `keywords:write` scope. | None | | `plan_required` | The workspace has no Starter or Pro plan. | `pricingUrl` | | `agent_spend_disabled` | The owner has not allowed agents to spend credits. | `settingsUrl` | | `validation_error` | The call could cost more than 50 credits, or the input is invalid. | None | | `daily_cap_reached` | The call would exceed the daily limit for agents. | `dailyCapCredits`, `spentTodayCredits`, `requiredCredits`, `resetsAt` | | `insufficient_credits` | The workspace does not have enough credits. | `requiredCredits`, `availableCredits`, `nextAllowanceDate`, `pricingUrl` | Tool results only link to the dashboard and to the [pricing page](https://serpel.app/pricing). They never link to a checkout or a top-up, so an agent cannot buy credits for you. ## Which prompts work well in Codex? Ask in plain language. Codex starts with `list_projects` to find the project ID. - “Use Serpel to find keywords that rank on positions 4 to 20 and propose three content changes for this repository.” - “Check the Serpel crawl issues for my project and fix the five most severe ones in the code.” - “Compare my tracked keywords with my competitors and write a short plan for the keywords where they are ahead.” - “Which routes in this repository have no title or description according to Serpel’s codebase overview?” - “Look up keyword ideas for ‘light roast coffee’ in the US. Show me the credit cost first and wait for my approval.” The codebase overview needs a snapshot of your routes. Upload one with `serpel scan --project <id>`. The [CLI reference](https://serpel.app/docs/cli) explains the scan. ## How do you troubleshoot the connection? | What you see | What to do | | --- | --- | | `codex mcp list` shows the server as not logged in | Run `codex mcp login serpel` and finish the browser sign-in. Without a credential source, Codex can connect without authentication, and Serpel then answers with HTTP 401. | | Codex cannot use the token from `bearer_token_env_var` | Export `SERPEL_TOKEN` in the shell that starts Codex, and check that the token is valid and has the `mcp:read` scope. | | HTTP 401, or the client reports that it needs authentication | The client has no valid token. Sign in again with OAuth, or check that your API token is correct, not expired and not revoked. | | HTTP 403 with `insufficient_scope` | The API token has no `mcp:read` scope. Create a token that includes it. | | A tool answers `forbidden` | The token or your project role lacks the scope the tool needs. The tool table above names the scope for each tool. | | A tool answers `not_found` | The project or job does not exist, or your token is restricted to other projects. Call `list_projects` and use an ID from that list. | | `get_search_performance` or `find_keyword_opportunities` ask for a connection | Connect Google Search Console or Bing Webmaster Tools for the project in the dashboard. The tool result contains the link. | | `analyze_competitors` returns no data | It reads stored results. Run a rank check and add competitors first. | | HTTP 429 or `rate_limited` | A token may send 120 requests per minute. Wait for the seconds in `Retry-After` or `retryAfterSeconds`. | | HTTP 405 on a GET request | Expected. The server is stateless and only accepts POST requests with JSON-RPC messages. MCP clients handle this themselves. | More about `codex mcp` and `config.toml` is in the [Codex MCP documentation](https://learn.chatgpt.com/docs/extend/mcp?surface=cli). --- # Free tools for technical SEO. No account needed. URL: https://serpel.app/tools These five tools run without an account and without credits. Validate structured data, trace redirect chains, check your XML sitemap and your robots.txt for AI crawlers, and create an llms.txt file that helps coding agents and AI assistants read your site. - [Redirect checker](https://serpel.app/tools/redirect-checker): Free redirect checker: trace every 301 and 302 hop of a URL, spot chains, loops and temporary redirects, and test as a browser or a crawler. - [Schema validator](https://serpel.app/tools/schema-validator): Free schema validator: test JSON-LD, Microdata and RDFa from a URL or pasted code and see errors, warnings and rich result eligibility. - [Robots.txt checker](https://serpel.app/tools/robots-txt-checker): Free robots.txt checker: see which of 13 AI crawlers can reach your homepage, list your sitemaps and check llms.txt. - [Sitemap checker](https://serpel.app/tools/sitemap-checker): Free sitemap checker: validate an XML sitemap, test its limits and URLs, and sample up to 20 listed pages for 404s, redirects and noindex headers. - [llms.txt generator](https://serpel.app/tools/llms-txt-generator): Free llms.txt generator: create a valid llms.txt for your website in the llmstxt.org format, then copy it to the root of your domain. --- # Redirect checker for 301 and 302 chains URL: https://serpel.app/tools/redirect-checker Updated: 2026-10-10 A redirect checker follows a URL through every redirect and shows each hop with its status code. This free tool traces 301, 302, 303, 307 and 308 redirects for up to 10 hops, flags chains, loops and downgrades to HTTP, and can send the request with a browser or crawler user agent. Enter a URL to see where it ends. ## What does a redirect checker do? A redirect checker requests a URL and follows every redirect the server answers with. This one lists each hop in order, with its status code, protocol, target and response time, so you see what a visitor or a crawler sees. Use it as a 301 redirect checker after a migration, a redirect chain checker for slow pages or a URL redirect checker to confirm that an old address ends on the right page. ## How do you use this redirect checker? 1. **Enter the URL** Type the address as people or links use it. Keep `http://` to see the upgrade to HTTPS, or leave the protocol out to start with HTTPS. 2. **Pick a user agent** Browser is the default. The Googlebot, Bingbot and GPTBot options send that crawler’s user agent string. 3. **Read the chain** Each card is one hop with its status code, protocol, response time and the Location it points to. The last card is marked Final. 4. **Fix what the recommendations name** Errors and warnings come first. Change the rule at the start of the chain, then run the check again. ## What is a redirect chain, and how many hops are too many? A redirect chain is a series of redirects between the URL you request and the page that finally answers, for example `http://example.com` to `https://example.com` to `https://www.example.com`. Every hop costs a round trip, so a chain slows the page down and delays crawlers. Google’s crawlers [follow up to 10 redirect hops](https://developers.google.com/crawling/docs/troubleshooting/http-status-codes) by default. [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-redirection-3xx) tells clients to detect and intervene in redirect loops, and notes that an older version recommended a maximum of five redirects. Aim for one hop. This tool warns from two redirects, reports a loop as soon as a URL repeats and stops after 10 redirects. ## 301, 302, 303, 307 and 308: which redirect should you use? Use 301 or 308 when a page has moved for good, and 302 or 307 when the move is temporary. [Google treats](https://developers.google.com/crawling/docs/troubleshooting/http-status-codes) 301 and 308 as a strong signal that the target should be canonical, and 302, 303 and 307 as a weak signal. The status also decides what happens to the request method, as [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-redirection-3xx) defines. **How the five redirect statuses differ** | Status | Type | Request method | Google’s signal | | --- | --- | --- | --- | | `301` Moved Permanently | Permanent | May change POST to GET, for historical reasons | Strong | | `308` Permanent Redirect | Permanent | Must not change the method | Strong | | `302` Found | Temporary | May change POST to GET, for historical reasons | Weak | | `307` Temporary Redirect | Temporary | Must not change the method | Weak | | `303` See Other | Temporary, points to another resource | Fetches the other resource with GET | Weak | ## What are the most common redirect problems? These findings appear most often, each with its fix: - **Redirect chain.** Several rules stack up, for example HTTP, www and path rules. Redirect the first URL straight to the final one and update internal links and your sitemap. - **Redirect loop.** Two rules send a URL back and forth, for example one that adds www and another that removes it. Decide on one canonical URL and delete the rule that contradicts it. - **Temporary redirect for a permanent move.** A `302` or `307` on a page that moved for good sends only a weak signal. Switch to `301` or `308`. - **Downgrade to HTTP.** A hop from HTTPS to HTTP leaves the connection unencrypted. Redirect to the `https://` URL instead. - **Ends at an error.** A chain that finishes at `404` or `410` leads nowhere. Point it at a live replacement, or remove the redirect and return the error from the old URL. A `5xx` at the end means the server failed. - **Everything goes to the homepage.** Google asks for a 301 when a clear replacement exists and a 404 or 410 when none does. Our guide to [soft 404 errors](https://serpel.app/blog/soft-404) explains why a blanket redirect fits neither. ## How do you redirect HTTP to HTTPS and www to non-www? Choose one canonical version of your site, for example `https://example.com`. Send the other combinations of protocol and host to it in a single `301`, then test each one. This nginx block redirects both HTTP hosts. The HTTPS www host needs its own `server` block with its certificate. ```text server { listen 80; server_name example.com www.example.com; return 301 https://example.com$request_uri; } ``` On Apache, `Redirect 301 /old-page https://example.com/new-page` moves a single path, and these rules in `.htaccess` send every other host and every HTTP request to the canonical URL: ```text RewriteEngine On RewriteCond %{HTTP_HOST} !^example\.com$ [NC,OR] RewriteCond %{HTTPS} off RewriteRule ^ https://example.com%{REQUEST_URI} [R=301,L] ``` In Next.js, `permanent: true` [sends a 308 status](https://nextjs.org/docs/app/api-reference/config/next-config-js/redirects) and `permanent: false` a 307. Our [Next.js SEO guide](https://serpel.app/blog/nextjs-seo) covers the rest of the setup. ```typescript import type { NextConfig } from "next"; const nextConfig: NextConfig = { async redirects() { return [ { source: "/old-page", destination: "/new-page", permanent: true }, ]; }, }; export default nextConfig; ``` ## Do meta refresh and JavaScript redirects count? A `<meta http-equiv="refresh">` tag redirects the browser from the page HTML, and a script can change `location`. [Google treats](https://developers.google.com/search/docs/crawling-indexing/301-redirects) an instant meta refresh and a JavaScript location redirect as permanent, and a meta refresh with a delay as temporary. It advises JavaScript redirects only if server-side or meta refresh redirects aren’t possible, because rendering can fail and Google may never see the redirect. See our [JavaScript SEO guide](https://serpel.app/blog/javascript-seo). ## What does this redirect checker not do? - It doesn’t run JavaScript, so it can’t see redirects that a script performs. It reports a meta refresh or Refresh header on the final page but never follows it. - It sends `GET` requests only, so it can’t show how a POST request is redirected. - The request comes from Serpel’s servers, asks for English pages with `Accept-Language: en` and sends no cookies, so redirects that depend on your country, language or login can differ from what you see. - The user agent strings are imitations. [Google notes](https://developers.google.com/search/docs/crawling-indexing/googlebot) that its user agent is often spoofed and verified by reverse DNS or published IP ranges, so a site that checks IP addresses can answer differently from how it answers the real crawler. - It checks one URL at a time. [Serpel’s site audit](https://serpel.app/features/site-audit) reports redirect chains and loops across every URL it crawls. ## Frequently asked questions ### What is the difference between a 301 and a 302 redirect? A 301 says that a page moved permanently and a 302 says that the move is temporary. [Google treats](https://developers.google.com/crawling/docs/troubleshooting/http-status-codes) 301 and 308 as a strong signal that the target should be canonical, and 302, 303 and 307 as a weak signal. Use a 301 or 308 whenever the old URL is not coming back. ### How many redirects are too many? Aim for one. Google’s crawlers follow up to 10 redirect hops by default, but every hop adds a request and delay, so this tool warns from two redirects. Redirect the first URL straight to the final URL instead of stacking rules. ### Should I use a 301 or a 308 redirect? Both are permanent, and Google treats both as a strong signal. The difference is the method: a 308 must not change it, while a 301 may turn a POST into a GET for historical reasons, as [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-redirection-3xx) explains. Use a 308 for form and API endpoints and either one for normal pages. ### Why does my redirect go in a loop? A loop means that two or more rules send a URL back to one that was already requested. Typical causes are a rule that adds www next to one that removes it, or a proxy that reaches your server over HTTP while the server forces HTTPS. The result shows which hop points back to which, so you can remove the rule that contradicts the other. ### Does the checker follow meta refresh and JavaScript redirects? No. It follows server-side redirects, meaning a 301, 302, 303, 307 or 308 response with a Location header. A meta refresh or Refresh header on the final page is reported, but not followed, and scripts aren’t executed. Google handles these redirects differently, as its [redirect documentation](https://developers.google.com/search/docs/crawling-indexing/301-redirects) describes. ### Can I check redirects as Googlebot? Pick Googlebot in the user agent list and Serpel sends that user agent string. The request still comes from Serpel’s servers and not from Google’s IP ranges, and [Google verifies](https://developers.google.com/search/docs/crawling-indexing/googlebot) the real Googlebot by IP. A site that checks IP addresses can therefore answer differently, so confirm the result with the URL Inspection tool in Search Console. --- # Schema validator for JSON-LD, Microdata and RDFa URL: https://serpel.app/tools/schema-validator Updated: 2026-10-10 A schema validator checks the structured data on a page and tells you what is wrong with it. This free schema validator reads JSON-LD, Microdata and RDFa from a live URL or from code you paste, reports errors and warnings with the exact property path and shows whether each item can qualify for a Google rich result. It needs no account. ## What does a schema validator check? Structured data is markup that describes a page to machines: a product with a price, an article with an author, a business with an address. A schema validator parses that markup and checks it against [schema.org](https://schema.org/) and, for the types Google supports, against Google’s requirements. For every item on the page, this tool shows the format, the errors and warnings with the property path (for example `Product.offers.price`), an eligibility note and the full property tree. Errors block eligibility: a required property is missing, a required value is invalid or the syntax is broken. Warnings point to recommended properties and optional values that look wrong. ## Which structured data formats does Google support? Google supports [JSON-LD, Microdata and RDFa](https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data) and recommends JSON-LD. All three are fine if the markup is valid. JSON-LD sits in its own `<script type="application/ld+json">` block, while Microdata and RDFa add attributes to the visible HTML elements. ## How do you validate schema markup? 1. **Choose what to test** Pick “Test a URL” for a live page, or “Paste code” for JSON-LD or an HTML snippet that is not published yet. 2. **Run the validation** Enter the page address or paste the code and press Validate. A page is fetched without running JavaScript, and the result appears in a few seconds. 3. **Fix the errors first** Read the property path and the message of each error, then correct the markup at its source. Groups with errors come first, and warnings follow the errors. 4. **Read the eligibility note** Each type says whether it can qualify for a Google rich result. Google has discontinued some, such as FAQ, how-to and the sitelinks search box, and the tool says so when it finds them. Google’s [structured data gallery](https://developers.google.com/search/docs/appearance/structured-data/search-gallery) lists what it supports today. 5. **Validate again** Run the check after every fix, then confirm important pages in Google’s [Rich Results Test](https://search.google.com/test/rich-results). ## What does valid JSON-LD look like? This Product markup passes with no errors and no warnings. It is the example that the Load example button inserts. Place the JSON in a `<script type="application/ld+json">` tag in the HTML of the product page. ```json { "@context": "https://schema.org", "@type": "Product", "name": "Aero 42 trail running shoe", "image": [ "https://example.com/images/aero-42.jpg" ], "description": "A light trail running shoe with a grippy outsole and a breathable mesh upper.", "sku": "AERO-42", "brand": { "@type": "Brand", "name": "Aero" }, "offers": { "@type": "Offer", "url": "https://example.com/products/aero-42", "price": "89.90", "priceCurrency": "EUR", "priceValidUntil": "2027-12-31", "availability": "https://schema.org/InStock" }, "aggregateRating": { "@type": "AggregateRating", "ratingValue": "4.6", "reviewCount": 128 } } ``` In Serpel’s rules a Product needs a `name` and at least one of `offers`, `review` or `aggregateRating`. A brand, an image, a product identifier and the offer details are recommended. ## Schema markup validator or Rich Results Test? Google [retired its Structured Data Testing Tool](https://developers.google.com/search/blog/2020/12/structured-data-testing-tool-update) in favour of two tools. The [Rich Results Test](https://search.google.com/test/rich-results) shows whether a page is eligible for Google’s rich results. The [Schema Markup Validator](https://validator.schema.org/) checks schema.org markup in general and is not specific to Google features. **Three ways to test structured data** | Tool | What it checks | Use it to | | --- | --- | --- | | This schema validator | Syntax, plus required and recommended properties for the types Serpel has rules for | Get quick feedback from a URL or pasted code | | Rich Results Test | Google’s requirements for rich results | Confirm eligibility for Google features | | Schema Markup Validator | schema.org markup in general | Check any schema.org type, including types Google ignores | ## What are common schema markup errors? Each example below shows the markup that fails and the fixed version. ### Missing required property A Product without a `name` fails with an error at `Product.name`. Add the property. ```json { "@context": "https://schema.org", "@type": "Product", "image": "https://example.com/images/aero-42.jpg", "offers": { "@type": "Offer", "price": "89.90", "priceCurrency": "EUR" } } ``` ```json { "@context": "https://schema.org", "@type": "Product", "name": "Aero 42 trail running shoe", "image": "https://example.com/images/aero-42.jpg", "offers": { "@type": "Offer", "price": "89.90", "priceCurrency": "EUR" } } ``` ### Price as text with a currency symbol The validator expects a plain number with a period as the decimal separator. Put the currency in `priceCurrency` as a three-letter ISO 4217 code. ```json { "@context": "https://schema.org", "@type": "Product", "name": "Aero 42 trail running shoe", "offers": { "@type": "Offer", "price": "€89,90", "priceCurrency": "EUR" } } ``` ```json { "@context": "https://schema.org", "@type": "Product", "name": "Aero 42 trail running shoe", "offers": { "@type": "Offer", "price": "89.90", "priceCurrency": "EUR" } } ``` ### Relative URL Values such as `image` and `url` need a full URL that starts with `https://`, not a path. ```json { "@context": "https://schema.org", "@type": "Product", "name": "Aero 42 trail running shoe", "image": "/images/aero-42.jpg", "offers": { "@type": "Offer", "price": "89.90", "priceCurrency": "EUR" } } ``` ```json { "@context": "https://schema.org", "@type": "Product", "name": "Aero 42 trail running shoe", "image": "https://example.com/images/aero-42.jpg", "offers": { "@type": "Offer", "price": "89.90", "priceCurrency": "EUR" } } ``` ### Wrong date format Dates use the ISO 8601 format described for [Date on schema.org](https://schema.org/Date), such as `2026-10-10`. A date like `10/10/2026` is flagged. ```json { "@context": "https://schema.org", "@type": "Article", "headline": "How to choose trail running shoes", "datePublished": "10/10/2026", "author": { "@type": "Person", "name": "Alex Example" } } ``` ```json { "@context": "https://schema.org", "@type": "Article", "headline": "How to choose trail running shoes", "datePublished": "2026-10-10", "author": { "@type": "Person", "name": "Alex Example" } } ``` ### Missing @context Without `"@context": "https://schema.org"` the keys are not schema.org properties, so the validator reports an error at `@context`. The [JSON-LD specification](https://www.w3.org/TR/json-ld11/) defines how a context maps keys to vocabulary terms. ```json { "@type": "Product", "name": "Aero 42 trail running shoe", "offers": { "@type": "Offer", "price": "89.90", "priceCurrency": "EUR" } } ``` ```json { "@context": "https://schema.org", "@type": "Product", "name": "Aero 42 trail running shoe", "offers": { "@type": "Offer", "price": "89.90", "priceCurrency": "EUR" } } ``` ### Trailing comma JSON allows neither trailing commas nor comments ([RFC 8259](https://www.rfc-editor.org/rfc/rfc8259.html)). One stray comma makes the whole block unreadable, and the validator reports a syntax error. Serpel’s [site audit](https://serpel.app/features/site-audit) flags JSON-LD with syntax errors on every page it crawls. ```json { "@context": "https://schema.org", "@type": "Product", "name": "Aero 42 trail running shoe", "sku": "AERO-42", } ``` ```json { "@context": "https://schema.org", "@type": "Product", "name": "Aero 42 trail running shoe", "sku": "AERO-42" } ``` ## Does valid markup guarantee a rich result? No. Google says valid markup [does not guarantee a rich result](https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data), and a page is eligible only if the required properties are present. The markup must also match what visitors see. Google’s [structured data policies](https://developers.google.com/search/docs/appearance/structured-data/sd-policies) ask you not to mark up content that is hidden from readers, to make the markup a true representation of the page and not to block pages with structured data from Googlebot. ## What does this schema validator not do? - **It does not run JavaScript.** It reads static HTML, so markup that a script adds after the page loads is not seen. Paste the rendered code into the “Paste code” tab instead, and read the [JavaScript SEO guide](https://serpel.app/blog/javascript-seo) for the background. - **It reads a limited page.** It follows at most 5 redirects and reads the first 3 MB of a page. - **It cannot tell whether Google shows a rich result.** Eligible here means the markup passes Serpel’s rules, not that Google will display it. - **It checks only the types Serpel has rules for.** Every other schema.org type gets a syntax check only. - **It cannot compare the markup with your page.** You have to make sure that it matches the visible content. If you build with Next.js, the [Next.js SEO guide](https://serpel.app/blog/nextjs-seo) shows how to add JSON-LD to a page safely. ## Frequently asked questions ### Which is better: this schema checker, the Rich Results Test or the Schema Markup Validator? They answer different questions, so none of them is simply the best. Use Google’s [Rich Results Test](https://search.google.com/test/rich-results) to see whether a page is eligible for Google’s rich results, and the [Schema Markup Validator](https://validator.schema.org/) to check schema.org markup in general. Use this tool for fast feedback with exact property paths, then confirm important pages in the Rich Results Test. ### What happened to Google’s Structured Data Testing Tool? Google retired it in favour of the Rich Results Test and the Schema Markup Validator. The first checks eligibility for Google’s rich results, and the second checks schema.org markup in general. Use one or both instead, and see [Google’s documentation](https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data) for how structured data is used in Search. ### Can I validate JSON-LD that is not published yet? Yes. Open the “Paste code” tab and paste raw JSON-LD or an HTML snippet with JSON-LD, Microdata or RDFa. The code is checked on our server and is not stored. ### Why do I see warnings when there are no errors? Warnings point to recommended properties or optional values that look wrong. A type with no errors still counts as eligible in this tool, so you decide whether each recommended property is worth adding. Treat errors as blockers and warnings as improvements. ### Why does the tool say my FAQ or how-to markup is not eligible? Google has discontinued these rich results, so valid markup of these types no longer produces one. The markup is still valid schema.org. Google’s [structured data gallery](https://developers.google.com/search/docs/appearance/structured-data/search-gallery) lists the features it supports today. ### Does the schema validator run JavaScript? No. It fetches the static HTML, so markup that a script injects after the page loads is not visible to it. Copy the rendered markup from your browser’s developer tools and paste it into the “Paste code” tab to test it. --- # Robots.txt checker for AI crawlers URL: https://serpel.app/tools/robots-txt-checker Updated: 2026-10-10 A robots.txt checker fetches a domain’s robots.txt and shows what the file allows for each crawler. This free tool tests it against all 13 AI crawlers Serpel knows, shows which of them can reach your homepage, lists your sitemaps and checks for llms.txt and llms-full.txt. ## What does a robots.txt checker test? A robots.txt checker fetches the `robots.txt` file of a domain and applies its rules the way a crawler would. This one tests all 13 AI crawlers Serpel knows and shows which of them can reach your homepage. It also lists the sitemaps declared in the file, which the [sitemap checker](https://serpel.app/tools/sitemap-checker) can validate, and checks whether `llms.txt` and `llms-full.txt` exist at the root of your domain. If the file is missing, the [llms.txt generator](https://serpel.app/tools/llms-txt-generator) builds one. Use the tool as a robots.txt tester after every change, or to check robots.txt before you launch a site. A crawler counts as allowed when the group that applies to it does not disallow the homepage. Check that every result matches your intent. ## How does robots.txt work? The Robots Exclusion Protocol is standardised in [RFC 9309](https://www.rfc-editor.org/rfc/rfc9309.html). Google describes its own reading in the [robots.txt specification](https://developers.google.com/crawling/docs/robots-txt/robots-txt-spec). These are the rules that matter most: - **Location.** The file must be served at `/robots.txt` in the top-level path of a host. Its rules apply only to that protocol, host and port, so every subdomain needs its own file. - **Groups.** A group starts with one or more `User-agent` lines, followed by `Allow` and `Disallow` rules. A crawler matches its product token case-insensitively and obeys only the matching group. If none matches, it falls back to the `*` group. Specific groups and the `*` group are never merged. - **Longest match wins.** The most specific rule, meaning the one with the longest matching path, applies. If an `Allow` and a `Disallow` rule are equally specific, RFC 9309 says the `Allow` rule should win. Google applies the least restrictive rule in a conflict. - **Status codes.** A `2xx` response is parsed. A `4xx` response means there is no file, so crawlers may fetch anything. A `5xx` response or a timeout means the file is unreachable, and RFC 9309 tells crawlers to assume that everything is disallowed. Google pauses crawling, keeps retrying and falls back to its last cached copy for up to 30 days. - **Size and caching.** Crawlers must parse at least 500 KiB. Google generally caches the file for up to 24 hours, so a change can take a day to reach every bot. - **Not access control.** RFC 9309 states that these rules are not a form of access authorisation. Private content needs authentication, not a `Disallow` line. ## Which AI crawlers should you know about? AI companies run crawlers for three different purposes, and you can treat each purpose separately: - **Training** crawlers collect content that may be used to train foundation models. - **AI search** crawlers index pages so that an assistant can cite and link them in its answers. - **User fetch** agents visit a single page because a person asked an assistant about it. **The AI crawlers Serpel checks** | User agent | Vendor | Purpose | | --- | --- | --- | | `OAI-SearchBot` | OpenAI | AI search | | `GPTBot` | OpenAI | Training | | `ChatGPT-User` | OpenAI | User fetch | | `Claude-SearchBot` | Anthropic | AI search | | `ClaudeBot` | Anthropic | Training | | `Claude-User` | Anthropic | User fetch | | `PerplexityBot` | Perplexity | AI search | | `Perplexity-User` | Perplexity | User fetch | | `Google-Extended` | Google | Training | | `Applebot-Extended` | Apple | Training | | `CCBot` | Common Crawl | Training | | `Meta-ExternalAgent` | Meta | Training | | `MistralAI-Index` | Mistral | AI search | Three details are easy to miss: - **Control tokens.** `Google-Extended` and `Applebot-Extended` do not crawl on their own. Google says [Google-Extended](https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers) has no separate user agent and does not affect inclusion in Google Search. Apple says [Applebot-Extended](https://support.apple.com/en-us/119829) only decides whether content already crawled by Applebot may be used for training. - **User fetches.** [OpenAI](https://developers.openai.com/api/docs/bots) says `ChatGPT-User` may not follow robots.txt, and [Perplexity](https://docs.perplexity.ai/docs/resources/perplexity-crawlers) says `Perplexity-User` generally ignores it. [Anthropic](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) says all of its bots honour robots.txt. - **Opt-outs look forward.** Blocking a training crawler is a request about future collection. Anthropic, for example, describes it as a signal that future content should be excluded. ## How do you allow AI search but block training? Give each training crawler a `Disallow: /` rule and leave the AI search crawlers alone. They then follow your `*` group, which allows everything except the paths you list. This example blocks the 6 training crawlers Serpel knows: ```text User-agent: GPTBot User-agent: ClaudeBot User-agent: Google-Extended User-agent: Applebot-Extended User-agent: CCBot User-agent: Meta-ExternalAgent Disallow: / User-agent: * Disallow: /account/ Sitemap: https://example.com/sitemap.xml ``` `OAI-SearchBot`, `Claude-SearchBot`, `PerplexityBot` and `MistralAI-Index` need no group of their own. If you add one, repeat the rules from your `*` group inside it, because a crawler obeys only one group. Run the checker again to confirm the result. Which setup is right depends on your goals, and our [guide to AI crawlers](https://serpel.app/blog/ai-crawlers) explains the trade-offs. ## What are the most common robots.txt mistakes? - **A leftover blanket block.** A staging file with `User-agent: *` and `Disallow: /` that reaches production blocks every crawler, search engines included. - **A bot group that replaces the wildcard group.** Adding a group for one bot with a single rule means that bot no longer sees the rules from your `*` group. - **Hiding pages from search with Disallow.** robots.txt controls crawling, not indexing. As Google explains, a blocked URL can still be indexed and shown without a snippet. - **Errors on the file itself.** A `5xx` response or a timeout on `/robots.txt` can stop crawling altogether. A bot-protection rule that answers crawlers with a challenge page blocks them whatever the file says. - **Typos in tokens.** `GPT-Bot` is not `GPTBot`, and a misspelt token matches nothing. ## What does this robots.txt checker not do? - It tests the homepage path `/` only. A rule such as `Disallow: /blog/` will not show up as a block. - It fetches only `robots.txt`, `llms.txt` and `llms-full.txt`. It does not crawl pages, render JavaScript or read `noindex` tags and `X-Robots-Tag` headers. - It does not request your pages as those crawlers. A firewall or CDN rule can still block them, and only your server logs show that. - It does not lint the file line by line, so it is not a strict robots.txt validator. For the whole site, Serpel’s [AI visibility features](https://serpel.app/features/ai-visibility) check crawler access and llms.txt on every crawl and show whether ChatGPT and Google AI Overviews cite you. ## Frequently asked questions ### How do I check whether GPTBot can crawl my site? Enter your domain in the checker and look for GPTBot in the results. It counts as allowed when the robots.txt group that applies to GPTBot does not disallow your homepage. The checker only reads robots.txt, so a firewall or bot-protection rule can still block the crawler. ### Does blocking GPTBot remove my site from ChatGPT? No. GPTBot is OpenAI’s training crawler, while ChatGPT search uses OAI-SearchBot. [OpenAI says](https://developers.openai.com/api/docs/bots) that sites opted out of OAI-SearchBot will not appear in ChatGPT search answers. Block GPTBot to opt out of training and leave OAI-SearchBot allowed to stay visible. ### What happens if my robots.txt returns a 404 or a server error? A 4xx response such as 404 means there is no file, so crawlers may fetch everything. A 5xx response or a timeout counts as unreachable, and [RFC 9309](https://www.rfc-editor.org/rfc/rfc9309.html) tells crawlers to assume that everything is disallowed. Google pauses crawling and falls back to its last cached copy for up to 30 days. ### Do AI crawlers have to obey robots.txt? robots.txt is a convention that crawlers choose to follow, and OpenAI, Anthropic and Perplexity document that their crawlers do. The exceptions are fetches a user triggers: OpenAI says ChatGPT-User may not follow robots.txt, and Perplexity says Perplexity-User generally ignores it. For anything private, use authentication instead. ### Does the checker test llms.txt too? Yes. It reports whether llms.txt and llms-full.txt exist at the root of the domain. If you need the file, the [llms.txt generator](https://serpel.app/tools/llms-txt-generator) creates it. --- # XML sitemap checker and validator URL: https://serpel.app/tools/sitemap-checker Updated: 2026-10-10 A sitemap checker fetches your XML sitemap and tests it against the sitemap protocol. This free tool reads your robots.txt, validates the file and its child sitemaps, and requests up to 20 listed URLs to find 404 errors, redirects and noindex pages. It needs no account. ## What does a sitemap checker test? An XML sitemap is a file that lists the URLs you want search engines to crawl, optionally with the date each page last changed. A sitemap checker fetches that file and tests it against the [sitemap protocol](https://www.sitemaps.org/protocol.html) and [Google’s sitemap guidelines](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap). This one works as an XML sitemap validator and as a live check of the URLs inside. For every file it reads, the checker tests: - **Delivery.** The HTTP status, the content type, and whether the file is compressed, either as a `.xml.gz` file or in transfer. - **Structure.** Well-formed XML with the line number of the first syntax error, the `<urlset>` or `<sitemapindex>` root element, the sitemap namespace and UTF-8 encoding. - **Limits.** No more than 50,000 URLs and 50 MB uncompressed per file. - **URLs.** Every `<loc>` must be absolute, escaped, shorter than 2,048 characters and on the same host and protocol as the sitemap. Duplicates are flagged too. - **Values.** Malformed or future `<lastmod>` dates, and `<changefreq>` or `<priority>` values outside the allowed range. It then requests up to 20 of the listed URLs, spread evenly across the sitemap, and reports any that return 404 or another error, redirect or send a `noindex` header. A sitemap should list the URLs you want to appear in search, preferably the canonical version of each, so these are the entries that waste crawl effort. ## How do you check a sitemap with this validator? 1. **Enter a domain or a sitemap URL** Enter a domain such as `example.com` and the checker reads robots.txt for Sitemap lines, then tries `/sitemap.xml` and `/sitemap_index.xml`. Or enter the full URL of one sitemap, including a `.xml.gz` file. 2. **Run the check** The checker reads up to 3 sitemaps from robots.txt and up to 10 child sitemaps of an index. Large sitemaps can take up to half a minute. 3. **Read the verdict and the file table** Check the status, size and entry count of each file, then work through the findings, errors first. 4. **Fix the source and check again** Change the file in your CMS, framework or sitemap generator, then check again. Results are cached for 5 minutes, so wait a little before you recheck. ## What does a correct XML sitemap look like? A sitemap is a UTF-8 XML file. The root element is `<urlset>` with the sitemap namespace, and every `<url>` needs a `<loc>`. Escape the characters `&`, `'`, `"`, `>` and `<` as entities, so `&` becomes `&` inside a URL. ```xml <?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <url> <loc>https://example.com/</loc> <lastmod>2026-10-01</lastmod> </url> <url> <loc>https://example.com/pricing</loc> <lastmod>2026-09-18</lastmod> </url> <url> <loc>https://example.com/blog?topic=seo&page=2</loc> <lastmod>2026-09-30T08:15:00+00:00</lastmod> </url> </urlset> ``` Large sites split their URLs into several files and list them in a sitemap index. The protocol allows up to 50,000 sitemaps in one index, and [Google](https://developers.google.com/search/docs/crawling-indexing/sitemaps/large-sitemaps) accepts up to 500 index files per Search Console property. ```xml <?xml version="1.0" encoding="UTF-8"?> <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap> <loc>https://example.com/sitemap-pages.xml</loc> <lastmod>2026-10-01</lastmod> </sitemap> <sitemap> <loc>https://example.com/sitemap-blog.xml.gz</loc> <lastmod>2026-09-30</lastmod> </sitemap> </sitemapindex> ``` **Limits and valid values in an XML sitemap** | Item | Rule | | --- | --- | | URLs per file | At most 50,000 | | File size | At most 50 MB (52,428,800 bytes) uncompressed. Gzip is allowed, but the limit applies after decompression. | | Sitemaps per index | At most 50,000 | | `<loc>` | A fully qualified URL under 2,048 characters, on the same protocol and host as the sitemap | | `<lastmod>` | A W3C date such as `2026-10-01`, or with a time such as `2026-09-30T08:15:00+00:00` | | `<changefreq>` | always, hourly, daily, weekly, monthly, yearly or never | | `<priority>` | 0.0 to 1.0, with a default of 0.5 | ## What are the most common sitemap errors? - **An unescaped ampersand.** A URL such as `?a=1&b=2` makes the file invalid XML. Write `&`. The checker shows the line of the first syntax error. - **A wrong or missing namespace.** Start the file with `<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">`. - **Relative URLs.** `/pricing` is not valid in `<loc>`. Use `https://example.com/pricing`. - **http, https and www mismatches.** The [protocol](https://www.sitemaps.org/protocol.html) and [Google](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap) expect the URLs to use the same protocol and host as the sitemap. List the canonical version of each page. - **URLs that redirect, return 404 or are noindex.** List the final URL, drop deleted pages and keep `noindex` pages out. A missing `/sitemap.xml` that answers 200 with a page is a soft 404, which our [soft 404 guide](https://serpel.app/blog/soft-404) explains. - **A lastmod that is always today.** Google uses `<lastmod>` only if it is consistently and verifiably accurate. In Next.js, set real dates in `sitemap.ts`, as our [Next.js SEO guide](https://serpel.app/blog/nextjs-seo) shows. - **A sitemap nobody can fetch.** A catch-all route that serves HTML, a login or a firewall rule that answers bots with 403 hides the file. Check the status and content type in the table. - **Files over the limits.** Split the sitemap into several files and list them in a sitemap index. ## How do you submit a sitemap? Add a Sitemap line with the full URL to your robots.txt, once per file, and submit the URL in the Sitemaps report in Google Search Console. The [robots.txt checker](https://serpel.app/tools/robots-txt-checker) lists the Sitemap lines it finds. [Google](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap) ignores `<priority>` and `<changefreq>`, so you can leave them out. ```text User-agent: * Allow: / Sitemap: https://example.com/sitemap.xml Sitemap: https://example.com/sitemap-blog.xml ``` ## What does this sitemap checker not do? - It reads XML sitemaps only, not text or RSS sitemaps, and it ignores image, video and news extensions. - It checks at most 10 child sitemaps and 20 URLs per request, and it does not descend into nested index files. - It requests each sampled URL once and does not render JavaScript. It reads the HTTP status, the redirect target and the `X-Robots-Tag` header, but not a `<meta name="robots">` tag in the HTML. - It does not show whether Google has processed your sitemap. The Sitemaps report in Search Console does. - Its requests come from Serpel’s servers with a SerpelBot user agent. A firewall may treat them differently from Googlebot. For the whole site, Serpel’s [site audit](https://serpel.app/features/site-audit) includes indexing and crawling checks, among them robots.txt and sitemap problems. ## Frequently asked questions ### How do I check whether my sitemap is valid? Enter your domain or the full sitemap URL in the checker. It validates the XML, the namespace, the limits and every `<loc>`, then requests a sample of the listed URLs. To see how Google processed the file, open the Sitemaps report in Search Console. ### How many URLs can a sitemap contain? At most 50,000 URLs and 50 MB uncompressed per file, according to the [sitemap protocol](https://www.sitemaps.org/protocol.html). Gzip shrinks the transfer, but the limits still apply after decompression. Larger sites split the URLs into several files and list them in a sitemap index. ### Does Google use changefreq and priority? No. [Google says](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap) it ignores `<priority>` and `<changefreq>`, and that it uses `<lastmod>` only if the value is consistently and verifiably accurate. Leave out the first two and set the last to the real modification date. ### Why does the checker report redirects, 404s or noindex in my sitemap? A sitemap should list the URLs you want to appear in search, preferably canonical, as [Google’s guidelines](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap) put it. A URL that redirects, returns 404 or sends a noindex header works against that. Replace it with the final URL or remove it. ### Where should I put my sitemap? Put it in the root of your host, for example `/sitemap.xml`. The protocol limits a sitemap to URLs at or below its own directory, unless you submit it through Search Console. Reference it from robots.txt with a Sitemap line as well. ### Can the checker read a .xml.gz sitemap? Yes. Enter the URL of the `.xml.gz` file and the checker decompresses it, up to 50 MB uncompressed. It also reports whether a file is compressed, either as gzip or in transfer. --- # llms.txt generator for your website URL: https://serpel.app/tools/llms-txt-generator Updated: 2026-10-10 An llms.txt generator builds the Markdown file that tells AI agents which pages on your site matter. Enter your site name, a short summary and your key pages, and this free tool creates a file in the llmstxt.org format that you copy to your domain root. It needs no account. ## What is llms.txt? `llms.txt` is a Markdown file at the root of a website that gives language models a short, curated map of the site. Jeremy Howard proposed it in September 2024 at [llmstxt.org](https://llmstxt.org/). It is a community proposal, not an IETF or W3C standard. Think of it as a reading list for agents. A web page is full of navigation, scripts and layout. The file points to the pages that matter and says in one line what each of them contains. ## What does an llms.txt file look like? The proposal defines a simple structure, in this order: 1. An `H1` with the name of the site or project. It is the only required part. 2. A blockquote with a short summary. 3. Optional paragraphs or lists with more detail, but no headings. 4. `H2` sections that hold lists of links. Each item is a Markdown link with an optional note after a colon. 5. An optional `H2` section called `Optional`. Agents can skip its links when they need a shorter context. ```markdown # Example Co > Example Co makes invoicing software for freelancers. This file lists the pages that explain the product, the API and the pricing. Prices are in euros and include VAT. The API is documented in the Docs section. ## Docs - [Getting started](https://example.com/docs/start.md): Install the SDK and send your first invoice - [API reference](https://example.com/docs/api.md): Every endpoint with request and response examples ## Product - [Pricing](https://example.com/pricing.md): Plans, limits and what each plan includes ## Optional - [Changelog](https://example.com/changelog.md): Release notes for every version ``` Write the summary and the notes as plain facts. An agent that reads “Plans, limits and what each plan includes” knows what it will find, while “Our amazing plans” tells it nothing. Prefer links to clean Markdown or simple HTML pages over pages that need JavaScript to show their content. ## How do you create an llms.txt file with the llms.txt generator? 1. **Name your site** Enter the name of your site or product and a one-sentence summary. Write the summary the way you would describe the site to a new colleague. 2. **Add your key pages** Group the pages that matter into sections such as Docs, Product and Pricing. Give every link a short, factual note. 3. **Move the rest to Optional** Put secondary pages, such as the changelog or older articles, in the Optional section so that agents can skip them. 4. **Copy the file** Copy the generated Markdown and save it as `llms.txt` in UTF-8. ## Where do you host llms.txt, and what is llms-full.txt? Serve the file at `/llms.txt` in the root of your domain, for example `https://example.com/llms.txt`. The proposal also allows a file in a subpath such as `/docs/llms.txt` for the pages under it. In Next.js, put the file in the `public` folder. On other stacks, upload it to the web root. Use absolute URLs, publish one file per host and make sure it returns a `200` response rather than a login page. `llms-full.txt` is a widely used companion convention that is not part of the proposal on llmstxt.org. It is one Markdown file with the full text of your key pages, so an agent can read everything in a single request. It suits documentation sites, while a large shop should skip it. Serpel publishes both files for its own site at [llms.txt](https://serpel.app/llms.txt) and [llms-full.txt](https://serpel.app/llms-full.txt). ## Does llms.txt work? The honest status No major AI provider has publicly confirmed that its production systems read llms.txt. In June 2025 Google’s John Mueller wrote that [no AI system currently uses llms.txt](https://www.seroundtable.com/google-ai-llms-txt-39607.html) and pointed to server logs as evidence. Coding agents and some developer tools do read the file when you point them at a documentation site, which is where it helps most. Treat it as cheap and low-risk, not as a ranking factor. It takes minutes to publish and does no harm, and it does not replace crawlable HTML, good titles or an open robots.txt. > **llms.txt blocks nothing:** To control which crawlers may fetch your pages, use robots.txt. The [robots.txt checker](https://serpel.app/tools/robots-txt-checker) shows how every AI crawler is treated. ## llms.txt checklist - The file starts with one `H1` that names the site. - A one- or two-sentence summary follows in a blockquote. - Every link is an absolute URL that returns `200`. - Every link has a short, factual note instead of a slogan. - Only pages you want agents to use are listed, and secondary pages sit under `Optional`. - The file is served as UTF-8 text at `/llms.txt`. - Your `robots.txt` does not block the file or the listed pages for the crawlers you care about. - You update the file when pages are added, renamed or removed. Looking for finished files to learn from? See our collection of [llms.txt examples](https://serpel.app/blog/llms-txt-examples). ## Frequently asked questions ### What is an llms.txt file? An llms.txt file is a Markdown file at /llms.txt that lists the most useful pages of a site, each with a one-line note, for language models and agents. It was proposed at [llmstxt.org](https://llmstxt.org/) in 2024 and is not an official web standard. ### Where do I put llms.txt? Put it in the root of your domain so that it is reachable at https://yourdomain.com/llms.txt. In Next.js, save it in the public folder. Open the address in a browser afterwards to confirm that it returns the text and not an error page. ### Does llms.txt improve rankings or AI citations? There is no confirmed effect. Google’s John Mueller has said that no AI system uses it, and no major AI provider has confirmed that it reads the file. It still costs almost nothing, and coding agents and some developer tools do use it. ### What is the difference between llms.txt and robots.txt? robots.txt tells crawlers which URLs they may fetch. llms.txt suggests which pages are worth reading and blocks nothing. Use the [robots.txt checker](https://serpel.app/tools/robots-txt-checker) to see how AI crawlers are treated by your robots.txt. ### Do I need llms-full.txt as well? No, it is optional. It bundles the full text of your key pages into one Markdown file, which helps agents that read documentation in a single request. For most sites, a well-kept llms.txt is enough. --- # Serpel compared. Facts, sources and dates. URL: https://serpel.app/alternatives These pages compare Serpel with well-known SEO and AI visibility tools. Every comparison cites public documentation and shows the date we last checked it, and each one says when the other tool is the better choice. - [Semrush alternative](https://serpel.app/alternatives/semrush): Semrush alternatives for developers: compare Serpel and Semrush on pricing, rank tracking, AI visibility, API and MCP, with sources and dates. - [Ahrefs alternative](https://serpel.app/alternatives/ahrefs): An Ahrefs alternative for developers: compare Serpel and Ahrefs on pricing, rank tracking, AI visibility, API and MCP, with sources and dates. - [AccuRanker alternative](https://serpel.app/alternatives/accuranker): An AccuRanker alternative for developers: compare Serpel and AccuRanker on pricing, rank tracking, AI visibility, API and MCP, with sources and dates. - [Peec AI alternative](https://serpel.app/alternatives/peec-ai): A Peec AI alternative for developers: compare Serpel and Peec AI on pricing, AI platforms, prompts, API and MCP, with sources and dates. --- # Semrush alternatives: Serpel for developers URL: https://serpel.app/alternatives/semrush Updated: 2026-10-10 Serpel is one of the Semrush alternatives for developers and small teams who want rank tracking, site audits and AI visibility checks without a subscription, from a CLI, an API or a coding agent. Semrush is the better fit if you need backlink analysis, a very large keyword database or a full marketing suite for a team. We checked every Semrush fact on 10 Oct 2026 against its own pages. ## Choose Serpel if - You want to pay for what you use: no subscription, 100 start credits and top-ups from €5. - You work in a terminal, in CI or with a coding agent, through the CLI, the REST API and an MCP server. - You build with Next.js and want recommendations tied to your routes through a codebase scan. - You track up to 500 keywords per project and want ChatGPT and Google AI Overview checks next to Google rankings. ## Choose Semrush if - You need backlink analysis: Semrush runs its own backlink index, over 43 trillion backlinks by its own count. Serpel has none. - You need a very large keyword database: Semrush lists more than 28 billion keywords in 142 regional databases. Serpel offers keyword research per search. - You need more AI platforms or sentiment: Semrush lists ChatGPT, Google AI Mode and Gemini and monitors AI sentiment. Serpel checks ChatGPT and Google AI Overviews. - You want content, advertising, social or PR tools in one account. Serpel covers SEO and AI visibility only and is in public beta. ## Which Semrush alternatives suit developers? Semrush is an all-in-one marketing platform. Its plans combine SEO and AI search tools, and its [pricing page](https://www.semrush.com/pricing/seo-ai-search/) also lists toolkits for traffic and market data, local, content, social, advertising and AI PR. Serpel is narrower and built for developers. It tracks Google rankings on desktop and mobile, audits sites with JavaScript rendering and Core Web Vitals, reads Search Console and Bing data, and checks whether ChatGPT and Google AI Overviews cite you. It runs in the dashboard, the CLI, the REST API and an [MCP server](https://serpel.app/developers/mcp). Serpel is in public beta. ## How do Serpel and Semrush differ? - **Billing:** Serpel charges credits per action. Semrush sells monthly or annual subscriptions. - **Interfaces:** both offer an API and an MCP server. Serpel adds a CLI with JSON output for CI. - **Scope:** Semrush adds backlinks, keyword research at scale, content, advertising and social tools. - **Codebase:** Serpel can scan a Next.js project and tie recommendations to its routes. ## How does Semrush pricing compare with Serpel? A Serpel credit is worth €0.01. Checking 100 keywords weekly in the top 50 costs about 858 credits (€8.58) a month. Checking 500 keywords daily costs about 30,000 credits (€300.00). Semrush’s entry plan tracks 500 keywords daily for €112.52 per month billed annually, with keyword research, Site Audit and AI search tracking included. Serpel is cheaper for occasional checks and more expensive for heavy daily tracking. All prices are on the [pricing page](https://serpel.app/pricing). ## What does Semrush do better than Serpel? Semrush runs its own backlink index and a keyword database of more than 28 billion keywords, according to [its own figures](https://www.semrush.com/features/backlink-analysis/). It also tracks featured snippets and local packs and follows up to 20 competitors. Serpel has no backlink data and follows up to 10 competitors per project. > **How we checked:** Semrush facts come from its public pages, read on 10 Oct 2026 and linked in the sources. Prices were shown in euros to a visitor in Germany and change over time. Semrush is a trademark of its owner, used here only to describe its product. Serpel is not affiliated with Semrush. Spotted something out of date? Tell us through the [About page](https://serpel.app/about). ## Serpel and Semrush compared ### Rank tracking | Feature | Serpel | Semrush | | --- | --- | --- | | Devices and locations | Desktop and mobile, nationwide, city or postal code, to position 100 | Daily updates by device and location, down to ZIP code | | Update frequency | Daily on Pro, otherwise every 3 days or weekly, plus manual checks | Daily ranking updates | | Keywords tracked | 50, 200, 500 per project (Free, Starter, Pro) | 500, 1,500 or 5,000 tracked daily, by plan | | Competitor tracking | Up to 10 per project | Up to 20 competitors | | SERP features | AI Overview presence and citation only | Featured snippets, video carousels, AI Overviews, local packs | ### AI visibility | Feature | Serpel | Semrush | | --- | --- | --- | | AI platforms | ChatGPT with web search and Google AI Overviews | ChatGPT, Google AI Mode, Gemini and AI Overviews | | Tracked prompts | 10, 25, 50 per project (Free, Starter, Pro) | 50, 100 or 200 daily on Starter, Pro+ and Advanced | | Citations and sources | Cited or mentioned, cited URL, sources, answer excerpt | Mentions, share of voice, outlets AI platforms cite | | AI sentiment | Not offered | Monitors how AI platforms describe your brand | | AI crawler access | robots.txt check for 13 AI crawlers, plus llms.txt | Site Audit checks access for 8 named bots | ### Site audit | Feature | Serpel | Semrush | | --- | --- | --- | | Audit checks | 94 checks in 18 categories, ranked by severity | Issues ranked by severity, with fix guidance | | JavaScript rendering | Automatic for empty pages, or on every page | Optional setting, tied to higher-tier plans in its knowledge base | | Core Web Vitals | Lab and field values for key pages, mobile and desktop | Lighthouse-based report for 10 pages | | Handing issues to developers | Tasks, CSV and JSON export, CLI exit codes | Trello integration and CSV export | ### Developer access | Feature | Serpel | Semrush | | --- | --- | --- | | CLI | Every feature, with JSON output | Not listed in its developer documentation | | REST API | Same API as the dashboard, 18 token scopes | API measured in API units | | MCP server | serpel.app/mcp, paid tools run within a budget you approve | Semrush MCP for ChatGPT, Claude, Cursor, VS Code, Gemini, Perplexity | | Codebase scan | Scans a Next.js project, never reads source code | Not listed on its plan or feature pages | ### Data depth and suite | Feature | Serpel | Semrush | | --- | --- | --- | | Backlink data | No backlink index | 43T+ backlinks, about 10B pages crawled daily, by its own count | | Keyword database | Research per search, priced in credits | 28B+ keywords in 142 regional databases, by its own count | | Domain and market research | Compares your keywords with up to 10 competitors | 808M domain profiles, by its own count | | Content, ads, social and PR tools | Not offered | Content, social, advertising and AI PR toolkits | ## Pricing - **Serpel:** Pay per credit, with optional monthly plans. Free: no subscription, 100 credits to start. Starter: €19 per month incl. VAT, 2,000 credits every month. Pro: €49 per month incl. VAT, 6,000 credits every month. One credit is worth €0.01. A scheduled check costs 2 credits per keyword in the top 50. Top-ups start at €5, and bought credits never expire. - **Semrush:** Monthly or annual subscription. Prices are per month when billed annually, with the monthly-billing price in brackets. SEO: €112.52 (€134 monthly), 5 websites, 500 keywords tracked daily. Starter: €158.40 (€190 monthly), adds 50 AI prompts daily and MCP access. Pro+: €238 (€286 monthly), 15 websites, 1,500 keywords, 100 AI prompts. Advanced: €436.99 (€526 monthly), 40 websites, 5,000 keywords, 200 AI prompts, API data integration. A free plan with one demo project exists, and the page offers paid plans free for seven days. Prices as shown to a visitor in Germany on 10 Oct 2026. ## How to switch from Semrush 1. **Export your keywords from Semrush** In Position Tracking, open the Overview tab and export the keyword table as CSV or Excel, as the [knowledge base](https://www.semrush.com/kb/549-position-tracking-overview-manual) describes. Save the keyword column as `keywords.txt`, one keyword per line. Serpel cannot import ranking history, so keep the export as your archive. 2. **Sign in and create a project** Install the CLI as described on the [CLI page](https://serpel.app/developers/cli), sign in and create a project for your domain. Copy the project ID from the output. ```bash serpel auth login serpel projects create --domain example.com --country US --language en export PROJECT=<projectId> ``` 3. **Import keywords with their metrics** Create a keyword list and import the CSV. Serpel maps header columns such as keyword, search volume, cpc and kd, and accepts files up to 2 MB. ```bash serpel keywords list-create --project $PROJECT --name "Imported from Semrush" serpel keywords import <listId> --file keywords.csv ``` 4. **Start tracking and run the first check** Add the keywords to rank tracking, up to 200 per call and within your plan’s limit. A check run right now books 5 credits per keyword in the top 50, so estimate it first. ```bash args=() while IFS= read -r keyword; do args+=(--keyword "$keyword"); done < keywords.txt serpel billing estimate --operation rankCheck --keywords "$(wc -l < keywords.txt)" serpel rankings add --project $PROJECT "${args[@]}" serpel rankings run --project $PROJECT --wait ``` 5. **Re-create competitors and AI questions** Add the competitor domains you follow in Semrush and the questions you want to check in ChatGPT and Google AI Overviews. ```bash serpel competitors add --project $PROJECT --domain competitor.com serpel ai add --project $PROJECT --prompt "which project management tool suits small agencies" ``` ## Frequently asked questions ### Is there a Semrush alternative without a subscription? Yes. Serpel needs no subscription: you start with 100 credits, top up from €5, and bought credits never expire. Semrush’s entry plan was €112.52 per month billed annually on 10 Oct 2026. Serpel does not replace Semrush’s backlink data or keyword database. ### Can Serpel replace Semrush? For rank tracking, site audits, search data and AI visibility on your own sites, often yes. For backlinks, large-scale keyword research or advertising, content and social tools, no. You can also run both. ### Does Semrush have an API or an MCP server? Yes. Semrush offers an API measured in API units and an MCP server for ChatGPT, Claude, Cursor, VS Code, Gemini and Perplexity. Serpel offers a REST API and an MCP server too, and adds a CLI. ### Does Serpel track the same AI platforms as Semrush? No. Serpel checks ChatGPT with web search and Google AI Overviews. Semrush lists ChatGPT, Google AI Mode, Gemini and AI Overviews and monitors AI sentiment, which Serpel does not. ## Sources - [Semrush: plans and pricing](https://www.semrush.com/pricing/seo-ai-search/), accessed 2026-10-10 - [Semrush: backlink analysis and data figures](https://www.semrush.com/features/backlink-analysis/), accessed 2026-10-10 - [Semrush: rank tracking](https://www.semrush.com/features/rank-tracking/), accessed 2026-10-10 - [Semrush: AI visibility](https://www.semrush.com/features/ai-visibility/), accessed 2026-10-10 - [Semrush: site audit](https://www.semrush.com/features/site-audit/), accessed 2026-10-10 - [Semrush: configuring Site Audit](https://www.semrush.com/kb/539-configuring-site-audit), accessed 2026-10-10 - [Semrush: Site Audit thematic reports](https://www.semrush.com/kb/959-site-audit-thematic-reports), accessed 2026-10-10 - [Semrush: Position Tracking overview](https://www.semrush.com/kb/549-position-tracking-overview-manual), accessed 2026-10-10 - [Semrush: MCP server](https://www.semrush.com/mcp/), accessed 2026-10-10 - [Semrush: API documentation](https://developer.semrush.com/api/v4/), accessed 2026-10-10 --- # The Ahrefs alternative for developers URL: https://serpel.app/alternatives/ahrefs Updated: 2026-10-10 Serpel is an Ahrefs alternative for developers who want Google rankings, site audits and AI visibility checks with usage-based pricing, from a CLI, an API or a coding agent. Ahrefs is the better fit if you rely on its backlink index, its research tools or a longer ranking history. We checked every Ahrefs fact on 10 Oct 2026 against its own pages. ## Choose Serpel if - You want usage-based pricing: Serpel needs no subscription, while the Ahrefs plans that list API and MCP access start at €119 per month. - You want a CLI with JSON output for scripts and CI, and a codebase scan for Next.js projects. - You track a small keyword set often: Serpel checks daily on Pro or on demand, while Ahrefs updates tracked keywords weekly by default. - You track a few prompts on ChatGPT and Google AI Overviews and prefer to pay per check, 2 credits per platform. ## Choose Ahrefs if - You need backlink data: Ahrefs lists 35 trillion backlink records and a live index refreshed every 15 to 30 minutes. Serpel has no backlink index. - You need research tools: Ahrefs lists a keyword index of 29 billion keywords in 217 locations, plus Site Explorer and Content Explorer. Serpel offers keyword research per search. - You want more AI platforms: Brand Radar covers AI Overviews, AI Mode, Perplexity, Copilot, Gemini and ChatGPT. Serpel checks ChatGPT and Google AI Overviews. - You track thousands of keywords or want more SERP detail: Ahrefs plans reach 5,000 tracked keywords and 19 SERP features. Serpel tracks up to 500 per project and is in public beta. ## Which Ahrefs alternative suits developers? Ahrefs is an SEO platform built on its own web crawler and index. Its plans combine Site Explorer, Keywords Explorer, Rank Tracker, Site Audit and Brand Radar, and its [pricing page](https://ahrefs.com/pricing) lists API and MCP access from the Lite plan. Serpel is narrower and built for developers. It tracks Google rankings on desktop and mobile, audits sites with JavaScript rendering and Core Web Vitals, reads Search Console and Bing data, and checks whether ChatGPT and Google AI Overviews cite you. It runs in the dashboard, the CLI, the REST API and an [MCP server](https://serpel.app/developers/mcp). Serpel is in public beta. ## How do Serpel and Ahrefs differ? - **Billing:** Serpel charges credits per action. Ahrefs sells monthly plans with fixed limits, and AI prompt packs as add-ons. - **Data:** Ahrefs runs a web-wide index of backlinks and keywords. Serpel does not, so it has no backlink data. - **Update rhythm:** the Ahrefs help centre says tracked keywords update weekly by default. Serpel schedules daily checks on Pro and runs manual checks at 5 credits per keyword in the top 50. - **Developer tools:** both offer an API and an MCP server. Serpel adds a CLI and a codebase scan. ## How does Ahrefs pricing compare with Serpel? Ahrefs shows Lite at €119 per month with 750 tracked keywords and Standard at €229 with 2,000, and those plans include research tools and the backlink index. Serpel bills only what you run. A small site with 100 keywords, 10 AI prompts on ChatGPT and Google AI Overviews and a 100-page crawl, all checked weekly, uses about 1,052 credits (€10.52) a month at €0.01 per credit. The Pro plan costs €49 per month incl. VAT with 6,000 credits, and heavy daily tracking needs more. See the [price list](https://serpel.app/pricing). ## What does Ahrefs do better than Serpel? According to [its data page](https://ahrefs.com/big-data), Ahrefs keeps 35 trillion backlink records and a keyword index of 29 billion keywords. Its Rank Tracker covers 190+ locations, 19 SERP features and 10 competitors, and Brand Radar follows six AI platforms. Serpel has no backlink data and checks 2 AI surfaces. > **How we checked:** Ahrefs facts come from its public pages, read on 10 Oct 2026 and linked in the sources. Prices were shown in euros to a visitor in Germany and change over time. Ahrefs is a trademark of its owner, used here only to describe its product. Serpel is not affiliated with Ahrefs. Spotted something out of date? Tell us through the [About page](https://serpel.app/about). ## Serpel and Ahrefs compared ### Rank tracking | Feature | Serpel | Ahrefs | | --- | --- | --- | | Devices and locations | Desktop and mobile, nationwide, city or postal code, to position 100 | Mobile and desktop, 190+ locations, down to ZIP code | | Update frequency | Daily on Pro, otherwise every 3 days or weekly, plus manual checks | Weekly by default on all plans | | Keywords tracked | 50, 200, 500 per project (Free, Starter, Pro) | 750, 2,000 or 5,000 on Lite, Standard and Advanced | | Competitor tracking | Up to 10 per project | Compare against 10 competitors | | SERP features | AI Overview presence and citation only | 19 SERP features, including AI Overviews and local packs | ### AI visibility | Feature | Serpel | Ahrefs | | --- | --- | --- | | AI platforms | ChatGPT with web search and Google AI Overviews | AI Overviews, AI Mode, Perplexity, Copilot, Gemini and ChatGPT in Brand Radar | | Tracked prompts | 10, 25, 50 per project (Free, Starter, Pro) | 5, 10 or 20 on Lite, Standard and Advanced, prompt packs from €46.7 per month | | Citations and sources | Cited or mentioned, cited URL, sources, answer excerpt | AI responses and cited pages for your prompts | | AI crawlers | robots.txt check for 13 AI crawlers, plus llms.txt | Bot Analytics shows AI crawler visits, free while in beta | ### Site audit | Feature | Serpel | Ahrefs | | --- | --- | --- | | Audit checks | 94 checks in 18 categories, ranked by severity | 170+ issues, split into errors, warnings and notices | | JavaScript rendering | Automatic for empty pages, or on every page | Executes JavaScript, can be enabled in crawl settings | | Core Web Vitals | Lab and field values for key pages, mobile and desktop | CrUX and Lighthouse metrics, with a Core Web Vitals issue group | | Handing issues to developers | Tasks, CSV and JSON export, CLI exit codes | Bulk export of all issues as ZIP or CSV | ### Developer access | Feature | Serpel | Ahrefs | | --- | --- | --- | | CLI | Every feature, with JSON output | Not listed in its developer documentation | | REST API | Same API as the dashboard, 18 token scopes | API access listed from Lite, uncapped on Enterprise | | MCP server | serpel.app/mcp, paid tools run within a budget you approve | Ahrefs SEO MCP from Lite, with guides for ChatGPT, Claude, Cursor, Copilot Studio | | Codebase scan | Scans a Next.js project, never reads source code | Not listed on its plan or product pages | ### Research and entry options | Feature | Serpel | Ahrefs | | --- | --- | --- | | Backlink data | No backlink index | 35T backlink records, live index refreshed every 15 to 30 minutes, by its own count | | Keyword index | Research per search, priced in credits | 29B keywords in 217 locations, by its own count | | Content and link prospecting | Not offered | Content Explorer from the Standard plan, Site Explorer | | Entry options | Free plan with 100 start credits | Ahrefs Free, and Starter at €27 per month | ## Pricing - **Serpel:** Pay per credit, with optional monthly plans. Free: no subscription, 100 credits to start. Starter: €19 per month incl. VAT, 2,000 credits every month. Pro: €49 per month incl. VAT, 6,000 credits every month. One credit is worth €0.01. A ChatGPT or Google AI Overview check costs 2 credits per question. Top-ups start at €5, and bought credits never expire. - **Ahrefs:** Monthly or annual subscription. Lite: €119 per month, 5 projects, 750 tracked keywords, 5 tracked AI prompts, API and MCP access. Standard: €229 per month, 20 projects, 2,000 tracked keywords, 10 tracked AI prompts. Advanced: €419 per month, 50 projects, 5,000 tracked keywords, 20 tracked AI prompts. Enterprise: €1,394 per month with an annual commitment. Starter: €27. Ahrefs Free: free. Brand Radar AI: from €179. Each plan includes 1 user and extra users cost extra. The page says paying annually saves up to 17%. Prices as shown to a visitor in Germany on 10 Oct 2026. ## How to switch from Ahrefs 1. **Export your keywords from Ahrefs** Open your project in Rank Tracker and export the keyword list. The Ahrefs [help centre](https://help.ahrefs.com/en/articles/2351513-can-i-duplicate-or-merge-projects-websites-in-rank-tracker) describes exporting keywords and re-uploading them as CSV or TXT. Save the keywords as `keywords.txt`, one per line. Ranking history does not carry over, so keep the export as your archive. 2. **Sign in and create a project** Install the CLI as described on the [CLI page](https://serpel.app/developers/cli), sign in and create a project for your domain. Copy the project ID from the output. ```bash serpel auth login serpel projects create --domain example.com --country DE --language de export PROJECT=<projectId> ``` 3. **Import keywords with their metrics** Create a keyword list and import the CSV. Serpel maps header columns such as keyword, search volume, cpc and kd, and sets country and language for the whole file. ```bash serpel keywords list-create --project $PROJECT --name "Imported from Ahrefs" serpel keywords import <listId> --file keywords.csv --country DE --language de ``` 4. **Start tracking and run the first check** Add the keywords to rank tracking, up to 200 per call and within your plan’s limit. A check run right now books 5 credits per keyword in the top 50, so estimate it first. ```bash args=() while IFS= read -r keyword; do args+=(--keyword "$keyword"); done < keywords.txt serpel billing estimate --operation rankCheck --keywords "$(wc -l < keywords.txt)" serpel rankings add --project $PROJECT --device desktop "${args[@]}" serpel rankings run --project $PROJECT --wait ``` 5. **Re-create competitors and AI prompts** Add the competitor domains from Rank Tracker and the prompts you track in Brand Radar. ```bash serpel competitors add --project $PROJECT --domain competitor.com serpel ai add --project $PROJECT --prompt "which seo tool has an api for developers" ``` ## Frequently asked questions ### Is there an Ahrefs alternative for developers? Yes. Serpel combines rank tracking, site audits and AI visibility checks with a CLI, a REST API and an MCP server, and it scans Next.js projects. Ahrefs offers an API and an MCP server as well, with no CLI listed in its developer documentation. ### How much does Ahrefs cost compared with Serpel? On 10 Oct 2026 Ahrefs showed Lite at €119, Standard at €229 and Advanced at €419 per month. Serpel needs no subscription, and its optional plans cost €19 and €49. The scope differs, because Ahrefs includes a backlink index and research tools. ### Does Serpel have a backlink checker like Ahrefs? No. Serpel has no backlink index. If backlinks matter, Ahrefs is the better choice, and you can use Serpel next to it for rankings, audits and AI visibility. ### How often does Ahrefs update rankings compared with Serpel? The Ahrefs help centre says tracked keywords update weekly by default on all plans. Serpel schedules daily checks on Pro, every 3 days or weekly otherwise, and runs manual checks on demand. ## Sources - [Ahrefs: plans and pricing](https://ahrefs.com/pricing), accessed 2026-10-10 - [Ahrefs: data and index figures](https://ahrefs.com/big-data), accessed 2026-10-10 - [Ahrefs: Rank Tracker](https://ahrefs.com/rank-tracker), accessed 2026-10-10 - [Ahrefs help centre: how often tracked keywords update](https://help.ahrefs.com/en/articles/604245-how-frequently-do-my-tracked-keywords-get-updated), accessed 2026-10-10 - [Ahrefs: Site Audit](https://ahrefs.com/site-audit), accessed 2026-10-10 - [Ahrefs: Brand Radar](https://ahrefs.com/brand-radar), accessed 2026-10-10 - [Ahrefs: SEO MCP](https://ahrefs.com/seo-mcp), accessed 2026-10-10 - [Ahrefs: developer documentation](https://docs.ahrefs.com/), accessed 2026-10-10 - [Ahrefs help centre: exporting and re-uploading Rank Tracker keywords](https://help.ahrefs.com/en/articles/2351513-can-i-duplicate-or-merge-projects-websites-in-rank-tracker), accessed 2026-10-10 --- # The AccuRanker alternative for developers URL: https://serpel.app/alternatives/accuranker Updated: 2026-10-10 Serpel is an AccuRanker alternative for developers who want Google rank tracking next to site audits and AI visibility checks, billed per use instead of per plan, from a CLI, an API or a coding agent. AccuRanker is the better fit if you track thousands of keywords every day, need on-demand refreshes or want its SERP feature and click-through metrics. We checked every AccuRanker fact on 10 Oct 2026 against its own pages. ## Choose Serpel if - You want rankings, site audits and AI visibility in one account: AccuRanker’s feature list does not include a site audit. - You track up to 500 keywords per project and prefer paying per check: a scheduled check costs 2 credits (€0.02) per keyword in the top 50. - You work in a terminal, in CI or with a coding agent: Serpel has a CLI with JSON output, and its REST API is available on every plan. - You build with Next.js and want a codebase scan that ties recommendations to your routes. ## Choose AccuRanker if - You track thousands of keywords daily: AccuRanker’s plans cover 2,000 to 25,000 keywords, while Serpel caps a project at 500. - You need fast refreshes: AccuRanker says you can refresh a single keyword or update an entire domain every two hours. - You want deeper SERP data: AccuRanker lists 45+ SERP features, pixel position, click-through rates and AI share of voice. Serpel tracks position, ranking URL and AI Overview presence. - You want a dedicated AI tracker with sentiment: AccuLLM covers ChatGPT, Perplexity, AI Overviews and AI Mode. Serpel checks ChatGPT and Google AI Overviews. ## When is Serpel an AccuRanker alternative? AccuRanker is a rank tracker. Its [feature list](https://www.accuranker.com/features/) covers rank tracking, tagging, reporting, a keyword database, SERP analysis, forecasting, an API, an MCP server and AccuLLM, a separate product for AI visibility. Serpel also tracks Google rankings, as one part of a wider SEO tool for developers. It audits sites with JavaScript rendering and Core Web Vitals, reads Search Console and Bing data, and checks whether ChatGPT and Google AI Overviews cite you. It runs in the dashboard, the CLI, the REST API and an [MCP server](https://serpel.app/developers/mcp). Serpel is in public beta. ## How do Serpel and AccuRanker differ? - **Scope:** AccuRanker focuses on rank tracking and SERP analysis. Serpel adds audits, search data and a codebase scan. - **Billing:** AccuRanker plans are sized by keyword count. Serpel charges credits per check, and the same credits pay for audits and AI checks. - **Speed:** AccuRanker updates daily and offers on-demand refreshes. Serpel checks on a schedule, daily on Pro, or on demand. - **AI visibility:** AccuLLM has its own price list. Serpel includes ChatGPT and Google AI Overview checks in its credits. ## How does AccuRanker pricing compare with Serpel? AccuRanker’s Professional plan shows $224 per month with yearly payment ($249 paid monthly) for 2,000 keywords updated daily. Checking 500 keywords daily with Serpel, its per-project limit, costs about 30,000 credits (€300.00) a month at €0.01 per credit, so a flat plan is cheaper at high daily volume. Checking 100 keywords weekly costs about 858 credits (€8.58), and that is where per-use pricing helps. See the [price list](https://serpel.app/pricing). ## What does AccuRanker do better than Serpel? AccuRanker is built for volume and speed. Its [rank tracking page](https://www.accuranker.com/features/rank-tracking/) describes daily updates, pixel position, click-through rates and on-demand refreshes, and its SERP analysis tracks 45+ SERP features. Serpel has no pixel position and tracks no SERP features beyond AI Overviews. > **How we checked:** AccuRanker facts come from its public pages, read on 10 Oct 2026 and linked in the sources. The pricing page showed dollar amounts and changes over time. AccuRanker is a trademark of its owner, used here only to describe its product. Serpel is not affiliated with AccuRanker. Spotted something out of date? Tell us through the [About page](https://serpel.app/about). ## Serpel and AccuRanker compared ### Rank tracking | Feature | Serpel | AccuRanker | | --- | --- | --- | | Update frequency | Daily on Pro, otherwise every 3 days or weekly, plus manual checks | Daily updates, plus on-demand refresh of a single keyword or an entire domain every two hours | | Keywords tracked | 50, 200, 500 per project (Free, Starter, Pro) | 2,000 to 25,000 on Professional and Expert, more on Enterprise | | Local and country tracking | Desktop and mobile, nationwide, city or postal code, to position 100 | Rankings at local or country level | | SERP features | AI Overview presence and citation only | 45+ SERP features, including AI Overviews | | Pixel position | Not tracked | Pixel position above or below the fold, plus click-through rate by position | ### AI visibility | Feature | Serpel | AccuRanker | | --- | --- | --- | | AI platforms | ChatGPT with web search and Google AI Overviews | AccuLLM: ChatGPT, Perplexity, AI Overviews and AI Mode | | Sentiment and sources | Cited or mentioned, cited URL, sources, no sentiment score | Sentiment score from 1 to 100, most cited domains, share of citations | | AI Overviews in rankings | Presence and citation of your domain per keyword | Keywords that trigger an AI Overview, contributing URLs, whether yours is one | | Prompts included | 10, 25, 50 per project (Free, Starter, Pro) | LLM Lite: 100 prompts, 1 brand. LLM Pro: 300 prompts, 5 brands. | ### Site, search data and reporting | Feature | Serpel | AccuRanker | | --- | --- | --- | | Site audit | 94 checks in 18 categories, with JavaScript rendering and Core Web Vitals | Not listed among its features | | Search Console and analytics | Google Search Console, Bing Webmaster Tools, GA4, PostHog, Plausible | Google Search Console and Google Analytics on every plan | | Keyword research | Research per search, priced in credits | Keyword database of 24B+ keywords with domain lookup, by its own count | | Reports and exports | Monitoring reports by email, Slack, Discord or webhook, CSV and JSON export | Report templates, CSV, Excel, Google Sheets and PDF downloads | ### Developer access | Feature | Serpel | AccuRanker | | --- | --- | --- | | CLI | Every feature, with JSON output | Not listed on its API and integrations page | | REST API | Same API as the dashboard, on every plan, 18 token scopes | Unlimited read API on Expert, write API on Enterprise | | MCP server | serpel.app/mcp, paid tools run within a budget you approve | AccuRanker MCP for ChatGPT and Claude, plan not stated | | Codebase scan | Scans a Next.js project, never reads source code | Not listed among its features | ## Pricing - **Serpel:** Pay per credit, with optional monthly plans. Free: no subscription, 100 credits to start. Starter: €19 per month incl. VAT, 2,000 credits every month. Pro: €49 per month incl. VAT, 6,000 credits every month. One credit is worth €0.01. A scheduled check costs 2 credits per keyword in the top 50. Top-ups start at €5, and bought credits never expire. - **AccuRanker:** Plans by keyword count, billed monthly or yearly. Professional: 2,000 keywords at $224 per month with yearly payment ($249 monthly), 3,000 at $332 ($369), 5,000 at $548 ($609). Daily updates. Expert: 10,000 keywords at $764 ($849 monthly), up to 25,000 at $1,412 ($1,569). Adds AI CTR, AI Search Volume and an unlimited read API. Enterprise: custom price for more than 25K keywords. AccuLLM, sold separately: LLM Lite at $229 per month ($206 yearly) for 100 prompts and 1 brand, LLM Pro at $579 ($521) for 300 prompts and 5 brands. The page says yearly payment gives a 10% discount. Prices as shown on 10 Oct 2026. ## How to switch from AccuRanker 1. **Export your keywords from AccuRanker** Open the keyword list of your domain, choose Download Report and pick CSV, as the [help guide](https://www.accuranker.com/help/reports/downloading-reports/) describes. Save the keyword column as `keywords.txt`, one keyword per line. Serpel cannot import ranking history, so keep the export as your archive. 2. **Sign in and create a project** Install the CLI as described on the [CLI page](https://serpel.app/developers/cli), sign in and create a project for your domain. Copy the project ID from the output. ```bash serpel auth login serpel projects create --domain example.com --country US --language en export PROJECT=<projectId> ``` 3. **Import keywords with their metrics** Create a keyword list and import the CSV. Serpel maps header columns such as keyword, search volume, cpc and kd. ```bash serpel keywords list-create --project $PROJECT --name "Imported from AccuRanker" serpel keywords import <listId> --file keywords.csv ``` 4. **Start tracking within your plan’s limit** Serpel tracks up to 500 keywords per project on Pro, so choose the keywords that matter most. A check run right now books 5 credits per keyword in the top 50, so estimate it first. ```bash args=() while IFS= read -r keyword; do args+=(--keyword "$keyword"); done < keywords.txt serpel billing estimate --operation rankCheck --keywords "$(wc -l < keywords.txt)" serpel rankings add --project $PROJECT "${args[@]}" serpel rankings run --project $PROJECT --wait ``` 5. **Re-create your AI prompts** Add the prompts you track in AccuLLM. Serpel checks them on ChatGPT and Google AI Overviews. ```bash serpel ai add --project $PROJECT --prompt "which rank tracker has an api" --country US --language en serpel ai run --project $PROJECT --wait ``` ## Frequently asked questions ### Is there an AccuRanker alternative that also audits my site? Yes. Serpel tracks Google rankings and also runs site audits with 94 checks, JavaScript rendering and Core Web Vitals, and it checks AI visibility. AccuRanker does not list a site audit among its features. ### How much does AccuRanker cost compared with Serpel? On 10 Oct 2026 AccuRanker’s Professional plan showed $224 per month with yearly payment for 2,000 keywords, or $249 paid monthly. Serpel needs no subscription and bills credits per check. Checking 500 keywords daily costs about €300.00 a month, so a flat plan is cheaper at high daily volume. ### Can Serpel track as many keywords as AccuRanker? No. Serpel allows up to 500 tracked keywords per project on Pro, while AccuRanker’s plans start at 2,000 keywords and reach 25,000 on Expert. Serpel is also in public beta. ### Does AccuRanker have an API or an MCP server? Yes. According to its pricing and feature pages, AccuRanker offers an unlimited read API on Expert and an MCP server that connects its data to ChatGPT and Claude. Serpel offers a REST API on every plan, an MCP server and a CLI. ## Sources - [AccuRanker: pricing](https://www.accuranker.com/pricing/), accessed 2026-10-10 - [AccuRanker: features](https://www.accuranker.com/features/), accessed 2026-10-10 - [AccuRanker: rank tracking](https://www.accuranker.com/features/rank-tracking/), accessed 2026-10-10 - [AccuRanker: SERP analysis](https://www.accuranker.com/features/serp-analysis/), accessed 2026-10-10 - [AccuRanker: AccuLLM](https://www.accuranker.com/features/accullm/), accessed 2026-10-10 - [AccuRanker: keyword research database](https://www.accuranker.com/features/keyword-research/), accessed 2026-10-10 - [AccuRanker: MCP](https://www.accuranker.com/features/accuranker-mcp/), accessed 2026-10-10 - [AccuRanker: API and integrations](https://www.accuranker.com/features/api-integrations/), accessed 2026-10-10 - [AccuRanker help: downloading reports](https://www.accuranker.com/help/reports/downloading-reports/), accessed 2026-10-10 --- # The Peec AI alternative for developers URL: https://serpel.app/alternatives/peec-ai Updated: 2026-10-10 Serpel is a Peec AI alternative for developers who want to see whether ChatGPT and Google AI Overviews cite their site next to their Google rankings, billed per check, from a CLI, an API or a coding agent. Peec AI is the better fit if you need more AI platforms, sentiment, source analytics or reporting for marketing teams and agencies. We checked every Peec AI fact on 10 Oct 2026 against its own pages. ## Choose Serpel if - You want Google rankings, site audits and AI visibility in one account: Peec AI’s pricing page does not list Google rank tracking or technical site audits. - You prefer to pay per check: 2 credits (€0.02) per question and platform, with no subscription. - You run checks from a terminal, in CI or from a coding agent: the CLI can add, run and read AI prompts with JSON output. - You build with Next.js and want a codebase scan that ties recommendations to your routes. ## Choose Peec AI if - You need more AI platforms: Peec AI lets you choose 3 of ChatGPT, AI Mode, AI Overviews, Copilot, Gemini and Naver AI, and Enterprise tracks up to 13 models. Serpel checks ChatGPT and Google AI Overviews. - You need sentiment and source analytics: Peec AI lists sentiment, brand perception, source classification and gap analysis. Serpel reports citations, mentions, sources and an excerpt. - You manage several brands, clients or countries: Peec AI lists multi-country tracking on Advanced and agency bundles. Serpel works per project and is in public beta. - You want a broader AI analytics suite: Peec AI lists a crawlability check across 40+ AI bots, AI referrals, AI shopping tracking and a Data Studio connector. ## When is Serpel a Peec AI alternative? Peec AI is an AI search analytics tool for marketing teams and agencies. According to its [documentation](https://docs.peec.ai/intro-to-peec-ai), it runs your prompts across AI platforms daily and measures visibility, position and sentiment, collecting answers through the platforms’ web interfaces instead of their APIs. Serpel treats AI visibility as one part of SEO. You add questions the way people ask an AI, and Serpel checks whether ChatGPT with web search and Google AI Overviews cite or mention your domain, then stores sources and an excerpt. The same account tracks Google rankings and audits your site, through the dashboard, the CLI, the REST API or an [MCP server](https://serpel.app/developers/mcp). Among Peec AI competitors, Serpel is a developer-focused option in public beta. ## How do Serpel and Peec AI differ? - **Focus:** Peec AI is built around AI search analytics. Serpel covers AI visibility next to rankings, audits and search data. - **Platforms:** Peec AI lets you choose 3 of 6 platforms and tracks up to 13 models on Enterprise. Serpel checks 2 AI surfaces. - **Billing:** Peec AI sells monthly plans sized by prompt count. Serpel charges credits per check. - **API access:** both offer an MCP server. Peec AI’s API is limited to Enterprise customers, while Serpel’s REST API and CLI work on every plan. ## How does Peec AI pricing compare with Serpel? Peec AI’s Starter plan shows €85 per month, billed monthly, for 50 prompts on 3 models of your choice, tracked daily. Checking 50 prompts daily on ChatGPT and Google AI Overviews with Serpel is 3,000 AI answers a month and costs about 6,000 credits (€60.00) at €0.01 per credit. Checking 10 prompts weekly costs about 172 credits (€1.72). Peec AI’s plans include sentiment, source analytics and unlimited users, so the two are not like for like. See the [price list](https://serpel.app/pricing). ## What does Peec AI do better than Serpel? Peec AI covers more platforms, scores sentiment and brand perception, classifies sources, suggests prompts and actions, and offers a Data Studio connector and agency bundles. Its [crawlability check](https://docs.peec.ai/crawlability) covers 40+ AI bots, while Serpel checks 13 AI crawlers and your llms.txt. > **How we checked:** Peec AI facts come from its public pricing page and documentation, read on 10 Oct 2026 and linked in the sources. Prices were shown in euros to a visitor in Germany and change over time. Peec AI is a trademark of its owner, used here only to describe its product. Serpel is not affiliated with Peec AI. Spotted something out of date? Tell us through the [About page](https://serpel.app/about). ## Serpel and Peec AI compared ### AI visibility tracking | Feature | Serpel | Peec AI | | --- | --- | --- | | AI platforms | ChatGPT with web search and Google AI Overviews | Choose 3 of ChatGPT, AI Mode, AI Overviews, Copilot, Gemini, Naver AI. Enterprise: up to 13 models | | Tracked prompts | 10, 25, 50 per project (Free, Starter, Pro) | 50, 150 or 350 on Starter, Pro and Advanced | | Check frequency | Daily on Pro, otherwise every 3 days or weekly, plus manual checks | Daily on Starter, Pro and Advanced | | Citations and sources | Cited or mentioned, cited URL, sources, answer excerpt | Source analytics by domain and URL, citation share, source classification | | Sentiment and brand perception | Not offered | Sentiment, brand attribute scoring, objections, fact-checking | ### Beyond AI answers | Feature | Serpel | Peec AI | | --- | --- | --- | | Google rank tracking | Desktop and mobile, nationwide, city or postal code, to position 100 | Not listed on its pricing page | | Site audit | 94 checks in 18 categories, with JavaScript rendering and Core Web Vitals | No technical SEO audit listed on its pricing page | | AI crawler access | robots.txt check for 13 AI crawlers, plus llms.txt | robots.txt check across 40+ AI bots from 20+ vendors | | Search Console and analytics | Google Search Console, Bing Webmaster Tools, GA4, PostHog, Plausible | Google Analytics integration and AI referral traffic | ### Developer access and reporting | Feature | Serpel | Peec AI | | --- | --- | --- | | CLI | Every feature, with JSON output | Not listed in its documentation index | | REST API | Same API as the dashboard, on every plan, 18 token scopes | Currently limited to Enterprise customers, per its API documentation | | MCP server | serpel.app/mcp, paid tools run within a budget you approve | MCP server for Claude, Cursor, VS Code, Windsurf, with read and write tools | | Codebase scan | Scans a Next.js project, never reads source code | Not listed on its pricing page | | Exports and reporting | CSV and JSON export, report emails, Slack, Discord and webhook alerts | CSV exports, shareable dashboards, Data Studio connector | ## Pricing - **Serpel:** Pay per credit, with optional monthly plans. Free: no subscription, 100 credits to start. Starter: €19 per month incl. VAT, 2,000 credits every month. Pro: €49 per month incl. VAT, 6,000 credits every month. One credit is worth €0.01. A ChatGPT or Google AI Overview check costs 2 credits per question. Top-ups start at €5, and bought credits never expire. - **Peec AI:** Monthly plans by prompt count, annual billing for Enterprise. Starter: €85 per month billed monthly, 50 prompts, 3 models of your choice, daily tracking, 1 project, unlimited users. Pro: €205, 150 prompts, 3 models, 2 projects. Advanced: €425, 350 prompts, 3 models, 5 projects, multi-country tracking, Looker Studio integration. Enterprise: custom price, annual billing, all models, API access, single sign-on. Extra models are sold as add-ons, and annual billing gives a 15% discount. Prices as shown to a visitor in Germany on 10 Oct 2026. ## How to switch from Peec AI 1. **Export your prompts from Peec AI** Copy the prompts you track into a text file called `prompts.txt`, one prompt per line. Peec AI’s [pricing page](https://peec.ai/pricing) lists custom CSV exports under data access, so check whether your plan includes them. Keep any exports as your archive, because history does not transfer. 2. **Sign in and create a project** Install the CLI as described on the [CLI page](https://serpel.app/developers/cli), sign in and create a project for your domain. Copy the project ID from the output. ```bash serpel auth login serpel projects create --domain example.com --country US --language en export PROJECT=<projectId> ``` 3. **Add your prompts** Add the prompts to AI visibility tracking. A project holds at most 50 prompts on Pro and fewer on the other plans, so start with the ones that matter most. ```bash args=() while IFS= read -r prompt; do args+=(--prompt "$prompt"); done < prompts.txt serpel ai add --project $PROJECT --country US --language en "${args[@]}" ``` 4. **Estimate, run and read the first check** Estimate the credits, run the check and read the summary of citations, mentions and sources. Add your Google keywords with `serpel rankings add` if you want rankings in the same project. ```bash serpel billing estimate --operation aiCheck --prompts "$(wc -l < prompts.txt)" serpel ai run --project $PROJECT --wait serpel ai status --project $PROJECT ``` ## Frequently asked questions ### Is there a Peec AI alternative with Google rank tracking? Yes. Serpel tracks Google rankings on desktop and mobile, audits websites and checks ChatGPT and Google AI Overviews in one account. Peec AI’s pricing page does not list Google rank tracking or technical site audits. ### How much does Peec AI cost compared with Serpel? On 10 Oct 2026 Peec AI showed Starter at €85, Pro at €205 and Advanced at €425 per month when billed monthly. Serpel needs no subscription and charges 2 credits (€0.02) per prompt and platform. The scope differs, because Peec AI adds sentiment, source analytics and more platforms. ### Which AI platforms does Serpel track compared with Peec AI? Serpel checks ChatGPT with web search and Google AI Overviews. Peec AI lets you choose 3 of ChatGPT, AI Mode, AI Overviews, Copilot, Gemini and Naver AI, and its Enterprise plan tracks up to 13 models. ### Does Peec AI have an API or an MCP server? Yes. Peec AI’s documentation describes an MCP server and an API, and says API access is currently limited to Enterprise customers. Serpel offers a REST API and a CLI on every plan, plus an MCP server. ## Sources - [Peec AI: pricing](https://peec.ai/pricing), accessed 2026-10-10 - [Peec AI documentation: introduction and data collection](https://docs.peec.ai/intro-to-peec-ai), accessed 2026-10-10 - [Peec AI documentation: index of all pages](https://docs.peec.ai/llms.txt), accessed 2026-10-10 - [Peec AI documentation: crawlability](https://docs.peec.ai/crawlability), accessed 2026-10-10 - [Peec AI documentation: API](https://docs.peec.ai/api/introduction), accessed 2026-10-10 - [Peec AI documentation: MCP server](https://docs.peec.ai/mcp/introduction), accessed 2026-10-10 --- # Notes on search, AI answers and the code behind them. URL: https://serpel.app/blog Practical guides on how search engines and AI assistants find, read and cite websites. Written by the Serpel team, with sources for every claim. - [AI crawlers: which to allow, which to block, and how to do it](https://serpel.app/blog/ai-crawlers): A practical list of AI crawlers: what GPTBot, ClaudeBot and others do, which to allow or block, plus robots.txt examples you can copy and test. - [AI SEO: how to use AI for SEO work and how to optimise for AI search](https://serpel.app/blog/ai-seo): AI SEO means using AI to do SEO work and optimising for AI search. What to automate, where AI fails, what Google allows and which tool types exist. - [Answer engine optimization (AEO): what it is and how to do it](https://serpel.app/blog/answer-engine-optimization): Answer engine optimization (AEO) means writing content AI answers can cite: how engines pick sources, which page patterns help and how to measure. - [Best AI visibility tools in 2026: what they track and what they cost](https://serpel.app/blog/best-ai-visibility-tools): The best AI visibility tools compared: what Semrush, Ahrefs, Peec AI, Otterly.AI, Profound and Serpel track, their public pricing and how to choose. - [Bing Search Console: how to set up Bing Webmaster Tools](https://serpel.app/blog/bing-webmaster-tools): Bing Search Console is Bing Webmaster Tools. Set it up step by step: import from Google, verify, submit sitemaps, use IndexNow and read the reports. - [Core Web Vitals test: how to measure and fix LCP, INP and CLS](https://serpel.app/blog/core-web-vitals-test): Run a Core Web Vitals test with PageSpeed Insights, Search Console, Lighthouse, CrUX and Serpel, then fix LCP, INP and CLS with a table of causes. - [Generative engine optimization (GEO): what it is and what works](https://serpel.app/blog/generative-engine-optimization): Generative engine optimization (GEO) is how you get cited in AI answers. See what the research found, what Google says and a checklist you can use. - [GEO vs SEO: what is different, what stays the same and how to measure it](https://serpel.app/blog/geo-vs-seo): GEO vs SEO: SEO earns rankings and clicks, GEO earns citations in AI answers. See the differences, what stays the same and how to measure both. - [Google Search Console MCP: connect your agent to search data](https://serpel.app/blog/google-search-console-mcp): Set up a Google Search Console MCP server for Claude Code and other agents: the options compared and Serpel’s search performance tool step by step. - [SEO for ChatGPT: how to rank in ChatGPT search and get cited](https://serpel.app/blog/how-to-rank-in-chatgpt): SEO for ChatGPT means letting OAI-SearchBot in and being quotable. How ChatGPT search picks sources, what to do and how to check citations. - [JavaScript SEO: how Google crawls, renders and indexes JavaScript](https://serpel.app/blog/javascript-seo): JavaScript SEO explained: how Googlebot crawls, renders and indexes JS, which rendering strategy to choose and how to test what Google sees. - [LLM SEO: how to get found and cited by AI assistants](https://serpel.app/blog/llm-seo): LLM SEO makes your site easy for ChatGPT, Claude and Gemini to find and cite. Training data vs live retrieval, crawler access, llms.txt and how to measure. - [llms.txt example: 3 complete files for docs, shops and blogs](https://serpel.app/blog/llms-txt-examples): Three complete llms.txt examples for a SaaS docs site, an online shop and a blog, plus the format rules, llms-full.txt and a validation script. - [Next.js SEO: the App Router guide to metadata, sitemaps and rendering](https://serpel.app/blog/nextjs-seo): Next.js SEO best practices for the App Router: metadata, sitemaps, robots, JSON-LD, rendering and Core Web Vitals, with working code for Next.js 15 and 16. - [SEO audit report example: Serpel’s own crawl of serpel.app](https://serpel.app/blog/seo-audit-report-example): An SEO audit report example from Serpel’s own crawl of serpel.app: score, issues by severity, Core Web Vitals, fixes and a format you can copy. - [Soft 404 errors: what they are and how to fix them](https://serpel.app/blog/soft-404): What is a soft 404? How Google defines it, what causes soft 404 errors, how Search Console reports them and how to fix each case, plus how to detect them. --- # AI crawlers: which to allow, which to block, and how to do it URL: https://serpel.app/blog/ai-crawlers Updated: 2026-10-10 AI crawlers are bots that fetch web pages for AI products, and they fall into three groups: training, AI search and user-requested fetches. Allow the search and user crawlers if you want to appear in AI answers, and block the training crawlers if you do not want your content used for model training. This guide lists the 13 AI crawlers Serpel checks, with robots.txt files you can copy. ## Key takeaways - AI crawlers fall into three groups: training (collecting content for models), AI search (building an index that answers cite) and user fetch (loading a page because a person asked). - Blocking a training crawler does not remove you from AI search. OpenAI says each robots.txt setting is independent, and Google says Google-Extended does not affect inclusion or ranking in Google Search. - User-triggered fetchers such as ChatGPT-User and Perplexity-User may ignore robots.txt, and robots.txt is not access control under RFC 9309, so hard blocking needs a firewall or authentication. - A crawler follows the group that names it and ignores your general rules, so repeat any Disallow lines you still need. Check your CDN too, because bot protection can block crawlers that robots.txt allows. - Test the result with the free robots.txt checker and by counting AI crawler requests in your server logs. ## What are AI crawlers? AI crawlers are the automated user agents that AI companies use to fetch web pages. They differ from Googlebot in what they do with the pages afterwards. OpenAI’s documentation draws the same three-way split that other vendors use: some bots collect content that may be used to train models, some build the index that powers AI search, and some visit a page only because a user asked a question. - **Training:** GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, CCBot, Meta-ExternalAgent. They collect content that may be used to train models, or in the case of Google-Extended and Applebot-Extended, they are control tokens that govern that use. - **AI search:** OAI-SearchBot, Claude-SearchBot, PerplexityBot, MistralAI-Index. They build the indexes that AI search products cite. Blocking them can remove you from that product’s answers. - **User fetch:** ChatGPT-User, Claude-User, Perplexity-User. They fetch a page on request, for example when someone asks a question that needs it. ## Which AI crawlers should you know? The table lists every crawler Serpel checks, with what each vendor’s own documentation says. The purpose column is Serpel’s classification. CCBot, for example, counts as training because Common Crawl’s open data can be used by any organisation. **AI crawlers checked by Serpel and what their vendors document** | User agent | Vendor | Purpose | What the vendor says | | --- | --- | --- | --- | | OAI-SearchBot | OpenAI | AI search | Surfaces websites in ChatGPT’s search features. Sites that opt out are not shown in ChatGPT search answers, and changes can take about 24 hours to apply. [OpenAI docs](https://developers.openai.com/api/docs/bots) | | GPTBot | OpenAI | Training | Crawls content that may be used to train OpenAI’s generative AI foundation models. Disallowing it signals that your content should not be used for training. [OpenAI docs](https://developers.openai.com/api/docs/bots) | | ChatGPT-User | OpenAI | User fetch | Visits a page when a user asks ChatGPT or a custom GPT something that needs it. OpenAI says robots.txt rules may not apply to these user-initiated requests. [OpenAI docs](https://developers.openai.com/api/docs/bots) | | Claude-SearchBot | Anthropic | AI search | Analyses content to improve the relevance of Claude’s search results. Anthropic says disabling it may reduce your visibility in search results. [Anthropic docs](https://privacy.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) | | ClaudeBot | Anthropic | Training | Collects web content that could contribute to model training. A block tells Anthropic to exclude your future materials from training datasets. [Anthropic docs](https://privacy.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) | | Claude-User | Anthropic | User fetch | Accesses pages when a Claude user asks a question that needs them. Anthropic says disabling it may reduce your visibility in user-directed search. [Anthropic docs](https://privacy.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) | | PerplexityBot | Perplexity | AI search | Surfaces and links websites in Perplexity search results. Perplexity says it is not used to crawl content for AI foundation models. [Perplexity docs](https://docs.perplexity.ai/docs/resources/perplexity-crawlers) | | Perplexity-User | Perplexity | User fetch | Visits a page to answer a user’s question and links to it. Perplexity says this fetcher generally ignores robots.txt because a user requested the fetch. [Perplexity docs](https://docs.perplexity.ai/docs/resources/perplexity-crawlers) | | Google-Extended | Google | Training | A robots.txt token rather than a separate crawler. It controls whether content Google crawls can be used for Gemini training and grounding, and Google says it does not affect inclusion or ranking in Search. [Google docs](https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers) | | Applebot-Extended | Apple | Training | Lets you opt out of Apple using your content to train its foundation models. Apple says it does not crawl pages itself and that blocked pages can still appear in search results. [Apple docs](https://support.apple.com/en-us/119829) | | CCBot | Common Crawl | Training | The crawler of Common Crawl, a non-profit that publishes open web crawl data for any organisation to use. A Disallow rule keeps your pages out of its future crawls. [Common Crawl docs](https://commoncrawl.org/ccbot) | | Meta-ExternalAgent | Meta | Training | Crawls to train AI models or to improve products by indexing content directly, according to Meta. Crawlers may cache robots.txt for up to 24 hours. [Meta docs](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers) | | MistralAI-Index | Mistral | AI search | Crawls for indexing only, to power Mistral’s search. Mistral says content it crawls is not used for generative AI training. [Mistral docs](https://docs.mistral.ai/robots) | ## Should you block AI crawlers or allow them? It depends on what you want from AI products. The groups are independent, so you can mix them. OpenAI states that a site can allow OAI-SearchBot to appear in search while disallowing GPTBot so its content is not used for training. **Three common robots.txt strategies for AI crawlers** | Goal | Allow | Block | | --- | --- | --- | | Appear in AI answers, opt out of training | OAI-SearchBot, Claude-SearchBot, PerplexityBot, MistralAI-Index, ChatGPT-User, Claude-User, Perplexity-User | GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, CCBot, Meta-ExternalAgent | | Appear everywhere, including training datasets | All of them | None | | Opt out of every AI crawler | None | OAI-SearchBot, GPTBot, ChatGPT-User, Claude-SearchBot, ClaudeBot, Claude-User, PerplexityBot, Perplexity-User, Google-Extended, Applebot-Extended, CCBot, Meta-ExternalAgent, MistralAI-Index | Two points often cause confusion. First, Google’s AI Overviews and AI Mode draw on the normal Search index, so they are controlled with Googlebot rules and snippet controls, not with Google-Extended. [Google says](https://developers.google.com/search/docs/appearance/ai-features) robots.txt rules for Googlebot govern crawling for Search including AI features, and that nosnippet, data-nosnippet, max-snippet and noindex limit what is shown. Blocking Googlebot would remove you from Google Search, so never use it to opt out of AI. Second, a block affects future crawling. Anthropic describes a ClaudeBot block as excluding a site’s future materials from training datasets, so do not expect robots.txt to undo collection that already happened. ## Ready-to-copy robots.txt examples The files below are generated from the same crawler list the table uses. For a GPTBot robots.txt rule or a ClaudeBot robots.txt rule, add the crawler’s name as a User-agent line. Replace the example paths with your own and keep your existing rules for other crawlers. Because a crawler follows the group that names it and ignores the general group, the allowed bots repeat the Disallow line for the private path. ```text User-agent: GPTBot User-agent: ClaudeBot User-agent: Google-Extended User-agent: Applebot-Extended User-agent: CCBot User-agent: Meta-ExternalAgent Disallow: / User-agent: OAI-SearchBot User-agent: Claude-SearchBot User-agent: PerplexityBot User-agent: MistralAI-Index User-agent: ChatGPT-User User-agent: Claude-User User-agent: Perplexity-User Allow: / Disallow: /admin/ ``` ```text User-agent: OAI-SearchBot User-agent: GPTBot User-agent: ChatGPT-User User-agent: Claude-SearchBot User-agent: ClaudeBot User-agent: Claude-User User-agent: PerplexityBot User-agent: Perplexity-User User-agent: Google-Extended User-agent: Applebot-Extended User-agent: CCBot User-agent: Meta-ExternalAgent User-agent: MistralAI-Index Disallow: / ``` ```text User-agent: GPTBot User-agent: ClaudeBot User-agent: Google-Extended User-agent: Applebot-Extended User-agent: CCBot User-agent: Meta-ExternalAgent Disallow: /members/ Disallow: /downloads/ ``` Several user-agent lines above one set of rules form a single group, which [RFC 9309](https://www.rfc-editor.org/rfc/rfc9309.html) allows. Product tokens are matched case-insensitively, so GPTBot and gptbot are the same rule. Search systems need time to notice a change: OpenAI says about 24 hours, and Meta says its crawlers may cache robots.txt for up to 24 hours. ## Does robots.txt actually stop AI crawlers? Only the ones that choose to obey it. RFC 9309 is explicit that the rules are not a form of access authorisation. Vendors document what their bots do: - **Training and search crawlers.** Anthropic says its bots honour robots.txt, OpenAI manages OAI-SearchBot and GPTBot through it, and Meta presents it as the way to block Meta-ExternalAgent. - **User-triggered fetchers.** OpenAI says robots.txt rules may not apply to ChatGPT-User, Perplexity says Perplexity-User generally ignores them, and Meta says Meta-ExternalFetcher may bypass them. - **Verification.** OpenAI, Perplexity, Anthropic, Common Crawl and Mistral publish IP ranges for their crawlers, so you can tell a real request from one that merely copies a name. Anthropic warns that blocking by IP address may not reliably guarantee an opt-out, so it recommends robots.txt. If you need a hard block, use a firewall rule or authentication. The reverse problem is just as common: bot protection can block a crawler that robots.txt allows. OpenAI recommends allowing its published IP ranges for OAI-SearchBot, and Vercel’s [analysis of AI crawlers](https://vercel.com/blog/the-rise-of-the-ai-crawler) mentions a firewall rule that blocks AI bots, which would also stop the ones you want. ## How do you check which AI crawlers your site allows? 1. **Test your robots.txt** Paste your domain into the free [robots.txt checker](https://serpel.app/tools/robots-txt-checker) to see which AI crawlers your file allows and which it blocks. 2. **Count the requests in your logs** The command below counts requests per AI crawler in an access log. Google-Extended and Applebot-Extended never appear, because Google says Google-Extended has no separate user agent and Apple says Applebot-Extended does not crawl pages. ```bash grep -o -i -E "OAI-SearchBot|GPTBot|ChatGPT-User|Claude-SearchBot|ClaudeBot|Claude-User|PerplexityBot|Perplexity-User|CCBot|Meta-ExternalAgent|MistralAI-Index" access.log | sort | uniq -c | sort -rn ``` 3. **Audit it with every crawl** Serpel’s site audit raises a warning when robots.txt blocks an AI search crawler, and `serpel ai status` lists which AI crawlers your robots.txt allows from the last crawl. [AI visibility in Serpel](https://serpel.app/features/ai-visibility) also shows whether answers cite you. Allowing the search crawlers is only the first step towards being cited. Our guide to [SEO for ChatGPT](https://serpel.app/blog/how-to-rank-in-chatgpt) covers the rest, and the [llms.txt examples](https://serpel.app/blog/llms-txt-examples) explain why llms.txt is a different file with a different job. ## Frequently asked questions ### Should I block GPTBot? Block GPTBot if you do not want OpenAI to use your content for training. It has no effect on ChatGPT search, which uses OAI-SearchBot, and OpenAI says each setting is independent. If you want to be cited in ChatGPT answers, keep OAI-SearchBot allowed. ### Does blocking AI crawlers hurt my Google rankings? Blocking the AI-specific tokens does not. Google says Google-Extended does not affect inclusion or ranking in Google Search, and GPTBot and ClaudeBot are separate from search crawlers. Blocking Googlebot would remove you from Google Search, so never use it to opt out of AI. ### How do I block all AI crawlers? Add one group to robots.txt with a User-agent line for each crawler and Disallow: / underneath. The second example above does this for all 13 crawlers Serpel checks. Robots.txt is a request, not enforcement, so user-triggered fetchers may still visit and a hard block needs a firewall rule or authentication. ### Can AI crawlers ignore robots.txt? Yes. Under RFC 9309 robots.txt is voluntary and not a form of access authorisation. Vendors say their training and search crawlers honour it, but OpenAI, Perplexity and Meta document that user-initiated fetchers such as ChatGPT-User, Perplexity-User and Meta-ExternalFetcher may not. ### What is Google-Extended? Google-Extended is a robots.txt token that controls whether content Google crawls can be used for Gemini training and grounding. It has no separate user agent, and Google says it does not affect inclusion or ranking in Google Search. ## Sources - [OpenAI: Overview of OpenAI crawlers](https://developers.openai.com/api/docs/bots), accessed 2026-10-10 - [Anthropic: Does Anthropic crawl data from the web, and how can site owners block the crawler?](https://privacy.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler), accessed 2026-10-10 - [Perplexity: Perplexity crawlers](https://docs.perplexity.ai/docs/resources/perplexity-crawlers), accessed 2026-10-10 - [Google Search Central: Google’s common crawlers (Google-Extended)](https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers), accessed 2026-10-10 - [Google Search Central: AI features and your website](https://developers.google.com/search/docs/appearance/ai-features), accessed 2026-10-10 - [Apple: About Applebot](https://support.apple.com/en-us/119829), accessed 2026-10-10 - [Common Crawl: CCBot](https://commoncrawl.org/ccbot), accessed 2026-10-10 - [Meta: Web crawlers (Meta-ExternalAgent, Meta-ExternalFetcher)](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers), accessed 2026-10-10 - [Mistral AI: Robots (MistralAI-Index)](https://docs.mistral.ai/robots), accessed 2026-10-10 - [RFC 9309: Robots Exclusion Protocol](https://www.rfc-editor.org/rfc/rfc9309.html), accessed 2026-10-10 - [Vercel: The rise of the AI crawler](https://vercel.com/blog/the-rise-of-the-ai-crawler), accessed 2026-10-10 --- # AI SEO: how to use AI for SEO work and how to optimise for AI search URL: https://serpel.app/blog/ai-seo Updated: 2026-10-10 AI SEO means two different things: using AI to do SEO work faster, and optimising your site so AI search products can find and cite it. AI helps most with research, briefs, internal linking, audits and automation, but it invents facts, and Google’s spam policies treat mass-produced pages without added value as scaled content abuse however they are made. This guide covers both meanings, what to automate, what to check by hand and which categories of AI SEO tools exist. ## Key takeaways - AI SEO has two meanings: using AI as a tool for SEO work (research, briefs, internal links, audits, automation) and optimising for AI search products such as ChatGPT search and Google’s AI Overviews. They need different skills and different measurements. - AI is strong at drafting, clustering and summarising, and weak at facts. Google says generative models predict a likely sequence of words instead of retrieving facts, so verify every number, quote and claim and review metadata too. - Google’s spam policies define scaled content abuse as generating many pages mainly to manipulate rankings, however the content is created. Using generative AI to produce many pages without adding value is one of the listed examples. - Give your assistant real data instead of asking it to guess. APIs and MCP servers, including Serpel’s, let an agent read rankings, search data and audit results as tools. - Google says optimising for AI search is still SEO. Focus on crawlable, original, quotable content, and measure citations with prompt checks and the AI reports in Search Console and Bing Webmaster Tools. ## What does AI SEO mean? AI SEO is a label for two jobs that people often mix up. The first is **using AI for SEO**: asking a language model or an agent to help with keyword research, content briefs, internal linking, technical audits and reporting. The second is **optimising for AI search**: making your content easy for ChatGPT search, Google’s AI Overviews and AI Mode, Perplexity and Copilot to retrieve and cite. People use “AI SEO” for both. “AI for SEO” and “AI in SEO” lean towards the first, and “SEO for AI” towards the second. **The two meanings of AI SEO** | Question | Using AI for SEO | Optimising for AI search | | --- | --- | --- | | Goal | Do SEO work faster and with less manual effort | Be retrieved, quoted and cited in AI answers | | Typical tasks | Research, briefs, internal links, audits, metadata drafts, reporting, automation | Crawler access, server-rendered content, quotable passages, entity clarity | | Main risk | Wrong facts, generic pages and scaled content abuse | Chasing unproven tactics and buying guaranteed citations | | How you measure it | Time saved, error rate, quality of the shipped pages | Citation and mention rates, AI report impressions, AI crawler hits | | Go deeper | The sections below | [GEO vs SEO](https://serpel.app/blog/geo-vs-seo), [LLM SEO](https://serpel.app/blog/llm-seo) and [answer engine optimization](https://serpel.app/blog/answer-engine-optimization) | ## How can you use AI for SEO work? Use AI where a wrong answer is cheap to catch and a right one saves hours. The pattern that works in every case below is the same: give the model real data, let it propose, and have a person check before anything ships. ### Research and clustering A model is good at grouping hundreds of queries by intent, spotting gaps in a topic and summarising what the top pages cover. It is not a source of numbers. Never ask it for search volumes, difficulty scores or rankings, because it will produce plausible figures that are not measurements. Take those from a data provider or your own Search Console, and let the model work on the data you give it. ### Briefs and outlines Google says generative AI can be useful when you research a topic and to add structure to original content. A brief that lists the question, the audience, the sources to use and the facts to include is a good job for a model. The expertise, the original data and the examples must come from you, because a page that only restates what exists adds nothing for a reader. ### Internal linking Give the model a list of your URLs with titles and ask which pages should link to which, with suggested anchor text. Then verify that each suggested URL exists and returns status 200 before you add a link, because models invent paths. A crawl is the easiest way to get a trustworthy list of pages. With the Serpel CLI you can export it as JSON. ```bash serpel crawl pages --crawl <crawl-id> --limit 200 --json > pages.json ``` ### Technical audits and fixes An audit produces a long list of findings. An assistant can explain each one, group them by cause and propose a code change, which is where it saves the most time for developers. Our [SEO audit report example](https://serpel.app/blog/seo-audit-report-example) shows what such findings look like, including a link finding where 31 of 43 flagged targets turned out to be fine after a human re-check. ### Metadata and structured data drafts Titles, descriptions, alt text and JSON-LD are all fair game for a first draft. Google’s [guidance on generative AI content](https://developers.google.com/search/docs/fundamentals/using-gen-ai-content) says the review duty applies to metadata as well, such as title elements, meta descriptions, structured data and image alt text. Validate markup with a tool such as our free [schema validator](https://serpel.app/tools/schema-validator) before you publish it. ### Automation with APIs, agents and MCP The step beyond chat is to let an agent read your SEO data itself. The [Model Context Protocol](https://modelcontextprotocol.io/docs/getting-started/intro) (MCP) is an open-source standard for connecting AI applications to external systems. Anthropic created it and contributed it to the Agentic AI Foundation, a fund of the Linux Foundation, in December 2025. Google’s [Search Console API](https://developers.google.com/webmaster-tools/v1/how-tos/search_analytics) exposes the data of the Performance report in batches of up to 25,000 rows, and Bing has a similar Webmaster API, covered in our [Bing Webmaster Tools guide](https://serpel.app/blog/bing-webmaster-tools). Serpel’s remote [MCP server](https://serpel.app/developers/mcp) offers 13 tools that let Claude Code, Cursor, Codex and other agents read rankings, search data, crawl issues and recommendations and, after a `serpel scan`, relate findings to the routes in your code. Only the keyword research tool spends credits, and only within a budget you approve. Our guide to the [Google Search Console MCP](https://serpel.app/blog/google-search-console-mcp) shows the same idea for Google’s data. Serpel does not write articles for you. It supplies the data and findings your own assistant works from. ## Where does AI go wrong in SEO? Google puts the core problem plainly: generative models don’t retrieve facts, but predict a likely sequence of words, so outputs can contain inaccuracies, known as hallucinations, and it is critical to fact-check AI-generated content. The 2025 paper [Why Language Models Hallucinate](https://arxiv.org/abs/2509.04664) by Adam Tauman Kalai and co-authors argues that models guess when uncertain because training and evaluation procedures reward guessing over admitting uncertainty. In SEO that shows up in predictable ways. **AI tasks in SEO and what to check before you ship** | Task | How well AI fits | Check before you ship | | --- | --- | --- | | Keyword metrics | Poor. A model has no measurements | Take numbers from a data provider or Search Console, never from the model | | Statistics and quotes | Poor. It can invent both | Find the primary source yourself and link it, or leave the claim out | | Clustering and summarising | Good | Spot-check a sample of clusters against the real queries | | Briefs and outlines | Good | Add the original data and expertise only you have | | Internal link suggestions | Good, given a real URL list | Every URL exists and returns 200 | | Metadata and schema drafts | Fair | Length limits, accuracy and a validator run | | Full articles | Risky | Fact-check every claim, add original input and decide whether the page deserves to exist | ## What does Google say about AI-generated content? Google does not ban AI content. Its [guidance on using generative AI](https://developers.google.com/search/docs/fundamentals/using-gen-ai-content) says to review AI-generated content manually before publishing, and warns that generating many pages without adding value for users may violate its spam policy on scaled content abuse. The [spam policies](https://developers.google.com/search/docs/essentials/spam-policies) define that abuse as many pages generated for the primary purpose of manipulating search rankings and not helping users, and they apply no matter how the content is created. The examples Google lists include: - using generative AI tools or similar tools to generate many pages without adding value for users - scraping feeds or search results to generate many pages, including through automated transformations such as synonymizing or translating, with little value for users - stitching or combining content from different pages without adding value - creating many pages whose content makes little sense to a reader but contains search keywords The tool is not the test. The purpose and the value are. Google’s [helpful content guidance](https://developers.google.com/search/docs/fundamentals/creating-helpful-content) also asks whether the use of automation or AI is clear to visitors through disclosures, and says such disclosures are useful where someone might wonder how the content was created. If you publish AI-assisted pages, a short note on how they were made costs nothing. > **A checklist for AI-assisted pages:** Would this page exist if search engines did not? Does it contain something a competitor’s page does not? Has a person checked every claim and every link? Did you avoid creating one page for every minor variation of a query? If you cannot answer yes to all four, do not publish it. ## How do you optimise for AI search? This is the second meaning of AI SEO, and Google’s position is that it is still SEO. Its [optimisation guide](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide) says you don’t need special files, markup or rewriting for its AI features, and that unique, non-commodity content will likely matter more than any other suggestion in it. The practical list is short: 1. Let the crawlers in, and keep pages indexable. See [AI crawlers](https://serpel.app/blog/ai-crawlers). 2. Serve the main content as server-rendered text. See [JavaScript SEO](https://serpel.app/blog/javascript-seo). 3. Lead each section with the answer and support it with attributed facts. See [answer engine optimization](https://serpel.app/blog/answer-engine-optimization). 4. Cover the follow-up questions a reader would ask, on one useful page. 5. Measure citations over time instead of trusting a single answer. See [how to rank in ChatGPT](https://serpel.app/blog/how-to-rank-in-chatgpt). ## Which AI SEO tools exist? AI SEO tools fall into categories, and a good choice depends on the job, not on a league table. We do not rank products here. Google also advises caution with third-party tools that promise ranking success or claim to use internal Google metrics, because no third-party tool has access to its internal ranking or AI systems. **Categories of AI SEO tools** | Category | Good for | Watch out for | | --- | --- | --- | | General-purpose assistants | Drafting, clustering, summarising and explaining findings | No live SEO data by default, and confident mistakes | | SEO suites with AI features | AI summaries and writing help on top of keyword and audit data | Check which data is measured and which is generated | | Content optimisation and writing tools | Briefs, outlines and first drafts | Generic output and scaled content abuse if used for volume | | Technical audit tools | Finding crawl, indexing, speed and markup problems, with explanations | Heuristic findings need a human check | | AI visibility trackers | Checking whether AI answers cite or mention your domain | Answers vary, so a single check is a sample | | APIs and MCP servers | Giving your own agent real rankings, search data and audit results | Permissions and spending limits for paid calls | Before you adopt any of them, ask where the data comes from, whether the tool shows its sources, whether you can export the results and who reviews the output. Serpel sits in three of these categories: a technical audit that renders JavaScript, an AI visibility tracker for ChatGPT with web search and Google AI Overviews, and an API, CLI and MCP server. Compare it with others in our guide to the [best AI visibility tools](https://serpel.app/blog/best-ai-visibility-tools). ## A practical AI SEO workflow 1. **Ground the model in your data** Connect Search Console and Bing, run a crawl and give your assistant those results, through exports or an MCP server, instead of asking it to guess. 2. **Let AI propose, not publish** Ask for clusters, briefs, link suggestions and fix proposals. Treat the output as a draft that a person owns. 3. **Verify facts and add original input** Check every number, quote and URL against a primary source, and add the data, examples and expertise the model cannot supply. 4. **Ship in small batches** Publish a few pages or changes at a time, so you can see what worked and roll back what did not. 5. **Measure and re-crawl** Read the Search Console performance data, re-run your rank and AI visibility checks and crawl again to confirm that the technical fixes landed. ```bash serpel crawl compare --project <project-id> ``` ## Frequently asked questions ### What is AI SEO? AI SEO means two things. One is using AI to do SEO work, such as research, briefs, internal linking and audits. The other is optimising your site for AI search products such as ChatGPT search and Google’s AI Overviews, so they can find and cite your content. Google says the second is still SEO. ### Can you use AI for SEO without being penalised? Yes. Google does not ban AI-generated content. Its spam policy on scaled content abuse targets generating many pages mainly to manipulate rankings without adding value, however the content is created. Review the output, add original value and do not mass-produce thin pages. ### Which AI SEO tools are best? It depends on the job, and no honest list can rank them all. General assistants suit drafting, audit tools find technical problems, AI visibility trackers measure citations and APIs or MCP servers feed your own agent real data. Check where a tool’s data comes from and avoid any that promise guaranteed rankings or citations. ### Can AI do keyword research? It can cluster and group keywords you give it and suggest topics, but it should not supply search volumes or difficulty scores, because a language model has no measurements and will produce plausible numbers. Take metrics from a data provider or Search Console. ### Is AI search replacing SEO? There is no evidence of that. Google says optimising for its generative AI features is still SEO, and AI answers still depend on pages that are crawlable and indexed. SEO now includes measuring citations in AI answers as well as rankings. ## Sources - [Google Search Central: Google Search’s guidance on using generative AI content on your website](https://developers.google.com/search/docs/fundamentals/using-gen-ai-content), accessed 2026-10-10 - [Google Search Central: Spam policies for Google web search](https://developers.google.com/search/docs/essentials/spam-policies), accessed 2026-10-10 - [Google Search Central: Creating helpful, reliable, people-first content](https://developers.google.com/search/docs/fundamentals/creating-helpful-content), accessed 2026-10-10 - [Google Search Central: Optimizing your website for generative AI features on Google Search](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide), accessed 2026-10-10 - [Kalai et al., Why Language Models Hallucinate (arXiv 2509.04664)](https://arxiv.org/abs/2509.04664), accessed 2026-10-10 - [Model Context Protocol: What is MCP?](https://modelcontextprotocol.io/docs/getting-started/intro), accessed 2026-10-10 - [Linux Foundation: Agentic AI Foundation announcement](https://www.linuxfoundation.org/press/linux-foundation-announces-the-formation-of-the-agentic-ai-foundation), accessed 2026-10-10 - [Google for Developers: Query your Search Analytics data with the Search Console API](https://developers.google.com/webmaster-tools/v1/how-tos/search_analytics), accessed 2026-10-10 --- # Answer engine optimization (AEO): what it is and how to do it URL: https://serpel.app/blog/answer-engine-optimization Updated: 2026-10-10 Answer engine optimization (AEO) is the practice of writing and structuring content so that answer engines such as Google’s AI Overviews, ChatGPT search, Perplexity and Microsoft Copilot can find it, lift a clear answer from it and cite it. Google calls AEO and GEO terms you may see used, and says the work is still SEO. This guide explains how answer engines pick sources, which page patterns help and how to measure the result. ## Key takeaways - Answer engine optimization (AEO) means making content that answer engines can retrieve, lift an answer from and cite. It overlaps almost completely with GEO, and Google says optimising for its AI features is still SEO. - No provider publishes how answer engines rank sources. What is documented is access (indexed and snippet-eligible pages for Google, OAI-SearchBot for ChatGPT search), query rewriting and the fact that results are not guaranteed. - The patterns that help are plain: a question-style heading, the direct answer in the first one or two sentences, one idea per section, tables for comparisons, attributed numbers and a visible author and date. - Google says Search needs no special schema markup for its AI features, and it deprecated the FAQ rich result in May 2026. Use structured data that matches visible content, not as an AEO trick. - Measure AEO with repeated prompt checks, Google’s Generative AI performance report, Bing’s AI Performance report and AI crawler hits in your logs. ## What is answer engine optimization? Answer engine optimization (AEO) is the work of making your content the source an answer engine uses. An answer engine does not return a list of links first. It returns a written answer, usually with a few cited sources. Google’s AI Overviews and AI Mode, ChatGPT search, Perplexity and Microsoft Copilot all work like this. AEO asks one question of every page: can a system find a clear answer here, lift it out and credit the page? The idea is older than AI assistants. Google’s [featured snippets](https://developers.google.com/search/docs/appearance/featured-snippets) pull a short answer from a page to the top of the results, and Google says its systems choose them automatically: you cannot mark up a page to become one. AEO extends that thinking to every system that writes answers. AEO is one of several overlapping labels. Google’s [optimisation guide](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide) says “AEO” stands for answer engine optimization and “GEO” for generative engine optimization, that you may see both used, and that the work is still SEO. Our [GEO vs SEO](https://serpel.app/blog/geo-vs-seo) comparison explains how the terms relate, and the [generative engine optimization](https://serpel.app/blog/generative-engine-optimization) guide covers the research behind GEO. ## How do answer engines pick their sources? Nobody outside the vendors knows the full recipe, and a list of “ranking factors” stated as fact goes beyond what any vendor documents. These are the parts each one does publish. **What the vendors document about finding and citing pages** | Answer engine | How it finds pages | What you control | | --- | --- | --- | | Google AI Overviews and AI Mode | May use “query fan-out”, issuing several related searches. A page must be indexed and eligible to be shown in Search with a snippet to appear as a supporting link | Googlebot access in robots.txt, noindex and snippet controls such as nosnippet and max-snippet | | ChatGPT search | Typically rewrites a question into one or more targeted queries and sometimes partners with other search providers. OpenAI says it ranks results using multiple factors and that placement is not guaranteed | robots.txt rules for OAI-SearchBot, and allowing OpenAI’s published IP ranges at your host or CDN | | Perplexity | PerplexityBot is designed to surface and link websites in Perplexity search results | robots.txt rules for PerplexityBot. Perplexity-User generally ignores robots.txt because a person requested the fetch | | Microsoft Copilot and Bing | Bing describes grounding as the system that connects AI to current, authoritative information | Bing Webmaster Tools, where the AI Performance report counts citations in Copilot and Bing’s AI summaries | Three gates follow from this: be crawlable, be retrievable for the sub-questions behind a prompt, and be easy to quote once retrieved. The first is covered by [OpenAI’s crawler documentation](https://developers.openai.com/api/docs/bots), [Perplexity’s crawler page](https://docs.perplexity.ai/docs/resources/perplexity-crawlers) and our guide to [AI crawlers](https://serpel.app/blog/ai-crawlers). The rest is what the next section is about. ## Answer engine optimization: on-page patterns that help These patterns make a passage easy to understand and lift. They are not secret signals. Microsoft’s guidance for [AI search answers](https://about.ads.microsoft.com/en/blog/post/october-2025/optimizing-your-content-for-inclusion-in-ai-search-answers) recommends direct one- or two-sentence answers, modular sections and tables, while Google says there is no requirement to break your content into tiny pieces and no ideal length. The principle behind both is that a reader, human or machine, should find the answer fast. 1. **Put a question or a clear topic in the heading** Headings that state the question people ask, such as “How long does a crawl take?”, give a retrieved passage context. Avoid vague labels like “Learn more”. 2. **Answer in the first one or two sentences** Lead each section with the direct answer, then add the detail. A passage that makes sense on its own survives being quoted out of context. 3. **Keep one idea per section** Short sections with clear boundaries are easy to retrieve. Do not split one topic across many thin pages, because Google warns that producing a page for every query variation can breach its scaled content abuse policy. 4. **Use tables and lists where the content is a comparison or a procedure** Give every option the same columns, so a fact can be lifted without losing its label. Use numbered lists for steps and bullets for parallel items. 5. **Attach numbers to sources and dates** Write “108 seconds, measured on 10 Oct 2026” instead of “fast”. Name the source of any figure and never invent a statistic or a quote. 6. **Name entities consistently** Use the same name for your company, product and people everywhere, and say what each one is where it first appears. Consistency helps readers and systems match the page to the right thing. 7. **Show who wrote it and when it was updated** Google’s [guidance on helpful content](https://developers.google.com/search/docs/fundamentals/creating-helpful-content) asks whether it is clear who created a page and how. A visible author or organisation and an update date also help a reader judge whether to trust a cited page. 8. **Keep the answer in the HTML** Do not hide key answers in tabs, images or PDFs, and render them on the server. Google asks that important content is available as text, and Vercel’s [December 2024 analysis](https://vercel.com/blog/the-rise-of-the-ai-crawler) found that crawlers such as GPTBot, ClaudeBot and PerplexityBot did not render JavaScript. Our guide to [JavaScript SEO](https://serpel.app/blog/javascript-seo) explains how to check. ### A worked example: from vague to answer-first This is how one section changes. The facts in the second version come from Serpel’s own audit of serpel.app on 10 Oct 2026, described in our [SEO audit report example](https://serpel.app/blog/seo-audit-report-example). **The same section before and after** | Version | Heading | Text | | --- | --- | --- | | Vague | Speed | Crawl time is something many people ask about. It depends on many factors, so there is no single answer, but our tool is designed to be fast. | | Answer-first | How long does a site crawl take? | A crawl of a 52-page site took 108 seconds in our audit of serpel.app on 10 Oct 2026. Time grows with the number of pages, the speed of your server and the delay a polite crawler leaves between requests. Serpel waits at least 250 milliseconds between requests to your site. | ## Does structured data help answer engines? Less than the folklore suggests. Google says there is no special schema.org markup you need for its AI features, and its guide states that structured data is not required for generative AI search. Google still recommends structured data for rich results, as long as it matches the visible text. Microsoft’s guidance suggests adding JSON-LD to label content types such as products and events. One change deserves a warning. Google announced in its search updates log that the FAQ rich result would no longer appear in Google Search from 7 May 2026, and the documentation was removed in June 2026. Earlier, in 2023, [Google had limited FAQ rich results](https://developers.google.com/search/blog/2023/08/howto-faq-changes) to well-known government and health sites, and said there was no need to proactively remove existing markup. So do not add FAQPage markup in the hope of an answer-engine boost. Write a visible FAQ section because readers ask follow-up questions, and treat any markup as a description of what is on the page. You can test markup with our free [schema validator](https://serpel.app/tools/schema-validator). ## Which answer engine optimization tools do you need? Be wary of tools that promise results. Google’s guide says no third-party tool has access to its internal ranking or AI systems and advises checking any tool’s advice against official guidance. Tools are useful for measurement and for finding technical problems. They fall into five groups, with no ranking implied. **Types of answer engine optimization tools** | Type | What it does | Example in practice | | --- | --- | --- | | Prompt and citation trackers | Run a list of questions on AI products and record whether your domain is cited or mentioned | [Serpel’s AI visibility tracking](https://serpel.app/features/ai-visibility), and the tools compared in our [best AI visibility tools](https://serpel.app/blog/best-ai-visibility-tools) guide | | Search engine reports | Show impressions and citations in AI features from the engines themselves | Google Search Console’s Generative AI performance report, Bing Webmaster Tools’ AI Performance report | | Site crawlers and audits | Find blocked crawlers, noindex pages, thin pages and pages that need JavaScript | [Serpel’s site audit](https://serpel.app/features/site-audit) | | Log analysis | Show which AI crawlers fetch which pages, and with what status | Your server or CDN logs | | Markup validators | Check that structured data is valid | The free [schema validator](https://serpel.app/tools/schema-validator) | Serpel covers the first and third rows. It asks ChatGPT with web search and checks Google AI Overviews for your prompts, and each crawl tests your robots.txt against 13 AI crawlers and looks for an llms.txt file. It does not query Perplexity, Gemini or Claude, so use their own tools for those. ## How do you measure answer engine optimization? Define a small set of numbers before you edit anything, and track them over the same prompts. - **Citation rate:** the share of your checked prompts where an answer links to your domain. - **Mention rate:** the share where your name or domain appears in the text. A cited answer counts as mentioned too. - **Cited competitors:** the domains that are cited when you are not, which shows whom the engine prefers. - **Engine reports:** impressions from Search Console’s [Generative AI performance report](https://support.google.com/webmasters/answer/16984139?hl=en), and citations from Bing’s [AI Performance report](https://blogs.bing.com/webmaster/2026/2/Introducing-AI-Performance-in-Bing-Webmaster-Tools-Public-Preview/). - **Crawler hits:** requests from OAI-SearchBot, PerplexityBot and others in your logs, with their status codes. Answers vary from run to run, so one check proves little. Take a baseline, change one page, wait for crawlers to return (OpenAI says search systems can take about 24 hours to adjust to a robots.txt change) and run the same prompts again. The [GEO vs SEO](https://serpel.app/blog/geo-vs-seo) guide shows how to combine the four measurement sources. ```bash serpel ai add --project <project-id> --prompt "How long does a site crawl take?" serpel ai run --project <project-id> --wait serpel ai prompts --project <project-id> ``` ## Frequently asked questions ### What is answer engine optimization? Answer engine optimization (AEO) is the practice of structuring content so that answer engines such as Google’s AI Overviews, ChatGPT search, Perplexity and Copilot can find a clear answer in it and cite it. It overlaps almost completely with generative engine optimization (GEO), and Google says the work is still SEO. ### Is answer engine optimization different from SEO? It builds on SEO. The foundations are the same: crawlable, indexed, original content. AEO adds a focus on direct answers, quotable passages and measuring citations instead of only rankings. Google says optimising for its generative AI features is still SEO. ### Do I need schema markup or an llms.txt file for AEO? Not for Google. Google says there is no special schema.org markup to add for its AI features and that Google Search ignores llms.txt. Structured data is still useful for rich results when it matches visible content, and the FAQ rich result no longer appears in Google Search. ### Which are the best answer engine optimization tools? The useful ones measure and diagnose: AI visibility trackers that check prompts for citations, the AI reports in Search Console and Bing Webmaster Tools, site crawlers that find blocked pages and server logs. Treat any tool that promises guaranteed citations with caution, because no provider guarantees placement. ### How do I know if AEO is working? Track the same set of prompts over time and record how often your domain is cited or mentioned. Add the impressions from Google’s Generative AI performance report, the citations in Bing’s AI Performance report and AI crawler requests in your logs. Compare before and after each change. ## Sources - [Google Search Central: Optimizing your website for generative AI features on Google Search](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide), accessed 2026-10-10 - [Google Search Central: AI features and your website](https://developers.google.com/search/docs/appearance/ai-features), accessed 2026-10-10 - [Google Search Central: Featured snippets and your website](https://developers.google.com/search/docs/appearance/featured-snippets), accessed 2026-10-10 - [Google Search Central: Creating helpful, reliable, people-first content](https://developers.google.com/search/docs/fundamentals/creating-helpful-content), accessed 2026-10-10 - [Google Search Central blog: Changes to HowTo and FAQ rich results](https://developers.google.com/search/blog/2023/08/howto-faq-changes), accessed 2026-10-10 - [Google Search Central: Search updates, deprecating the FAQ rich result feature](https://developers.google.com/search/updates), accessed 2026-10-10 - [Search Console Help: Generative AI performance report](https://support.google.com/webmasters/answer/16984139?hl=en), accessed 2026-10-10 - [Microsoft Advertising: Optimizing your content for inclusion in AI search answers](https://about.ads.microsoft.com/en/blog/post/october-2025/optimizing-your-content-for-inclusion-in-ai-search-answers), accessed 2026-10-10 - [Bing Search Blog: Elevating the role of grounding on the AI web](https://blogs.bing.com/search/2026/2/Elevating-the-Role-of-Grounding-on-the-AI-Web/), accessed 2026-10-10 - [Bing Webmaster Blog: Introducing AI Performance in Bing Webmaster Tools Public Preview](https://blogs.bing.com/webmaster/2026/2/Introducing-AI-Performance-in-Bing-Webmaster-Tools-Public-Preview/), accessed 2026-10-10 - [OpenAI: Overview of OpenAI crawlers](https://developers.openai.com/api/docs/bots), accessed 2026-10-10 - [Perplexity: Perplexity crawlers](https://docs.perplexity.ai/docs/resources/perplexity-crawlers), accessed 2026-10-10 - [Vercel: The rise of the AI crawler](https://vercel.com/blog/the-rise-of-the-ai-crawler), accessed 2026-10-10 --- # Best AI visibility tools in 2026: what they track and what they cost URL: https://serpel.app/blog/best-ai-visibility-tools Updated: 2026-10-10 AI visibility tools check whether ChatGPT, Google’s AI Overviews and other AI answer engines mention or cite your brand for the questions your customers ask. This guide compares six of the best AI visibility tools on the platforms they cover and the pricing they publish, checked on 10 Oct 2026. Serpel is one of them, and its limits are listed next to everyone else’s. ## Key takeaways - AI visibility tools fall into three groups: SEO suites with an AI add-on (Semrush, Ahrefs), dedicated AI visibility platforms (Peec AI, Otterly.AI, Profound) and developer-first tools (Serpel). - Public entry prices differ widely: Ahrefs Brand Radar starts at €47 per month, Otterly.AI at €29, Peec AI at €85 and Semrush at €94.94 per domain. Profound sells by quote and offers a free trial. - Platform coverage is the biggest difference. Serpel covers only two surfaces, ChatGPT with web search and Google AI Overviews, while most of the other tools list four or more engines. - Search Console and Bing Webmaster Tools now include free AI reports for Google’s and Microsoft’s own AI features, but neither covers ChatGPT. - AI answers change between runs, so pick a tool that repeats checks and shows a trend instead of a single snapshot. > **Disclosure:** Serpel is our product. The other rows use only facts from each vendor’s public pages, checked on 10 Oct 2026. Prices and features change, so confirm them on the vendor’s site before you buy. ## What do AI visibility tools measure? An AI visibility tool takes a list of prompts, which are the questions your customers ask AI assistants, runs them on one or more AI platforms on a schedule and records what comes back. From the answers it works out whether your brand is mentioned, whether your pages are cited as sources, which competitors appear instead and which websites the AI relies on. Vendors use different names for the same ideas. Ahrefs [defines](https://ahrefs.com/brand-radar) mentions (your brand name appears in the answer text), citations (the answer links to your pages), AI share of voice and estimated impressions. Semrush [adds](https://www.semrush.com/kb/1493-ai-visibility-toolkit) an AI visibility score and brand sentiment. Whatever the names, the data is a sample of generated answers. Google [points out](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide) that no third-party tool has access to its internal ranking or AI systems. Answers also change from run to run. A July 2026 [survey of GEO research](https://arxiv.org/abs/2607.14035) reports that commercial audits found low source overlap and substantial run-to-run variability. For you that means a tool should repeat its checks and show a trend, not report one snapshot. ## Which kinds of AI visibility tools exist? - **SEO suites with an AI add-on.** Semrush and Ahrefs add AI tracking to their existing products. They suit teams that already work in those tools. - **Dedicated AI visibility platforms.** Peec AI, Otterly.AI and Profound are built around AI answer tracking and usually cover more engines. - **Developer-first tools.** Serpel exposes AI checks through a dashboard, CLI, API and MCP server, with pay-per-check pricing and a narrower scope. - **Free first-party reports.** Google Search Console and Bing Webmaster Tools now report how often your pages appear in their own AI features. ## Best AI visibility tools compared The table lists what each vendor says it tracks and the pricing on its own public page on 10 Oct 2026. Prices are shown in euros as the pages displayed them, vary by region, and may differ in tax treatment and billing period, so treat them as a guide to scale rather than a quote. **AI visibility tools compared, vendor pages checked on 10 Oct 2026** | Tool | What it tracks | Public pricing | Notes | | --- | --- | --- | --- | | [Semrush AI Visibility Toolkit](https://www.semrush.com/pricing/ai/) | ChatGPT, Google’s AI features, Gemini and Perplexity, with visibility score, share of voice and sentiment | Base plan €94.94 per month per domain with 25 custom prompts. Enterprise on request | Extra domains, users and prompts cost extra | | [Ahrefs Brand Radar](https://ahrefs.com/brand-radar) | ChatGPT, Perplexity, Gemini, Copilot, Google AI Overviews and AI Mode, plus Claude on custom prompts | Custom Prompts from €47 per month. AI Visibility Index from €179 per month | Usage is counted in checks. Paid Ahrefs plans include a small daily prompt quota | | [Peec AI](https://peec.ai/pricing) | Three models of your choice from ChatGPT, AI Mode, AI Overviews, Copilot, Gemini and Naver AI. Extra models are paid add-ons | Starter €85 per month (50 prompts), Pro €205 (150), Advanced €425 (350). Enterprise on request | Unlimited users, daily tracking, 15% off annual billing | | [Otterly.AI](https://otterly.ai/pricing) | ChatGPT, Google AI Overviews, Perplexity and Copilot. Claude, AI Mode and Gemini are paid add-ons | Lite €29 per month (15 prompts), Standard €189 (100), Premium €489 (400). Enterprise from 1,000 prompts | Free trial. API and MCP access from the Standard plan | | [Profound](https://www.tryprofound.com/pricing) | Trial: ChatGPT, Gemini and Google AI Overviews. Enterprise: up to nine answer engines, including Perplexity, AI Mode, Copilot, DeepSeek, Claude and Exa | Free trial with 50 prompts run daily for 7 days. Enterprise pricing on request | API and exports are listed for enterprise only | | Serpel | ChatGPT with web search and Google AI Overviews only | 2 credits (€0.02) per prompt on ChatGPT and 2 credits (€0.02) on AI Overviews. 100 start credits, then Starter at €19 or Pro at €49 per month | 10, 25 or 50 prompts per project depending on the plan. Dashboard, CLI and API on every plan | ## Where does each tool fit best? - **Semrush.** A natural fit if your team already works in Semrush. The Base plan includes Brand Performance analysis for one domain and 25 prompts, so several brands or markets need extra domains or an enterprise plan. - **Ahrefs Brand Radar.** Lists six platforms and two modes: Custom Prompts, where you choose the questions, and an AI Visibility Index modelled on more than 470 million prompts from Ahrefs’ keyword database. Ahrefs itself says the Index works best once your brand already appears in AI search. - **Peec AI.** Pricing follows prompts and models rather than seats, with unlimited users on every plan. It also lists a crawlability audit that shows which AI bots your robots.txt allows or blocks. - **Otterly.AI.** The lowest entry price in this list, which suits small teams that need a few prompts. Cost rises with prompts and with each extra engine you add. - **Profound.** Aimed at companies running marketing agents at scale. Its self-serve option is a seven-day trial and full pricing is by quote, so you cannot compare its price publicly. ## Where does Serpel fit, and where does it not? Serpel’s scope is narrow. It checks ChatGPT with web search and Google AI Overviews and nothing else today: no Perplexity, Gemini, Claude, Copilot or Google AI Mode tracking, and no sentiment scoring. If your customers mainly use those platforms, a dedicated platform covers more ground. What you get is a precise record of the two surfaces it covers. Add prompts to a project, run them on demand and see for each one whether ChatGPT’s answer cited your domain (linked a page of yours) or only mentioned it, together with the answer’s sources and the history over time. Serpel also shows which domains are cited most often in answers, with yours highlighted, whether Google showed an AI Overview for your tracked keywords and cited you, and, from your last site crawl, which AI crawlers your robots.txt allows and whether an llms.txt file is reachable. Serpel bills each check in credits. A credit is worth €0.01 at list price, so checking 25 prompts on both surfaces costs €1.00. The dashboard, CLI and API read the same data, so a CI job or a coding agent can run the checks and read the results. See [AI visibility in Serpel](https://serpel.app/features/ai-visibility) and the [pricing](https://serpel.app/pricing) page for the current numbers. The limits to know are 50 prompts per project on the largest plan and two AI surfaces. ## How do you choose an AI visibility tool? 1. **Start from your audience.** Track the AI products your customers actually use. Paying for nine engines is wasteful if two of them matter. 2. **Count prompts, markets and engines.** Cost grows with each. Peec AI says its pricing is based on tracked prompts and models, and Ahrefs counts one check per prompt, platform, location and update. 3. **Check how often it repeats.** Daily or weekly runs give you a trend. A single snapshot is noisy because answers vary between runs. 4. **Look at data access.** If you want results in dashboards, CI or an agent, check for an API, exports or MCP. Otterly.AI lists API and MCP access from its Standard plan, Profound lists API and exports for enterprise, and Serpel offers the dashboard, CLI and API on every plan. 5. **Compare pricing models.** Flat plans suit steady tracking. Per-check credits suit occasional audits and agent-driven workflows. 6. **Ask how answers are collected.** Which country and language does the tool use, is web search on, and how often is a prompt re-run? A reproducible method matters more than a polished dashboard. ## Can you track AI visibility for free? Partly. Two first-party reports are free. Search Console’s [Generative AI performance report](https://support.google.com/webmasters/answer/16984139?hl=en) shows impressions in AI Overviews and AI Mode by page, country and device, and Google says it rolled out to all sites worldwide on 31 Aug 2026. Bing Webmaster Tools’ AI Performance report [shows citation counts and cited pages](https://www.searchenginejournal.com/bing-webmaster-tools-adds-ai-citation-performance-data/566874/) for Copilot and AI summaries in Bing, in public preview. Neither covers ChatGPT. For ChatGPT you can test prompts by hand: turn on search, ask your questions, open the sources and note whether your domain appears. That works for a handful of prompts and stops scaling quickly, which is where paid tools earn their price. Our guide on [how to rank in ChatGPT](https://serpel.app/blog/how-to-rank-in-chatgpt) explains the manual check in detail, and the [generative engine optimization](https://serpel.app/blog/generative-engine-optimization) guide covers what to change once you have a baseline. ## Frequently asked questions ### What is an AI visibility tool? An AI visibility tool runs a list of prompts on AI platforms such as ChatGPT, Google’s AI Overviews and Perplexity, and records whether your brand is mentioned or your pages are cited. Most also show which sources the AI relies on and how you compare with competitors. They sample generated answers, so results vary between runs. ### How much do AI visibility tools cost? On the public price lists checked on 10 Oct 2026, entry plans started at €29 per month for 15 prompts at Otterly.AI, €47 per month for Ahrefs Brand Radar custom prompts, €85 per month at Peec AI and €94.94 per month per domain at Semrush. Profound sells by quote, and Serpel charges per check. Costs rise with the number of prompts, engines and markets. ### Do I need a paid tool to track ChatGPT and Google AI Overviews? Not to start. Search Console’s Generative AI performance report and Bing’s AI Performance report are free, and you can test ChatGPT prompts by hand. A paid tool becomes worthwhile when you track many prompts, need history, or want the data in a dashboard, API or coding agent. ### Which AI platforms should an AI visibility tool cover? The ones your customers use to ask questions. For most sites that means ChatGPT and Google’s AI features first, then Perplexity, Gemini, Copilot or Claude if your audience uses them. Check the vendor’s platform list before you pay, because coverage differs a lot between tools. ## Sources - [Semrush: AI Visibility Toolkit pricing](https://www.semrush.com/pricing/ai/), accessed 2026-10-10 - [Semrush Help: AI Visibility Toolkit](https://www.semrush.com/kb/1493-ai-visibility-toolkit), accessed 2026-10-10 - [Ahrefs: Brand Radar](https://ahrefs.com/brand-radar), accessed 2026-10-10 - [Peec AI: Pricing](https://peec.ai/pricing), accessed 2026-10-10 - [Otterly.AI: Pricing](https://otterly.ai/pricing), accessed 2026-10-10 - [Profound: Pricing](https://www.tryprofound.com/pricing), accessed 2026-10-10 - [Google Search Central: Optimizing your website for generative AI features on Google Search](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide), accessed 2026-10-10 - [Search Console Help: Generative AI performance report](https://support.google.com/webmasters/answer/16984139?hl=en), accessed 2026-10-10 - [Search Engine Journal: Bing Webmaster Tools adds AI citation performance data](https://www.searchenginejournal.com/bing-webmaster-tools-adds-ai-citation-performance-data/566874/), accessed 2026-10-10 - [Martinez, Optimizing Visibility in Generative Engines: A Critical Survey of Generative Engine Optimization (arXiv 2607.14035)](https://arxiv.org/abs/2607.14035), accessed 2026-10-10 --- # Bing Search Console: how to set up Bing Webmaster Tools URL: https://serpel.app/blog/bing-webmaster-tools Updated: 2026-10-10 Bing Search Console is the common name for Bing Webmaster Tools, Microsoft’s service for verifying your site, submitting sitemaps and URLs, and reading how your pages perform in Bing. Setup takes a few minutes: sign in, import your site from Google Search Console or verify it yourself, then submit a sitemap and switch on IndexNow. It matters beyond Bing, because Microsoft says Bing powers AI tools such as ChatGPT and Bing’s AI Performance report now counts citations in Copilot. ## Key takeaways - Bing Search Console is Bing Webmaster Tools. Sign in at bing.com/webmasters with a Microsoft, Google or Facebook account, then add your site by importing it from Google Search Console or by verifying it manually. - You can verify ownership with Domain Connect, a BingSiteAuth.xml file, a meta tag or a DNS record. An imported site is verified automatically, and Bing re-validates ownership through Google Search Console from time to time. - Submit an XML sitemap (up to 50,000 URLs and 50 MB uncompressed per file) and set up IndexNow, which notifies Bing and other participating engines when a URL is added, updated or deleted. Manual URL submission allows up to 10,000 URLs per domain per day, and your quota can be lower. - The Search Performance report shows clicks, impressions, click-through rate and average position with 16 months of data, and the AI Performance report (public preview) counts citations in Copilot and Bing’s AI summaries. - Bing matters for AI search: Bing’s guidelines say Copilot relies on the same crawling, indexing and ranking foundation as Bing Search, and OpenAI says ChatGPT search sometimes partners with other search providers. Neither publishes how much each source contributes. ## What is Bing Search Console? Bing Search Console is the name people use for **Bing Webmaster Tools** (BWT), the dashboard Microsoft gives site owners to manage how Bing sees their site. It is the Bing counterpart of Google Search Console. You use it to prove that you own a site, submit sitemaps and individual URLs, inspect how Bing sees a page, read search performance data and, since 2026, see where your content is cited in AI answers. Google Search Console and Bing Webmaster Tools overlap, but they are separate systems with separate data. Bing’s [Search Performance report](https://www.bing.com/webmasters/help/search-performance-c680da36) counts impressions and clicks across Bing sources such as web and chat, news, images and videos. If you only look at Google’s data, you are missing the traffic and the signals that come from Bing and from the products built on it. ## Why does Bing Webmaster Tools matter for ChatGPT search and Copilot? Because several AI answer products depend on Bing’s index, at least in part. These are the statements the companies have actually made. - **Microsoft on Copilot:** the [Bing Webmaster Guidelines](https://www.bing.com/webmasters/help/webmaster-guidelines-30fba23a) say Bing and Copilot search experiences rely on the same core crawling, indexing and ranking foundation as traditional search. Microsoft’s documentation adds that Copilot generates a [search query that it sends to the Bing search service](https://learn.microsoft.com/en-us/microsoft-365/copilot/manage-public-web-access). - **Microsoft on ChatGPT:** a Bing Webmaster Blog post says Bing [powers a wide range of AI and search experiences](https://blogs.bing.com/webmaster/2025/6/Start-Using-Bing-Webmaster-Tools-to-Improve-Your-Site-Visibility/), including AI tools like ChatGPT. - **OpenAI on ChatGPT search:** OpenAI’s [help page](https://help.openai.com/en/articles/9237897-searching-the-web-with-chatgpt) says ChatGPT search sometimes partners with other search providers and links Microsoft’s privacy statement. It does not say how much of the results come from Bing, and OpenAI tells site owners to allow OAI-SearchBot. So Bing is a cheap safeguard, not a proven lever. Setting up Bing Webmaster Tools takes minutes, and it gives you Microsoft’s own report on citations in Copilot. For the ChatGPT side of the story, read our guide to [SEO for ChatGPT](https://serpel.app/blog/how-to-rank-in-chatgpt). ## How do you set up Bing Webmaster Tools? 1. **Sign in** Open [Bing Webmaster Tools](https://www.bing.com/webmasters/about) and sign in with an existing Microsoft, Google or Facebook account. You can also create a new Microsoft account. 2. **Add your site** Choose one of two ways: import your sites from Google Search Console, or add the site manually by entering its URL. A property can be a whole domain, such as example.com, or a single branch, such as example.com/clothing/. 3. **Verify ownership** Imported sites are verified automatically. For a manually added site, choose one of the verification methods described below. 4. **Submit a sitemap and switch on IndexNow** Both are covered in the next sections. Do them straight away, because they decide how quickly Bing finds your pages. 5. **Wait for data** Bing says it usually takes about 48 hours before analytics data for a new site appears. ### How do you import a site from Google Search Console? Importing is the fastest route if your site is already verified in Google. Bing’s own [import guide](https://blogs.bing.com/webmaster/2019/9/Import-sites-from-Search-Console-to-Bing-Webmaster-Tools/) lists these steps: 1. Sign in to Bing Webmaster Tools, open the **My Sites** page and click **Import**. 2. Sign in with your Google Search Console account and click **Allow**. This gives Bing access to your list of verified sites and sitemaps. 3. Bing lists your verified sites with their sitemap counts and your role on each. Select the sites you want and click **Import**. Imported sites are verified automatically and their sitemaps are connected. An import brings in up to 100 sites at a time, and the limit of 1,000 sites per Bing Webmaster Tools account still applies. Ownership of an imported site is validated again from time to time through your Google account. If you revoke Bing’s access in Google, reconnect the account or verify the site another way. ### How do you verify a site in Bing Webmaster Tools manually? Bing’s [add and verify help page](https://www.bing.com/webmasters/help/add-and-verify-site-12184f8b) lists four methods. Pick the one you can do fastest. **Ways to verify ownership in Bing Webmaster Tools** | Method | What you do | Good to know | | --- | --- | --- | | DNS auto verification | Let Bing change your DNS records through Domain Connect | Only offered for DNS providers that support Domain Connect | | XML file | Download BingSiteAuth.xml and upload it to the root directory of your site | Easy if you can deploy files to the site root | | Meta tag | Paste the displayed meta tag into the head section of your site | The tag’s name is msvalidate.01 and its content is your verification code | | DNS record | Add the verification code to a DNS record, shown as a CNAME on the help page | Needs no change to the site’s files. Use the record type the verification screen shows you | ```html <meta name="msvalidate.01" content="<your-verification-code>" /> ``` Microsoft’s documentation does not say whether you may remove the file, tag or record after verification, so keep it in place. A note on DNS: the help page describes a CNAME record, while a 2025 Bing blog post mentions a TXT record, so follow whatever the verification screen displays for your site. ## How do you submit a sitemap to Bing? Open the **Sitemaps** tool, click **Submit sitemaps** at the top right and enter the sitemap URL. Bing accepts XML sitemaps and sitemap index files, RSS 2.0, Atom 0.3 and 1.0 feeds, and text files with one URL per line. The list shows each sitemap’s last processing date, status and the number of URLs Bing discovered. You can also point Bing to your sitemap in robots.txt. ```text Sitemap: https://www.example.com/sitemap.xml ``` Bing’s [guidance on sitemaps](https://blogs.bing.com/webmaster/2025/7/Keeping-Content-Discoverable-with-Sitemaps-in-AI-Powered-Search/) sets these limits: up to 50,000 URLs per sitemap file, up to 50,000 child sitemaps in an index file, and 50 MB per file uncompressed. Use an accurate lastmod value in ISO 8601 format with date and time. Bing says it ignores changefreq and priority. It fetches a sitemap when you submit it and then typically at least once a day. To check your own file before you submit it, use our free [sitemap checker](https://serpel.app/tools/sitemap-checker). ## What is IndexNow, and should you use it? [IndexNow](https://www.indexnow.org/documentation) is an open protocol that lets you tell search engines the moment a URL is added, updated or deleted, instead of waiting for them to recrawl. Bing’s [URL submission page](https://www.bing.com/webmasters/help/url-submission-62f2860b) describes it as automatically notifying Bing and participating search engines, and strongly recommends it. The IndexNow site lists Microsoft Bing, Naver, Seznam.cz, Yandex and Yep as supporters. Google is not on the list, so keep your sitemap too. 1. **Generate a key** A key has 8 to 128 characters and may contain a to z, A to Z, 0 to 9 and hyphens. 2. **Host the key file** Create a UTF-8 text file named after the key, such as `<your-key>.txt`, at the root of your site. It contains only the key. This is how engines confirm that you own the host. 3. **Submit changed URLs** Send one URL with a GET request, or up to 10,000 URLs with a POST request. You need to submit to only one endpoint, because participating engines share the URLs with each other. ```bash curl -X POST "https://api.indexnow.org/indexnow" \ -H "Content-Type: application/json; charset=utf-8" \ -d '{"host":"www.example.com","key":"<your-key>","urlList":["https://www.example.com/new-page","https://www.example.com/updated-page"]}' ``` 4. **Check the result** A 200 or 202 response means the submission was received, and a 202 means key validation is pending. A 403 means the key is invalid. Bing shows IndexNow activity under the IndexNow tab in Bing Webmaster Tools. The IndexNow FAQ is clear about the limits: submitting a URL is not a guarantee that it will be indexed, every URL counts towards crawl quota, and you should wait about five minutes before you resubmit the same URL. Several content management systems and Cloudflare’s Crawler Hints support IndexNow, as Bing’s [getting started guide](https://www.bing.com/indexnow/getstarted) notes. ## How do URL submission and URL inspection work? If you cannot use IndexNow, Bing Webmaster Tools lets you submit URLs by hand. You can submit up to 10,000 URLs per domain per day, the quota resets at midnight UTC, and the last 1,000 submitted URLs stay visible in your history. Bing says that quota is set on several parameters and may be lower for your site, and that these manual methods may be replaced by IndexNow in future. The [URL Inspection tool](https://www.bing.com/webmasters/help/url-inspection-55a30305) shows how Bing sees one page through three cards: an Index card (discovered, crawled and indexed, with the HTML and HTTP response and any errors), an SEO card (errors, warnings and notices with fixes) and a Markup card (Microdata, Microformats, Open Graph and JSON-LD found on the page). The Live URL tab sends a request through Bingbot to show what it sees right now, and you can request indexing from there, subject to your URL submission quota. ## What do the Search Performance and AI Performance reports show? **Search Performance** reports impressions, clicks, average click-through rate and average position. Keywords and pages are listed in tables, you can compare two periods, and you can export to CSV. Microsoft [extended the data to 16 months](https://blogs.bing.com/webmaster/2024/10/Bing-Webmaster-Tools-Extends-Search-Performance-Data-to-16-Months/) in October 2024. **AI Performance** launched in [public preview in February 2026](https://blogs.bing.com/webmaster/2026/2/Introducing-AI-Performance-in-Bing-Webmaster-Tools-Public-Preview/). It shows how your content appears in Microsoft Copilot, AI-generated summaries in Bing and select partner integrations, with total citations, average cited pages, a sample of grounding queries and page-level citation activity. Microsoft’s [help page](https://www.bing.com/webmasters/help/ai-performance-9f8e7d6c) is explicit about the limits: citations are not clicks or traffic, the data is sampled, and the report does not measure rankings, authority or importance. A June 2026 update added intents, topics, citation share and a compare view, also in preview. ## How do you get a Bing Webmaster API key? The Webmaster API gives programmatic access to the data in the dashboard. Microsoft’s [getting access guide](https://learn.microsoft.com/en-us/bingwebmaster/getting-access) offers two methods: OAuth 2.0, which Microsoft recommends, or an API key. To create a key, sign in, make sure at least one site is verified, click **Settings** at the top right, open **API Access**, accept the terms and click **API Key**, then **Generate API Key**. - **One key per user.** It works for all the sites you have verified. To rotate it, delete the old key and generate a new one. - **Keep it secret.** Anyone with the key can read the data of all your verified sites. - **Use the JSON endpoint.** Microsoft [retired the legacy SOAP and POX APIs on 31 Aug 2026](https://learn.microsoft.com/en-us/bingwebmaster/api-protocols). The JSON methods live under `https://ssl.bing.com/webmaster/api.svc/json/` and answer with the result wrapped in a `d` property. - **Traffic methods.** `GetQueryStats` and `GetPageStats` return clicks, impressions and average positions per query or page, and `GetUrlInfo` returns what Bing knows about one URL. ```bash curl "https://ssl.bing.com/webmaster/api.svc/json/GetUserSites?apikey=$BING_WEBMASTER_API_KEY" ``` ## How does Serpel use Bing Webmaster Tools data? Serpel reads your Bing Webmaster Tools data with that API key. You paste the key once, and Serpel lists your verified sites and matches each one to a project by domain. It then shows Bing clicks, impressions, click-through rate and average position per query and per page, next to Google’s numbers on the [Search data](https://serpel.app/features/search-data) page, for 7, 28 or 90 days. Reading the data costs no credits, and Serpel stores the key encrypted. Two limits are worth knowing. Serpel keeps no history of Bing data and reads it live, so the history you see is whatever Bing returns. And it shows Search Performance numbers only, not the AI Performance report. The same data is available in the CLI and to your coding agent through the `get_search_performance` tool of the [MCP server](https://serpel.app/developers/mcp). ```bash serpel search connect-bing --project <project-id> --api-key-file bing.key serpel search queries --project <project-id> --provider bing --dimension page --days 90 ``` ## Frequently asked questions ### Is Bing Search Console the same as Bing Webmaster Tools? Yes. Bing Search Console is a nickname for Bing Webmaster Tools, Microsoft’s dashboard for site owners. It plays the role Google Search Console plays for Google, with site verification, sitemaps, URL inspection and performance reports. ### How do I verify my site in Bing Webmaster Tools? Import it from Google Search Console and it is verified automatically, or add it manually and verify with Domain Connect, a BingSiteAuth.xml file, a meta tag or a DNS record. The verification screen shows the exact file, tag or record to use for your site. ### Does Bing Webmaster Tools help with ChatGPT search? Possibly. OpenAI says ChatGPT search sometimes partners with other search providers, and Microsoft says Bing powers AI tools such as ChatGPT. Neither company publishes how much of ChatGPT’s results come from Bing, so treat Bing indexing as a low-cost safeguard and also allow OAI-SearchBot. ### Do I still need a sitemap if I use IndexNow? Yes. IndexNow tells participating engines about changes as they happen, but it does not replace a sitemap. The IndexNow FAQ recommends using both, and Google is not among the engines that participate. ### Can I see Bing Webmaster Tools data in other tools? Yes. Create an API key under Settings, API Access and connect it to a tool that supports the Webmaster API. Serpel reads Bing clicks, impressions, click-through rate and position per query and page with that key and shows them next to Google Search Console data. ## Sources - [Bing Webmaster Tools help: Add and verify site](https://www.bing.com/webmasters/help/add-and-verify-site-12184f8b), accessed 2026-10-10 - [Bing Webmaster Blog: Import sites from Search Console to Bing Webmaster Tools](https://blogs.bing.com/webmaster/2019/9/Import-sites-from-Search-Console-to-Bing-Webmaster-Tools/), accessed 2026-10-10 - [Bing Webmaster Blog: Start using Bing Webmaster Tools to improve your site visibility](https://blogs.bing.com/webmaster/2025/6/Start-Using-Bing-Webmaster-Tools-to-Improve-Your-Site-Visibility/), accessed 2026-10-10 - [Bing Webmaster Blog: Keeping content discoverable with sitemaps in AI-powered search](https://blogs.bing.com/webmaster/2025/7/Keeping-Content-Discoverable-with-Sitemaps-in-AI-Powered-Search/), accessed 2026-10-10 - [IndexNow documentation](https://www.indexnow.org/documentation), accessed 2026-10-10 - [IndexNow FAQ](https://www.indexnow.org/faq), accessed 2026-10-10 - [Bing: How to add IndexNow to your website](https://www.bing.com/indexnow/getstarted), accessed 2026-10-10 - [Bing Webmaster Tools help: URL submission](https://www.bing.com/webmasters/help/url-submission-62f2860b), accessed 2026-10-10 - [Bing Webmaster Tools help: URL inspection](https://www.bing.com/webmasters/help/url-inspection-55a30305), accessed 2026-10-10 - [Bing Webmaster Tools help: Search performance](https://www.bing.com/webmasters/help/search-performance-c680da36), accessed 2026-10-10 - [Bing Webmaster Blog: Bing Webmaster Tools extends search performance data to 16 months](https://blogs.bing.com/webmaster/2024/10/Bing-Webmaster-Tools-Extends-Search-Performance-Data-to-16-Months/), accessed 2026-10-10 - [Bing Webmaster Blog: Introducing AI Performance in Bing Webmaster Tools Public Preview](https://blogs.bing.com/webmaster/2026/2/Introducing-AI-Performance-in-Bing-Webmaster-Tools-Public-Preview/), accessed 2026-10-10 - [Bing Webmaster Tools help: AI Performance](https://www.bing.com/webmasters/help/ai-performance-9f8e7d6c), accessed 2026-10-10 - [Microsoft Learn: Getting access to the Bing Webmaster Tools API](https://learn.microsoft.com/en-us/bingwebmaster/getting-access), accessed 2026-10-10 - [Microsoft Learn: Bing Webmaster API protocols](https://learn.microsoft.com/en-us/bingwebmaster/api-protocols), accessed 2026-10-10 - [Bing Webmaster Guidelines](https://www.bing.com/webmasters/help/webmaster-guidelines-30fba23a), accessed 2026-10-10 - [Microsoft Learn: Data, privacy and security for web search in Microsoft Copilot](https://learn.microsoft.com/en-us/microsoft-365/copilot/manage-public-web-access), accessed 2026-10-10 - [OpenAI Help Center: Searching the web with ChatGPT](https://help.openai.com/en/articles/9237897-searching-the-web-with-chatgpt), accessed 2026-10-10 --- # Core Web Vitals test: how to measure and fix LCP, INP and CLS URL: https://serpel.app/blog/core-web-vitals-test Updated: 2026-10-10 A Core Web Vitals test checks three metrics: Largest Contentful Paint for loading, Interaction to Next Paint for responsiveness and Cumulative Layout Shift for visual stability. A page passes when the 75th percentile of real-user data is 2.5 seconds or less for LCP, 200 milliseconds or less for INP and 0.1 or less for CLS. Judge a page by field data from Search Console or PageSpeed Insights, and use lab tools such as Lighthouse to find and fix the cause. ## Key takeaways - Core Web Vitals are LCP, INP and CLS. INP replaced First Input Delay on 12 March 2024. - The “good” values are 2.5 seconds for LCP, 200 milliseconds for INP and 0.1 for CLS, measured at the 75th percentile of real page loads. - Field data (CrUX) decides whether a page passes. Lab data (Lighthouse) explains why it fails and cannot measure INP. - Test with PageSpeed Insights, the Search Console report, Lighthouse, the CrUX API, your own users and a crawl, then fix by metric and wait for the 28-day field window to catch up. ## What does a Core Web Vitals test measure? The [Core Web Vitals](https://web.dev/articles/vitals) are three user-centred metrics. Each has a “good” threshold, and a tool should only call a page passing when it meets the target at the 75th percentile of page loads, segmented by mobile and desktop, for all three. **Core Web Vitals thresholds** | Metric | What it measures | Good | Needs improvement | Poor | | --- | --- | --- | --- | --- | | Largest Contentful Paint (LCP) | Loading: when the [largest image, text block or video](https://web.dev/articles/lcp) in the viewport is rendered | 2.5 s or less | 2.5 to 4.0 s | Over 4.0 s | | Interaction to Next Paint (INP) | Responsiveness: the [worst interaction latency](https://web.dev/articles/inp) across clicks, taps and key presses | 200 ms or less | 200 to 500 ms | Over 500 ms | | Cumulative Layout Shift (CLS) | Visual stability: the [largest burst](https://web.dev/articles/cls) of unexpected layout shifts | 0.1 or less | 0.1 to 0.25 | Over 0.25 | INP [became a Core Web Vital and replaced First Input Delay on 12 March 2024](https://web.dev/blog/inp-cwv-march-12). Google Search Console dropped FID the same day. Hovering, zooming and scrolling do not count towards INP. Google’s page experience documentation says Core Web Vitals are [used by its ranking systems](https://developers.google.com/search/docs/appearance/page-experience). It also says that good scores do not guarantee a top ranking and that there is more to page experience than the scores alone. Treat the metrics as a quality bar, not a shortcut. ## Lab data or field data: which one should you trust? Every Core Web Vitals tester falls into one of two groups. Field tools report what real visitors experienced. Lab tools load the page once in a controlled setup. [web.dev recommends](https://web.dev/articles/lab-and-field-data-differences) using field data to prioritise work and lab data to debug and to test changes before release. **Lab data compared with field data** | Aspect | Lab data | Field data | | --- | --- | --- | | Source | A Lighthouse run on one device, network and location | Real Chrome users, collected in the Chrome UX Report (CrUX) | | Metrics | LCP and CLS. Total Blocking Time stands in for INP | LCP, INP and CLS at the 75th percentile | | Best for | Debugging and checking a fix before it ships | Deciding which pages need work and whether you pass | | Availability | Any URL you can load | Only pages and origins with enough eligible traffic | The numbers differ for good reasons. A lab run usually starts with a cold cache, while real visits include cached loads. Field LCP stops tracking larger elements once the user interacts. INP depends on real interaction timing, which is why Lighthouse [cannot measure it](https://web.dev/articles/vitals) and suggests Total Blocking Time as a proxy. Field data has its own limits. A page must be publicly discoverable and have enough visitors, and the exact number is not disclosed. Only desktop Chrome and Chrome on Android contribute, as the [CrUX methodology](https://developer.chrome.com/docs/crux/methodology) explains. Chrome on iOS, Android WebView apps and other Chromium browsers do not. ## How do you test Core Web Vitals? ### PageSpeed Insights Enter a URL and [PageSpeed Insights](https://developers.google.com/speed/docs/insights/v5/about) shows field data from CrUX for the previous 28-day collection period at the 75th percentile, plus Lighthouse lab data, for mobile and desktop. If a URL has too little data it falls back to the whole origin. The page passes the assessment when the 75th percentile of all three metrics is good. When INP data is missing, LCP and CLS decide. ### Google Search Console The [Core Web Vitals report](https://support.google.com/webmasters/answer/9205520) uses CrUX field data, split by mobile and desktop. It groups URLs with a similar experience, and the group’s status is set by its worst metric. Use it to find which page types fail across the site. ### Lighthouse and Chrome DevTools [Lighthouse](https://developer.chrome.com/docs/lighthouse/overview) runs in the DevTools Lighthouse tab, from the command line, as a Node module or inside PageSpeed Insights. It is the right tool for finding the LCP element and long tasks and for confirming a fix straight after a deploy. ### The CrUX API The [CrUX API](https://developer.chrome.com/docs/crux/api) returns field data for a URL or an origin and needs an API key. Google documents a limit of 150 queries per minute per project. The 75th percentile is under `percentiles.p75` for each metric. ```bash curl -s --request POST "https://chromeuxreport.googleapis.com/v1/records:queryRecord?key=$CRUX_API_KEY" \ --header "Content-Type: application/json" \ --data '{"url":"https://example.com/","formFactor":"PHONE","metrics":["largest_contentful_paint","interaction_to_next_paint","cumulative_layout_shift"]}' ``` ### Your own users The [web-vitals library](https://github.com/GoogleChrome/web-vitals) measures the three metrics in your visitors’ browsers, so you see your own audience, including browsers CrUX ignores. ```javascript import { onCLS, onINP, onLCP } from 'web-vitals' function sendToAnalytics(metric) { const body = JSON.stringify({ name: metric.name, value: metric.value, rating: metric.rating, id: metric.id, }) navigator.sendBeacon('/analytics', body) } onCLS(sendToAnalytics) onINP(sendToAnalytics) onLCP(sendToAnalytics) ``` ### Serpel With every crawl, [Serpel’s site audit](https://serpel.app/features/site-audit) asks PageSpeed Insights to measure the home page and a small, fixed number of the most-linked indexable pages, on mobile and desktop. For each URL and device it stores the performance score, the field values for LCP, INP and CLS and the lab values for LCP, CLS, Total Blocking Time, First Contentful Paint and Speed Index. Findings follow a fixed order. Serpel rates the URL’s field value against the thresholds above, falls back to origin-level field data when the URL has none, and only then uses the lab value for LCP and CLS. It reports the worse of mobile and desktop. If PageSpeed Insights is unreachable or the quota is used up, the crawl still finishes and the audit notes that the vitals could not be measured. ```bash serpel crawl start --project <project-id> --wait serpel audit web-vitals <crawl-id> ``` > **A sample, not full coverage:** A crawl measures a handful of pages. Use the Search Console report for whole-site coverage and treat the crawl as a regression check after deploys. Run it from the terminal with the [Serpel CLI](https://serpel.app/developers/cli) or in CI. ## How do you read a PageSpeed Insights report? Start at the top. The field section answers whether real users have a good experience. It shows the page itself or, when the page has too little data, its whole origin, and it has separate results for mobile and desktop, which can differ a lot. Only these field values decide the Core Web Vitals assessment. The lab section below is a single Lighthouse run. Its performance score summarises lab metrics and does not decide the assessment. Use the diagnostics for the metric that fails in the field instead of chasing a score of 100. ## Which pages should you test first? Test one URL per page template instead of every URL: the home page, a listing, a detail page, a content page and any checkout or form. Pages of a template share code, so a fix usually carries over. Start with the templates that get the most traffic or that Search Console flags. Test mobile first, because Google [primarily indexes the mobile version](https://developers.google.com/search/docs/crawling-indexing/googlebot) of most sites. ## How do you fix a failing metric? Start with the metric that fails in field data, then use lab tools to find the cause. web.dev splits LCP into [four parts](https://web.dev/articles/optimize-lcp). As a guideline, time to first byte and resource load duration should each take about 40% of the total, while resource load delay and element render delay should each stay under 10%. **Causes and fixes per metric** | Metric | Likely cause | How to confirm | Fix | | --- | --- | --- | --- | | LCP | Slow first byte from redirects, uncached HTML or a slow backend | Time to first byte in the PageSpeed report | Deliver the HTML as fast as possible: cache it, avoid redirects and avoid unique URL parameters that miss the cache | | LCP | The hero image is found late or lazy-loaded | The LCP element in Lighthouse, a `loading="lazy"` attribute on it, an image set only in CSS | Put the image in the HTML, never lazy-load it and add `fetchpriority="high"` | | LCP | The main content is rendered by JavaScript | A large element render delay | Server-render or prerender the main content. See [JavaScript SEO](https://serpel.app/blog/javascript-seo) | | INP | Long tasks block the main thread | The Performance panel in DevTools | Keep event callbacks light and split work into separate tasks | | INP | Heavy rendering after an interaction | A large DOM, or styles changed and layout read in the same task | Reduce the DOM size and avoid forced synchronous layout | | CLS | Images and videos without dimensions | Elements that jump when media loads | Set `width` and `height` or a CSS `aspect-ratio` | | CLS | Ads, embeds or banners inserted late | Shifts in the DevTools layout shift regions | Reserve space with `min-height` or `aspect-ratio`, or load late content after a user action | | CLS | Web fonts that swap and reflow text | Text that jumps when the font loads | Preload critical fonts and use `font-display: optional` or a matched fallback with `size-adjust` | ```html <img src="/images/hero.webp" width="1200" height="630" alt="Product dashboard" fetchpriority="high"> ``` If you build with Next.js, the framework handles part of this for images and fonts. The [Next.js SEO guide](https://serpel.app/blog/nextjs-seo) shows the settings. ## How long until a fix shows up? Field data moves slowly because CrUX reports a 28-day window. After you fix a Search Console issue, [fix validation](https://support.google.com/webmasters/answer/9205520) monitors the URLs for 28 days. Confirm the change with a lab run on the same day, then watch the field numbers for a month before you call it done. ## Frequently asked questions ### Is Core Web Vitals a ranking factor? Google says Core Web Vitals are used by its ranking systems. It also says that good scores do not guarantee a top ranking and that great page experience involves more than the scores. Fix failing pages, but do not expect the numbers alone to move you up. ### What replaced FID in Core Web Vitals? Interaction to Next Paint replaced First Input Delay as a Core Web Vital on 12 March 2024. FID looked at the first interaction only, while INP observes all clicks, taps and key presses during a visit. ### Why do PageSpeed Insights, Lighthouse and Search Console show different numbers? Lighthouse is a single lab run, while Search Console and the field section of PageSpeed Insights use real-user CrUX data from the last 28 days. Search Console also groups similar URLs. Differences between lab and field data are expected. ### Why does PageSpeed Insights show no field data for my page? CrUX only includes pages that are publicly discoverable and have enough visitors. PageSpeed Insights then falls back to origin-level data, and shows no real-user data if the origin lacks enough as well. Use lab data and your own measurements for low-traffic pages. ### How can I test Core Web Vitals for a whole site? Use the Core Web Vitals report in Search Console, which groups URLs by similar experience, or query the CrUX API for the origin. A crawl that measures a sample of key pages on every run helps catch regressions after a deploy. ## Sources - [web.dev: Web Vitals](https://web.dev/articles/vitals), accessed 2026-10-10 - [web.dev: Largest Contentful Paint (LCP)](https://web.dev/articles/lcp), accessed 2026-10-10 - [web.dev: Interaction to Next Paint (INP)](https://web.dev/articles/inp), accessed 2026-10-10 - [web.dev: Cumulative Layout Shift (CLS)](https://web.dev/articles/cls), accessed 2026-10-10 - [web.dev: INP becomes a Core Web Vital on March 12](https://web.dev/blog/inp-cwv-march-12), accessed 2026-10-10 - [web.dev: Why lab and field data can be different](https://web.dev/articles/lab-and-field-data-differences), accessed 2026-10-10 - [web.dev: Optimize Largest Contentful Paint](https://web.dev/articles/optimize-lcp), accessed 2026-10-10 - [Google Search Central: Understanding page experience in Google Search results](https://developers.google.com/search/docs/appearance/page-experience), accessed 2026-10-10 - [Search Console Help: Core Web Vitals report](https://support.google.com/webmasters/answer/9205520), accessed 2026-10-10 - [Google PageSpeed Insights: About PageSpeed Insights](https://developers.google.com/speed/docs/insights/v5/about), accessed 2026-10-10 - [Chrome for Developers: CrUX methodology](https://developer.chrome.com/docs/crux/methodology), accessed 2026-10-10 - [Chrome for Developers: CrUX API](https://developer.chrome.com/docs/crux/api), accessed 2026-10-10 - [Google Search Central: Googlebot](https://developers.google.com/search/docs/crawling-indexing/googlebot), accessed 2026-10-10 - [Chrome for Developers: Lighthouse overview](https://developer.chrome.com/docs/lighthouse/overview), accessed 2026-10-10 - [GitHub: GoogleChrome/web-vitals](https://github.com/GoogleChrome/web-vitals), accessed 2026-10-10 --- # Generative engine optimization (GEO): what it is and what works URL: https://serpel.app/blog/generative-engine-optimization Updated: 2026-10-10 Generative engine optimization (GEO) is the practice of making your content easy for AI answer engines to retrieve, understand and cite. The original GEO research found that adding statistics, quotations and source citations raised visibility in its test setup, while keyword stuffing lowered it. Google says optimising for its AI features is still SEO, so the sensible approach is to build on good SEO and measure the result. ## Key takeaways - Generative engine optimization (GEO) means making pages that AI answer engines such as ChatGPT search, Google’s AI Overviews and Perplexity retrieve and cite. The term comes from a 2023 research paper accepted to KDD 2024. - In that paper, adding quotations, statistics and source citations raised a page’s visibility in generated answers by up to 40% in a lab setup, and keyword stuffing did worse than changing nothing. - Those results come from a simplified engine with the sources already in context. A 2026 survey of 45 studies finds no technique with a proven, lasting effect on organic discoverability, so read the numbers as an upper bound. - Google says you don’t need special files, markup or rewriting for its AI features. Crawlable, indexed, snippet-eligible and unique content is the foundation. - Measure GEO by repeating real prompts, reading Google’s Generative AI performance report and Bing’s AI Performance report, and checking your server logs for AI crawlers. ## What is generative engine optimization? Generative engine optimization (GEO) is the work of increasing how often AI answer engines use your content when they write an answer. A generative engine retrieves pages, passes passages to a language model and returns a written answer, usually with citations. ChatGPT search, Google’s AI Overviews and AI Mode, Perplexity and Microsoft Copilot all work this way. GEO aims to make your pages the ones that get retrieved, quoted and linked. Researchers at IIT Delhi, Princeton University and other institutions coined the term in the paper [GEO: Generative Engine Optimization](https://arxiv.org/abs/2311.09735). Its first author is Pranjal Aggarwal. It was first posted in November 2023 and accepted to KDD 2024. You will also see AEO (answer engine optimisation), LLMO and “AI SEO” used for the same idea, and Google’s own guide treats GEO and AEO as common names. One note on the phrase “geo seo”. It sometimes means geographic targeting for local search. This article is about generative engines, not locations. ## How do AI answers choose which sources to cite? No provider publishes the full recipe, but the documentation of OpenAI and Google describes the same broad pipeline. 1. **Query rewriting.** ChatGPT search [rewrites your question into one or more targeted queries](https://help.openai.com/en/articles/9237897-searching-the-web-with-chatgpt) and may send more specific ones after reading the first results. Google describes the same idea as [query fan-out](https://developers.google.com/search/docs/appearance/ai-features) for AI Overviews and AI Mode. 2. **Retrieval.** The system fetches candidate pages. Google says a page must be indexed and eligible to appear in Search with a snippet to show up as a supporting link. OpenAI says sites that block OAI-SearchBot [will not be shown in ChatGPT search answers](https://developers.openai.com/api/docs/bots). 3. **Ranking.** OpenAI says ChatGPT ranks search results using multiple factors and that placement is not guaranteed. Its help page does not list the factors. 4. **Answer writing and citation.** The model writes the answer from the retrieved passages and attaches links. OpenAI warns that citations can be incomplete, outdated or incorrect. That gives you three gates to pass: be crawlable, be retrievable for the sub-questions behind a prompt, and be worth quoting once retrieved. Results also vary. A July 2026 [survey of GEO research](https://arxiv.org/abs/2607.14035) reports that commercial audits found low source overlap and substantial run-to-run variability, so a single answer tells you very little. ## GEO vs SEO: what is different and what is the same? **GEO compared with classic SEO** | Aspect | Classic SEO | GEO | | --- | --- | --- | | Goal | Rank a URL for a keyword | Be retrieved, quoted or cited in a generated answer | | Unit of competition | A page in a list of results | A passage or fact inside one written answer | | Crawlers that matter | Googlebot and Bingbot | Those two plus OAI-SearchBot, Claude-SearchBot, PerplexityBot and others | | Content levers | Relevance, links, page experience and unique content | The same, plus quotable facts, statistics and cited sources | | Consistency | A position is stable enough to track daily | Answers vary by run, location and user, so you track rates over many checks | | What you measure | Rankings, impressions and clicks | Citation rate, mention rate, cited sources, AI referrals and crawler hits | The overlap is large. Google’s [AI optimisation guide](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide) says optimising for generative AI search is still SEO, and that you don’t need new machine-readable files, AI text files, markup or special rewriting for Google’s features. The practical difference is that you now care about several crawlers and about being quotable, not only about being rankable. ## What did the original GEO research measure? The authors built **GEO-bench**, a set of 10,000 queries from nine sources across 25 domains, with the text of the top five Google results attached to each query. Their test engine fetched those five sources and had gpt-3.5-turbo write a cited answer. For each query they rewrote one source with a GEO method and measured how much of the answer was credited to it, weighted by where the citation appears. That score is called position-adjusted word count. They tested nine methods. The table shows each method’s score from the paper’s Table 1. Higher is better, and the unmodified source scored 19.3. **Position-adjusted word count per GEO method, from Table 1 of the paper** | Method | What was changed | Score | | --- | --- | --- | | Quotation addition | Added relevant quotations from credible sources | 27.2 | | Statistics addition | Used quantitative statistics instead of qualitative discussion where possible | 25.2 | | Fluency optimisation | Improved the fluency of the text | 24.7 | | Cite sources | Added citations from credible sources | 24.6 | | Technical terms | Added technical terms where possible | 22.7 | | Easy-to-understand | Simplified the language | 22.0 | | Authoritative | Made the style more persuasive and authoritative | 21.3 | | Unique words | Added unique terms where possible | 20.5 | | Keyword stuffing | Added more keywords from the query, as in classic SEO | 17.7 | The best methods improved on the baseline by 41% on this metric, and keyword stuffing, the classic SEO tactic, scored below doing nothing. According to the authors, a more authoritative tone made no significant difference, which suggests the engines are fairly robust to persuasive phrasing. Two further results matter for smaller sites. In an experiment where all sources were optimised at once, citing sources raised the visibility of the fifth-ranked result by 115.1% on average, while the top-ranked result lost 30.3%. And the effect depended on the topic: adding statistics worked best for law and government, debate and opinion queries, while citing sources worked best for statement and fact queries. The authors also ran the methods against Perplexity.ai on a 200-query subset. Because Perplexity does not let you choose source URLs, they supplied the source text as uploaded files. Quotation addition was best on position-adjusted word count there, with a reported 22% improvement. Statistics addition reached 37% on the subjective impression metric, and keyword stuffing again fell below the baseline on position-adjusted word count. ## How far can you trust those results? They are useful, but narrower than the headline “up to 40%” suggests. The test engine was simplified, the sources were already in the model’s context, and the score measures a share of an answer, not clicks or traffic. The July 2026 survey of 45 GEO studies (a single-author preprint) concludes that the paper’s gains hold when a source is already in context, but do not show better organic discoverability or lasting traffic effects. It also finds that generic optimisation heuristics transfer poorly and that citation-oriented rewrites can hurt retrieval. > **Be careful with GEO promises:** Google advises being wary of third-party tools that promise ranking success or claim to use internal Google metrics, and OpenAI says placement in ChatGPT search is not guaranteed. Take the paper as evidence about what makes a passage quotable once it is retrieved, not as a guarantee of being retrieved. ## GEO checklist: what to do in practice 1. **Let the right crawlers in** Check robots.txt, your CDN and your firewall. Sites that block OAI-SearchBot are not shown in ChatGPT search answers, and Google’s AI features need Googlebot access. Our guide to [AI crawlers](https://serpel.app/blog/ai-crawlers) lists which ones to allow. 2. **Put the answer in the HTML** Serve key content as server-rendered text. In a [December 2024 analysis](https://vercel.com/blog/the-rise-of-the-ai-crawler) of Vercel’s network, GPTBot, ClaudeBot and PerplexityBot did not execute JavaScript. Google also asks that important content is available as text. 3. **Write what only you can write** Google says unique, useful, non-commodity content will likely influence your presence in AI search more than any other suggestion in its guide. Original data, tested procedures and expert detail also give a model something specific to quote. 4. **State facts with numbers and sources** The strongest methods in the GEO paper added statistics, quotations and citations to credible sources. Do it only where it is true: attribute every number, link the primary source and never invent a quote. 5. **Answer the sub-questions** Because ChatGPT and Google run several related searches per prompt, cover the follow-up questions a reader would ask, on one useful page or a few distinct ones. Google warns that producing a page for every query variation can breach its scaled content abuse policy. 6. **Earn genuine mentions** Google says its AI features can show what is being said about products and services across the web, and that seeking inauthentic mentions is less helpful than it seems. Documentation links, honest reviews and independent comparisons are the mentions worth earning. 7. **Skip what does not help** Keyword stuffing scored below the baseline in the paper. For Google’s AI features you don’t need an llms.txt file, special schema markup or content chunked for AI. Our [llms.txt examples](https://serpel.app/blog/llms-txt-examples) explain where the file does make sense. ## How do you measure GEO? Use four sources, because none of them is complete on its own. - **Prompt checks.** Pick 20 to 50 questions your customers really ask, run them on the AI products that matter and record whether your domain is cited (linked) or mentioned (named). Repeat the checks, because answers change between runs. - **Google’s Generative AI performance report.** Search Console [reports impressions](https://support.google.com/webmasters/answer/16984139?hl=en) in AI Overviews and AI Mode by page, country and device. The report is built around impressions, and Google says it rolled out to all sites worldwide on 31 Aug 2026. - **Bing’s AI Performance report.** Bing Webmaster Tools [shows how often your pages are cited](https://www.searchenginejournal.com/bing-webmaster-tools-adds-ai-citation-performance-data/566874/) in Copilot and AI summaries in Bing. It launched as a public preview in February 2026. - **Server logs.** Count requests from OAI-SearchBot, Claude-SearchBot, PerplexityBot and user-triggered fetchers such as ChatGPT-User. They show which pages AI systems actually read. Serpel automates the first one for ChatGPT with web search and Google AI Overviews, and also reports which AI crawlers your robots.txt allows. Add your questions, run the checks and read how often answers cite or mention your domain and which sources they cite most. [AI visibility in Serpel](https://serpel.app/features/ai-visibility) covers the details, and our comparison of the [best AI visibility tools](https://serpel.app/blog/best-ai-visibility-tools) shows the alternatives. ```bash serpel ai add --project <id> --prompt "What is the best SEO tool for developers?" --prompt "How do I check if ChatGPT cites my website?" serpel ai run --project <id> --wait serpel ai status --project <id> ``` ## Is generative engine optimization worth the effort? At the level of fundamentals, yes. Crawl access, server-rendered text, unique content and attributed facts cost little and help classic SEO too. Paying for a service that promises guaranteed citations is a different matter: both OpenAI and Google say outcomes cannot be guaranteed. Treat GEO as an experiment loop. Pick a set of prompts, record the baseline, change one thing, wait for the crawlers to return and measure again. That habit matters more than any single tactic, because the engines and their sources change often. ## Frequently asked questions ### What is generative engine optimization? Generative engine optimization (GEO) is the practice of making content easy for AI answer engines to retrieve, understand and cite. The term was introduced in a 2023 research paper by Pranjal Aggarwal and co-authors, accepted to KDD 2024. AEO and “AI SEO” are common names for the same work. ### Is GEO different from SEO? Mostly it builds on SEO. Google’s guidance says optimising for its generative AI features is still SEO, and the same foundations apply: crawlable, indexed, unique content. What changes is that you also care about AI crawlers such as OAI-SearchBot, about being quotable, and about measuring citations as well as rankings. ### Do generative engine optimization tactics really work? In the original paper, adding quotations, statistics and source citations improved visibility by up to 40% in a test setup, and keyword stuffing did not help. A 2026 survey of the research finds those gains hold once a page is already retrieved, but no technique has a proven lasting effect on being discovered. Use them as good writing practice and check the results with your own measurements. ### What are the best generative engine optimization tools? Tools fall into three groups: SEO suites with an AI add-on, dedicated AI visibility platforms and developer-first tools such as Serpel. They check whether AI answers cite or mention your brand for a list of prompts. Our guide to the best AI visibility tools compares what each one tracks and what it costs. ## Sources - [Aggarwal et al., GEO: Generative Engine Optimization (arXiv 2311.09735, KDD 2024)](https://arxiv.org/abs/2311.09735), accessed 2026-10-10 - [Martinez, Optimizing Visibility in Generative Engines: A Critical Survey of Generative Engine Optimization (arXiv 2607.14035)](https://arxiv.org/abs/2607.14035), accessed 2026-10-10 - [Google Search Central: Optimizing your website for generative AI features on Google Search](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide), accessed 2026-10-10 - [Google Search Central: AI features and your website](https://developers.google.com/search/docs/appearance/ai-features), accessed 2026-10-10 - [OpenAI Help Center: Searching the web with ChatGPT](https://help.openai.com/en/articles/9237897-searching-the-web-with-chatgpt), accessed 2026-10-10 - [OpenAI: Overview of OpenAI crawlers](https://developers.openai.com/api/docs/bots), accessed 2026-10-10 - [Vercel: The rise of the AI crawler](https://vercel.com/blog/the-rise-of-the-ai-crawler), accessed 2026-10-10 - [Search Console Help: Generative AI performance report](https://support.google.com/webmasters/answer/16984139?hl=en), accessed 2026-10-10 - [Search Engine Journal: Bing Webmaster Tools adds AI citation performance data](https://www.searchenginejournal.com/bing-webmaster-tools-adds-ai-citation-performance-data/566874/), accessed 2026-10-10 --- # GEO vs SEO: what is different, what stays the same and how to measure it URL: https://serpel.app/blog/geo-vs-seo Updated: 2026-10-10 GEO vs SEO comes down to the goal: SEO earns rankings and clicks on a results page, while generative engine optimization (GEO) earns mentions and citations inside an AI-written answer. The craft overlaps heavily, and Google says optimising for its generative AI features is still SEO. What changes is where results appear and how you measure them, so this guide compares the two and shows how to track GEO with prompts, citations, AI reports and server logs. ## Key takeaways - SEO aims to rank pages in a list of results. GEO (generative engine optimization) aims to get your content retrieved, quoted and cited inside an AI-generated answer. The term comes from a 2023 paper by Aggarwal and co-authors, accepted to KDD 2024. - Google says optimising for generative AI search is still SEO, and that its AI features need no special files, markup or rewriting. Bing’s guidelines describe GEO as a related practice focused on grounding and citations. - The foundations stay the same: crawlable, indexed, original, well-organised content. What changes is that more crawlers matter, passages must be quotable, and success is measured per answer instead of per position. - In the GEO paper, adding quotations, statistics and source citations raised visibility by up to 40% in a test setup. A 2026 survey of 45 studies found no technique with a proven, lasting effect on being discovered, so treat each tactic as a test. - Measure GEO with four sources: repeated prompt checks, Google’s Generative AI performance report, Bing’s AI Performance report and your server logs for AI crawlers. ## GEO vs SEO: what is the difference? **SEO** (search engine optimisation) is the work of making pages that search engines can crawl, index, understand and rank, so that people find them in a list of results. **GEO** (generative engine optimization) is the work of making content that AI answer engines can retrieve, understand and cite when they write an answer. The first competes for a position. The second competes for a place inside one written response. The term GEO comes from the research paper [GEO: Generative Engine Optimization](https://arxiv.org/abs/2311.09735) by Pranjal Aggarwal and co-authors from IIT Delhi and Princeton University, first posted in November 2023 and accepted to KDD 2024. You will also meet AEO (answer engine optimisation), LLMO and “AI SEO”. Google’s [optimisation guide](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide) says AEO and GEO are terms you may see used and that the work is still SEO. Our own deep dive is [generative engine optimization](https://serpel.app/blog/generative-engine-optimization), and we cover the answer-focused label in [answer engine optimization](https://serpel.app/blog/answer-engine-optimization). Two notes on wording. “GEO SEO” sometimes means geographic targeting for local search, and “GEO AI” can refer to geospatial topics. This article is about generative engines only. ## SEO vs AEO vs GEO: how do the three terms relate? Vendors do not agree on exact definitions, so use the labels as a rough map, not a taxonomy. In everyday use, SEO means ranking in classic results, AEO usually means writing content that answers a question directly enough to be lifted into a snippet, voice answer or AI response, and GEO means being cited in a generated answer. Microsoft’s [Bing Webmaster Guidelines](https://www.bing.com/webmasters/help/webmaster-guidelines-30fba23a) describe GEO as focusing on content eligibility for grounding and reference in AI responses, and add that GEO does not guarantee grounding or citations, just as SEO does not guarantee rankings or traffic. In practice the three share one foundation. A page that is crawlable, indexed and written for people is the starting point for all of them. The extra work sits on top: direct answers, quotable facts and measurement of citations. ## GEO vs SEO compared: goal, results, signals and measurement **GEO compared with classic SEO** | Aspect | SEO | GEO | | --- | --- | --- | | Goal | Rank a URL for a query and earn the click | Be retrieved, quoted or cited in a generated answer | | Where results show up | Search results pages: links, snippets and rich results | ChatGPT search, Google AI Overviews and AI Mode, Perplexity and Microsoft Copilot | | Unit of competition | A page in a ranked list | A passage or fact inside one written answer | | What decides success | Relevance, content quality, links, page experience and indexability | The same foundations, plus being retrievable for the sub-questions behind a prompt and quotable once retrieved. No provider publishes ranking factors | | Crawlers that matter | Googlebot and Bingbot | Those two plus OAI-SearchBot, PerplexityBot, Claude-SearchBot and others | | Stability | A position is stable enough to track daily | Answers vary by run, location and user, so you track rates over many checks | | How you measure | Rankings, impressions, clicks and click-through rate | Citation rate, mention rate, cited sources, AI report impressions, AI referrals and crawler hits | | Typical tools | Rank trackers, Search Console, site crawlers | AI visibility trackers, Search Console’s Generative AI report, Bing’s AI Performance report, server logs | ## What stays the same between GEO and SEO? More than most GEO advice admits. Google’s guide to [AI features and your website](https://developers.google.com/search/docs/appearance/ai-features) says a page must be indexed and eligible to be shown in Search with a snippet to appear as a supporting link, and that there are no additional technical requirements. Its best-practice list reads like an SEO checklist: - **Access:** let crawlers in through robots.txt and your CDN, and keep pages indexable. - **Content in text:** keep important content available as text, and keep structured data consistent with what is visible on the page. - **Page experience and internal links:** the same signals that help classic ranking. - **Original content:** Google says unique, non-commodity content will likely influence your presence in AI search more than any other suggestion in its guide. - **No shortcuts:** producing a page for every query variation can breach Google’s scaled content abuse policy. If these basics are broken, no GEO tactic will fix it. A [technical audit](https://serpel.app/features/site-audit) is therefore the first GEO step: it finds blocked crawlers, noindex pages, thin pages and pages that only render with JavaScript. Our [SEO audit report example](https://serpel.app/blog/seo-audit-report-example) shows what that looks like on a real site. ## What changes with GEO? 1. **More crawlers, with different jobs.** OpenAI separates [OAI-SearchBot, GPTBot and ChatGPT-User](https://developers.openai.com/api/docs/bots), and blocking the training crawler does not remove you from search. Our guide to [AI crawlers](https://serpel.app/blog/ai-crawlers) lists the other vendors. 2. **Queries are split.** Google says AI Overviews and AI Mode may use “query fan-out”, issuing several related searches across subtopics. Cover the follow-up questions a reader would ask, rather than one keyword. 3. **Passages matter more than pages.** An answer engine quotes a fact, not a page. A clear heading, a direct answer and an attributed number make a passage easy to lift. 4. **Results vary.** The same prompt can produce different answers and sources, so a single check is a sample. You track a rate across many checks. 5. **Bing matters more.** OpenAI says ChatGPT search sometimes partners with other search providers and links Microsoft’s privacy statement, and Bing Webmaster Tools now reports citations in Copilot. Verifying your site there is a cheap safeguard, covered in our [Bing Webmaster Tools guide](https://serpel.app/blog/bing-webmaster-tools). ## What does the GEO research show? The original paper built GEO-bench, a benchmark of 10,000 queries, and tested nine rewriting methods against a simulated generative engine. The three strongest were adding quotations, adding statistics and citing sources, which the authors report as relative improvements of 30 to 40% on their position-adjusted word count metric. Adding keywords, the classic SEO tactic, offered little to no improvement. The abstract’s headline is that GEO can boost visibility by up to 40% in generative engine responses. Read that as an upper bound for a test setup, not a promise. In the experiment the sources were already in the engine’s context, so the results show what makes a retrieved passage quotable, not what makes it retrieved. A [July 2026 survey of 45 GEO studies](https://arxiv.org/abs/2607.14035) concludes that no reviewed technique shows a stable, longitudinal, cross-platform causal effect on discoverability, and that topical relevance and context position are the most reproducible levers. Our [GEO guide](https://serpel.app/blog/generative-engine-optimization) walks through the paper’s numbers in detail. ## How do you measure GEO? SEO has one dominant data source in each search engine. GEO has several partial ones, and you need more than one. **Ways to measure GEO and what each one misses** | Method | What it tells you | Blind spot | | --- | --- | --- | | Prompt tracking | Whether an assistant cites or mentions your domain for the questions you care about | Answers vary, so a single run is a sample | | Citation checks | Which sources are cited instead of you, and how often | Shows who wins, not why | | Google’s Generative AI performance report | Impressions in AI Overviews and AI Mode by page, country and device | Impressions only, and Google features only | | Bing’s AI Performance report | Citations in Copilot and AI summaries in Bing, with cited pages and grounding queries | Public preview, and Microsoft says it does not measure rankings or importance | | Server logs | Which AI crawlers fetch which pages, and with what status code | Shows crawling, not whether a page was cited | ### Prompt tracking and citation checks Write 20 to 50 questions your customers really ask, run them on the assistants that matter and record whether your domain is cited (linked) or mentioned (named). Repeat on a schedule and compare. [Serpel’s AI visibility tracking](https://serpel.app/features/ai-visibility) does this for ChatGPT with web search and Google AI Overviews, and stores each answer’s sources so you can see who is cited instead of you. Our comparison of the [best AI visibility tools](https://serpel.app/blog/best-ai-visibility-tools) covers the alternatives. ```bash serpel ai add --project <project-id> --prompt "What is the difference between GEO and SEO?" serpel ai run --project <project-id> --wait serpel ai status --project <project-id> ``` ### Google’s and Bing’s AI reports Search Console’s [Generative AI performance report](https://support.google.com/webmasters/answer/16984139?hl=en) shows impressions in AI Overviews and AI Mode over time and by page, country, date, device and search type. Google says it rolled the insights out to all websites worldwide on 31 Aug 2026, and that traffic from AI features is also counted in the normal Performance report under the Web search type. Bing’s [AI Performance report](https://blogs.bing.com/webmaster/2026/2/Introducing-AI-Performance-in-Bing-Webmaster-Tools-Public-Preview/) launched in public preview in February 2026 and shows total citations, average cited pages and a sample of grounding queries for Microsoft Copilot, AI summaries in Bing and select partner integrations. ### Server logs for AI crawlers Your access log shows which AI systems read your pages. Count requests per crawler, then check the status codes, because a crawler that receives 403 or 404 cannot cite you. User agents can be faked, so match them against each vendor’s published IP ranges before you trust a number. ```bash grep -E -o "OAI-SearchBot|GPTBot|ChatGPT-User|PerplexityBot|Perplexity-User|ClaudeBot|Claude-SearchBot|Claude-User" access.log | sort | uniq -c | sort -rn ``` ## Should you prioritise GEO or SEO? SEO first, GEO as an experiment loop on top. Google says its AI features are rooted in its core Search ranking and quality systems, and OpenAI says ChatGPT search sometimes partners with other search providers, so being crawlable and indexed comes first either way. A sensible order: 1. **Fix access and indexing** Verify your site in Google Search Console and Bing Webmaster Tools, allow the crawlers you want and remove accidental noindex rules. 2. **Serve content as text** Put the main content, headings and tables in the initial HTML. Our guide to [JavaScript SEO](https://serpel.app/blog/javascript-seo) explains why. 3. **Write answers worth quoting** Lead each section with the answer, add attributed numbers and cover the follow-up questions. Do not invent statistics or quotes. 4. **Measure a baseline** Run your prompt set, read the Google and Bing AI reports and count AI crawler hits before you change anything. 5. **Change one thing at a time** Edit a page, wait for crawlers to return and measure again. Be wary of third-party tools that promise ranking success. Google says no third-party tool has access to its internal ranking or AI systems. ## Frequently asked questions ### What is the difference between GEO and SEO? SEO makes pages rank in a list of search results. GEO makes content get retrieved, quoted and cited inside an AI-generated answer. They share the same foundations of crawlable, indexed and original content, but GEO adds quotable passages, more crawlers and measurement per answer. ### Is GEO replacing SEO? There is no evidence for that. Google says optimising for its generative AI features is still SEO, and OpenAI says ChatGPT search sometimes partners with other search providers. AI answers still depend on pages being crawlable and indexed, so SEO remains the base. ### What is the difference between SEO, AEO and GEO? SEO ranks pages in results, AEO (answer engine optimisation) writes content that answers a question directly enough to be lifted into an answer, and GEO gets content cited in AI-generated responses. Vendors define the terms differently, and the work overlaps almost completely. ### What does GEO AI mean? In search marketing, GEO AI usually means generative engine optimization, the practice of being cited by AI answer engines such as ChatGPT search and Google’s AI Overviews. The same letters are also used for geographic and geospatial topics that have nothing to do with search. ### How do you measure GEO? Combine repeated prompt checks, Google’s Generative AI performance report, Bing’s AI Performance report and server logs for AI crawlers. No single source is complete, and answers vary between runs, so track citation and mention rates over time. ## Sources - [Aggarwal et al., GEO: Generative Engine Optimization (arXiv 2311.09735, KDD 2024)](https://arxiv.org/abs/2311.09735), accessed 2026-10-10 - [Martinez, Optimizing Visibility in Generative Engines: A Critical Survey of Generative Engine Optimization (arXiv 2607.14035)](https://arxiv.org/abs/2607.14035), accessed 2026-10-10 - [Google Search Central: Optimizing your website for generative AI features on Google Search](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide), accessed 2026-10-10 - [Google Search Central: AI features and your website](https://developers.google.com/search/docs/appearance/ai-features), accessed 2026-10-10 - [Search Console Help: Generative AI performance report](https://support.google.com/webmasters/answer/16984139?hl=en), accessed 2026-10-10 - [Bing Webmaster Blog: Introducing AI Performance in Bing Webmaster Tools Public Preview](https://blogs.bing.com/webmaster/2026/2/Introducing-AI-Performance-in-Bing-Webmaster-Tools-Public-Preview/), accessed 2026-10-10 - [Bing Webmaster Guidelines](https://www.bing.com/webmasters/help/webmaster-guidelines-30fba23a), accessed 2026-10-10 - [OpenAI: Overview of OpenAI crawlers](https://developers.openai.com/api/docs/bots), accessed 2026-10-10 --- # Google Search Console MCP: connect your agent to search data URL: https://serpel.app/blog/google-search-console-mcp Updated: 2026-10-10 A Google Search Console MCP server lets an AI agent read your clicks, impressions and queries through the Model Context Protocol instead of CSV exports. You can wrap the Search Console API yourself, run a community server or connect a hosted one. This guide walks through the hosted route with Serpel’s MCP server and its `get_search_performance` tool, including what the tool returns and which permissions it needs. ## Key takeaways - MCP is an open standard for connecting AI applications to external systems. A Search Console MCP server is a tool layer on top of the Search Console API. - You can build your own wrapper, run a community server with your own Google Cloud credentials, or use a hosted server where you only sign in. - Serpel’s `get_search_performance` tool returns totals, the previous period and the top queries or pages for the connected property. It is read-only and free to call. - Keep the Google scope read-only, approve tool calls, treat query text as untrusted, and revoke connected apps you no longer use. ## What is a Google Search Console MCP server? The [Model Context Protocol (MCP)](https://modelcontextprotocol.io/docs/getting-started/intro) is an open-source standard for connecting AI applications to external systems. Its documentation compares it to a USB-C port for AI applications: one standard way to plug in data sources, tools and workflows. An MCP server offers tools. According to the [MCP specification](https://modelcontextprotocol.io/specification/2025-11-25/server/tools), each tool has a name, a description and an input schema. The model can discover and call tools on its own, while the application should keep a human able to deny calls. A Google Search Console MCP server is that tool layer on top of the Search Console API. The API’s [Search Analytics query method](https://developers.google.com/webmaster-tools/v1/searchanalytics/query) returns clicks, impressions, click-through rate and average position, grouped by dimensions such as query, page or date. One request returns at most 25,000 rows, and Google notes that the API does not guarantee to return all data rows. It accepts either the read-only scope `https://www.googleapis.com/auth/webmasters.readonly` or the broader `webmasters` scope. Instead of exporting a report and pasting it into a chat, the agent calls a tool, gets rows back and reasons over them. That also lets it combine search data with other sources, such as crawl results or your code. ## What are your options for giving an agent Search Console data? **Ways to connect an AI agent to Search Console** | Option | Authentication | What you maintain | Watch out for | | --- | --- | --- | --- | | Your own wrapper around the Search Console API | OAuth client or service account | A service that exposes the API as MCP tools | You handle quotas, paging and the 25,000-row limit per request | | Community server [mcp-server-gsc](https://github.com/ahonn/mcp-server-gsc) | Service account JSON key | A Node.js process on your machine. Its README describes one `search_analytics` tool | The service account email must be added to your property, and the key file is a secret | | Community server [mcp-gsc](https://github.com/AminForou/mcp-gsc) | OAuth desktop client or service account | A Python process. Its README lists around 15 tools, including URL inspection and sitemap management | Some tools can change data. Destructive ones are off unless you enable them | | Hosted server, such as Serpel’s | Browser sign-in (OAuth) or an API token with scopes | Nothing to host. The Search Console connection lives in your Serpel project | Needs a Serpel account and a connected project | Neither community project lives in a Google GitHub organisation. Read the code, check how recently it was updated and pin a version before you give any server credentials. The rest of this guide covers the hosted route, because it needs no Google Cloud project. Choose by what else the agent needs. If it only reads Search Console, a community server with a service account is a small job, provided you are comfortable running and updating it. If the agent also needs crawl results, rankings or your code structure, a hosted server saves you from wiring several sources together. If all data must stay on your own infrastructure, build the wrapper yourself. ## How do you connect Search Console to Claude Code with Serpel’s MCP server? The server runs at `https://serpel.app/mcp`. You can follow the same steps in Cursor or Codex, using the [Claude Code guide](https://serpel.app/docs/mcp/claude-code) as the model, or read the [MCP overview for developers](https://serpel.app/developers/mcp) first. 1. **Connect Search Console to a project** Open the project’s Search data page and connect Google Search Console, or run the CLI command below. Google asks for read-only access. Serpel selects the property that matches the project’s domain when it can, and you pick one if it can’t. Bing Webmaster Tools works too and connects with an API key. ```bash serpel search connect-google --project <project-id> serpel search status --project <project-id> ``` 2. **Add the MCP server** Register Serpel as a remote HTTP server in Claude Code. ```bash claude mcp add --transport http serpel https://serpel.app/mcp ``` 3. **Sign in** Run `/mcp` inside Claude Code and follow the browser steps. You approve the access once on Serpel’s consent screen, and per the [Claude Code documentation](https://code.claude.com/docs/en/mcp) the client stores the token securely and refreshes it automatically. 4. **Find your project** Ask the agent to list your Serpel projects. It calls `list_projects` and gets a project ID for each one. 5. **Ask a question** Ask something like “Which landing pages brought the most clicks in the last 28 days?” The agent chooses `get_search_performance` and sends arguments like these. ```json { "projectId": "<project-id>", "days": 28, "dimension": "page", "provider": "google", "limit": 25 } ``` ## What does the get_search_performance tool return? The tool is one of 13 on the server and needs a connected Search Console or Bing property. Only `projectId` is required. **Input of get_search_performance** | Parameter | Values | Default | | --- | --- | --- | | `projectId` | A project ID from `list_projects` | Required | | `days` | 7, 28, 90 | 28 | | `dimension` | `query` or `page` | `query` | | `provider` | `google` (Search Console) or `bing` (Webmaster Tools) | `google` | | `limit` | 1 to 25 rows | 25 | A short summary sentence comes first, followed by compact JSON. The data contains the totals, the previous period of the same length, the percentage change in clicks and the top rows sorted by clicks. For queries, a `tracked` flag shows whether the query is already a rank-tracking keyword in the project. Google results use the web search type. ```json { "provider": "google", "site": "sc-domain:example.com", "period": { "startDate": "2026-09-08", "endDate": "2026-10-05", "days": 28 }, "totals": { "clicks": 1240, "impressions": 38900, "ctrPercent": 3.19, "averagePosition": 14.8 }, "previousPeriod": { "clicks": 1165, "impressions": 37200, "ctrPercent": 3.13, "averagePosition": 15.3 }, "clicksChangePercent": 6.4, "dimension": "page", "rows": [ { "page": "https://example.com/pricing", "clicks": 212, "impressions": 4300, "ctrPercent": 4.93, "averagePosition": 6.1, "tracked": false } ], "truncated": false, "cached": false, "fetchedAt": "2026-10-07T08:15:00.000Z" } ``` - **The period ends two days before today.** Search Console data arrives with a delay, and [Google’s guide to retrieving all data](https://developers.google.com/webmaster-tools/v1/how-tos/all-your-data) says it is typically available after two to three days. - **Results are cached for up to six hours.** The `cached` and `fetchedAt` fields tell the agent when the data was fetched. - **Without a connection the tool returns an error** that points to the project settings. Nothing is guessed. - **Reading is free.** The tool is read-only and costs no credits. ## What do the common errors mean? Tool errors come back as results with an error code and a short message, so the agent can explain the problem or retry. **Errors you may see from get_search_performance** | Error code | Meaning | What to do | | --- | --- | --- | | `validation_error` | The project has no Search Console or Bing connection with a selected property, or an input is invalid | Connect the property in the project’s search data settings and ask again | | `forbidden` | The token lacks a scope, or your role in the project is too low | Sign in again with OAuth, or create a token with `mcp:read` and `projects:read` | | `provider_error` | The data provider is unreachable, or the connection must be reconnected | Reconnect Search Console and retry later | | `rate_limited` | A provider or workspace limit was reached | Wait for the number of seconds in `retryAfterSeconds`, then retry | | `not_found` | The project does not exist or you cannot access it | Use an ID returned by `list_projects` | ## Which prompts work well with search data? Good prompts name a period and a decision. Each one below maps to what the tools actually return. - “Compare clicks over the last 7 days with the previous 7 days and tell me whether the change looks meaningful.” The agent uses `days: 7` and the change in clicks. - “List my top landing pages over 90 days and flag any with a click-through rate below the site average.” It compares each row’s `ctrPercent` with the totals. - “Which of my top queries are not tracked as keywords yet?” It reads the `tracked` flag and can then suggest additions. - “Show queries between positions 4 and 20 with at least 100 impressions and propose page updates.” This uses `find_keyword_opportunities` with the `minImpressions` input. - “Which high-traffic pages have crawl errors, and which source files render them?” It chains `get_search_performance`, `get_crawl_issues` and `get_codebase_overview`. To see how the connection works with search data in the dashboard, read about [Search Console and Bing data in Serpel](https://serpel.app/features/search-data). ## Is it safe to give an AI agent access to Search Console? It can be, if you control the permissions. These points follow the [MCP security guidance](https://modelcontextprotocol.io/docs/tutorials/security/security_best_practices) and the [authorization specification](https://modelcontextprotocol.io/specification/2025-11-25/basic/authorization). - **Keep Google access read-only.** Serpel asks Google for the `webmasters.readonly` scope. The tool is also marked read-only, but the MCP specification says clients must treat such annotations as untrusted unless the server is trusted. The read-only Google scope is the real protection. - **Use tokens bound to the server.** The authorization specification says an MCP server must only accept tokens issued for it. Serpel’s OAuth access tokens work only at the MCP server, and the REST API rejects them. - **Grant the minimum scopes.** With an API token, `mcp:read` and `projects:read` are enough for this tool. You can also limit a token to single projects. A token for Cursor looks like this: ```json { "mcpServers": { "serpel": { "url": "https://serpel.app/mcp", "headers": { "Authorization": "Bearer ${env:SERPEL_TOKEN}" } } } } ``` - **Treat local servers like any program you run.** The MCP security guidance warns that a local server runs with your privileges. Check the exact command before you approve it, and handle a service account key like a password. - **Treat query text as data, not instructions.** Search queries are typed by strangers. The Claude Code documentation warns that servers which fetch external content can expose you to prompt injection, so keep approval on for tools that can change anything. - **Revoke what you no longer use.** Connected apps are listed under Settings, API, and disconnecting one revokes its access immediately. The only paid tool, `research_keywords`, runs only if the workspace owner has allowed agents to spend credits. ## Frequently asked questions ### Can an AI agent read my Google Search Console data? Yes. An agent can call the Search Console API directly through code you write, or through an MCP server that wraps it. Either way, Google decides access through the scope you grant, and the read-only scope is enough for performance data. ### What is the difference between the Search Console API and an MCP server? The API is a set of HTTP endpoints you code against. An MCP server publishes selected API calls as named tools with input schemas, so any MCP-capable agent can discover and call them without custom integration code. ### Which Google permission does a Search Console MCP server need? For clicks, impressions, CTR and position, Google’s Search Analytics method accepts the read-only scope `webmasters.readonly` or the broader `webmasters` scope. Choose the read-only one unless a tool needs to change data, such as submitting sitemaps. ### Does get_search_performance cost credits? No. Reading Search Console data through the tool is free. The only paid MCP tool is `research_keywords`, which books credits and runs only when the workspace owner allows agent spending. ### How fresh is the search data an agent sees? The period ends two days before today because Search Console data arrives with a delay, and results are cached for up to six hours. The response includes `cached` and `fetchedAt` so an agent can say how current its answer is. ## Sources - [Model Context Protocol: What is MCP?](https://modelcontextprotocol.io/docs/getting-started/intro), accessed 2026-10-10 - [Model Context Protocol specification: Tools](https://modelcontextprotocol.io/specification/2025-11-25/server/tools), accessed 2026-10-10 - [Model Context Protocol specification: Authorization](https://modelcontextprotocol.io/specification/2025-11-25/basic/authorization), accessed 2026-10-10 - [Model Context Protocol: Security best practices](https://modelcontextprotocol.io/docs/tutorials/security/security_best_practices), accessed 2026-10-10 - [Google Search Console API: Search Analytics query](https://developers.google.com/webmaster-tools/v1/searchanalytics/query), accessed 2026-10-10 - [Google Search Console API: Get all of your data](https://developers.google.com/webmaster-tools/v1/how-tos/all-your-data), accessed 2026-10-10 - [Claude Code documentation: Connect Claude Code to tools via MCP](https://code.claude.com/docs/en/mcp), accessed 2026-10-10 - [GitHub: ahonn/mcp-server-gsc](https://github.com/ahonn/mcp-server-gsc), accessed 2026-10-10 - [GitHub: AminForou/mcp-gsc](https://github.com/AminForou/mcp-gsc), accessed 2026-10-10 --- # SEO for ChatGPT: how to rank in ChatGPT search and get cited URL: https://serpel.app/blog/how-to-rank-in-chatgpt Updated: 2026-10-10 SEO for ChatGPT starts with access: ChatGPT search answers do not show sites that block OpenAI’s OAI-SearchBot, and OpenAI says placement is not guaranteed. After that it comes down to clear, server-rendered pages that answer the questions people ask, with facts worth quoting. You can check your results by testing prompts, reading your server logs and tracking citations over time. ## Key takeaways - ChatGPT search answers do not show sites that block OAI-SearchBot, and OpenAI says placement is not guaranteed. Blocking GPTBot, the training crawler, does not affect search visibility. - ChatGPT rewrites each question into targeted queries and sends them to search partners. OpenAI’s help page links Microsoft’s privacy statement for this, but it publishes no ranking factors, so be sceptical of anyone who claims to know them. - Check that your host or CDN lets OpenAI’s published IP ranges through. Bot protection can block a crawler that robots.txt allows. - Serve key content as server-rendered text. In Vercel’s 2024 analysis, GPTBot fetched JavaScript files but did not execute them. - Check citations by testing real prompts in a clean session, counting OAI-SearchBot and ChatGPT-User requests in your logs, and tracking results over time. ## How does ChatGPT search choose which sites to cite? OpenAI’s [help page on web search](https://help.openai.com/en/articles/9237897-searching-the-web-with-chatgpt) describes the process in outline. ChatGPT may search automatically when a question benefits from current information, or you can start a search yourself. It typically rewrites the question into one or more targeted queries, sends them to search providers, and may send further, more specific queries after reading the first results. The answer then carries citations that link to the sources, and a Sources view lists the pages used. Web search is available on every ChatGPT plan, including to people who are not signed in. What OpenAI does not publish is how it weighs sources. The help page says only that ChatGPT ranks search results using multiple factors intended to help users find relevant, reliable information, and that placement is not guaranteed. A list of “ChatGPT ranking factors” stated as fact goes beyond what OpenAI documents. What you can control is what is documented: access, retrievability and quotability. That is the practical answer to how to rank on ChatGPT. ## Does ChatGPT search use Bing? OpenAI’s help page says ChatGPT search sometimes partners with other search providers and links the privacy statements of Microsoft and Shopify for how they handle queries. When search launched in late 2024, [InfoQ reported](https://www.infoq.com/news/2024/11/chatgpt-search-release/) that internet searches were performed by third-party engines including Bing. OpenAI does not say how much each source contributes, and its [crawler documentation](https://developers.openai.com/api/docs/bots) describes OAI-SearchBot as the bot that surfaces websites in ChatGPT’s search features. Treat Bing as a cheap safeguard, not a proven lever. Confirm that your site is indexed in Bing Webmaster Tools and consider [IndexNow](https://www.indexnow.org/), a simple ping that tells participating engines when content changes. Bing, Yandex, Naver, Seznam.cz and Yep are listed as supporters. ## Which OpenAI bots will you see in your logs? **OpenAI’s crawlers and what they mean for ChatGPT search** | Bot | Purpose | Affects ChatGPT search visibility? | robots.txt | | --- | --- | --- | --- | | OAI-SearchBot | Surfaces websites in ChatGPT’s search features | Yes. Sites that opt out are not shown in search answers | Applies. OpenAI recommends allowing it | | GPTBot | Crawls content that may be used to train OpenAI’s foundation models | No. It only signals training use | Applies. Disallow it to opt out of training | | ChatGPT-User | Visits a page when a user’s question or a GPT Action needs it | No. It is not used to decide whether content appears in search | OpenAI says the rules may not apply to user-initiated requests | OpenAI also lists OAI-AdsBot, which only visits pages submitted as ads on ChatGPT and whose data is not used to train models. Each bot publishes its own IP ranges, so you can tell the real crawler from a copy of its name. ## How to rank in ChatGPT: the steps that matter 1. **Allow OAI-SearchBot in robots.txt** OpenAI recommends allowing OAI-SearchBot and says sites that opt out are not shown in ChatGPT search answers, though they can still appear as navigational links. Each setting is independent, so this file allows search while asking OpenAI not to use your pages for training. OpenAI says its search systems can take about 24 hours to adjust after a change. ```text User-agent: OAI-SearchBot Allow: / User-agent: GPTBot Disallow: / ``` 2. **Let OpenAI’s IP ranges through** OpenAI says to confirm that your host or CDN allows traffic from its [published OAI-SearchBot IP addresses](https://openai.com/searchbot.json). Use the list for a firewall allow rule rather than trusting the user-agent string alone, because other bots can claim any name. ```bash node -e "fetch('https://openai.com/searchbot.json').then(r=>r.json()).then(d=>d.prefixes.forEach(p=>console.log(p.ipv4Prefix??p.ipv6Prefix)))" ``` 3. **Keep GPTBot and ChatGPT-User separate** Manage the three bots in the table above on their own terms. Opting out of training with GPTBot has no effect on search, and blocking ChatGPT-User with robots.txt may not stop user-initiated fetches. Our guide to [AI crawlers](https://serpel.app/blog/ai-crawlers) covers the other vendors’ bots. 4. **Serve the answer as server-rendered text** Vercel’s [December 2024 analysis](https://vercel.com/blog/the-rise-of-the-ai-crawler) found that GPTBot fetched JavaScript files but did not execute them, and Google [asks](https://developers.google.com/search/docs/appearance/ai-features) that important content is available as text. The study covered GPTBot, not OAI-SearchBot, so read it as a strong hint rather than proof. Put the main content, headings, tables and key facts in the initial HTML. 5. **Lead with the answer and support it with specifics** ChatGPT sends targeted queries, so a page that answers one clear question near the top gives it something to quote. Add attributed numbers, named sources and dates. The [GEO research](https://serpel.app/blog/generative-engine-optimization) found that adding statistics, quotations and citations raised visibility in its test engines. 6. **Cover the follow-up questions** ChatGPT may send more specific queries after the first results, so a page that also answers the obvious next questions has more chances to match. Write one thorough page rather than several thin ones. 7. **Stay indexable in Google and Bing** Keep pages free of noindex, return correct status codes and list them in your sitemap. Verify the site in Bing Webmaster Tools, because OpenAI links Microsoft’s privacy statement among its search partners. 8. **Show when the page was updated** OpenAI tells users to check when a cited source was published or updated, because results can be outdated. A visible update date helps readers decide whether to trust your page. ## What does a page that ChatGPT can quote look like? OpenAI has not published a template, so this is a practical reading of what it does say: ChatGPT sends targeted queries, quotes sources and tells users to verify them. Write for a reader who will check your claims. - **A heading that states the question or topic**, so a retrieved passage makes sense on its own. - **A direct answer in the first one or two sentences**, followed by the detail. - **Numbers with a source and a date**, for example “median response time 420 ms, measured on 10 Oct 2026”, instead of “fast”. - **A table for comparisons**, with the same columns for every option, so a fact can be lifted without losing its label. - **A named author or organisation and an update date**, so a reader can judge who is speaking and how current the page is. None of this is a ChatGPT-only trick. [Google’s guide](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide) asks for content that is well organised with clear headings and written for people, and the [GEO paper](https://arxiv.org/abs/2311.09735) found that attributed statistics and quotations raised visibility in its test engines. > **Test paraphrases:** Because ChatGPT rewrites questions before it searches, test the same need phrased three ways: a short keyword query, a full question and a comparison request. If your page appears for only one phrasing, it is probably missing a sub-question. ## What does not work for SEO for ChatGPT? - **Blocking GPTBot to hide from ChatGPT.** It only opts content out of training. Search visibility depends on OAI-SearchBot. - **Keyword stuffing.** In the [GEO paper](https://arxiv.org/abs/2311.09735) it scored below the unmodified baseline in the authors’ test engines. - **Buying guaranteed placement.** OpenAI says placement is not guaranteed, so a service that promises it cannot deliver it. - **Relying on llms.txt.** [Google says](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide) its Search ignores the file, and we found no statement from OpenAI that ChatGPT search reads it. Our [llms.txt examples](https://serpel.app/blog/llms-txt-examples) show where it still helps. ## How do you check if ChatGPT cites your website? Use three signals together, because each one has blind spots. 1. **Test prompts by hand.** Write 10 to 20 questions your customers ask. In ChatGPT, open View all tools and select Search (or type / and pick Search), ask the question and open Sources to see which domains were cited. Record whether you were cited (linked) or only mentioned (named without a link). OpenAI says saved memories can be used when it rewrites a search query and that your approximate location can shape local results, so test in a session without memory and note the country. 2. **Read your server logs.** Requests from OAI-SearchBot show that your pages are being crawled for ChatGPT search. Requests from ChatGPT-User may mean a user’s question led ChatGPT to fetch your page. Both should return status 200. 3. **Look for referral visits.** Check your analytics referral sources for chatgpt.com. Visits without referrer data appear as direct traffic, so treat the count as a minimum. ```bash grep -E "OAI-SearchBot|ChatGPT-User" access.log | awk '{print $9}' | sort | uniq -c | sort -rn ``` A citation shows that ChatGPT used your page for that answer. It does not show that the answer was right or that anyone clicked, so pair citation checks with referral and log data before you draw conclusions. A manual test is also a sample, not a measurement: answers vary between runs, and OpenAI warns that results and citations can be incomplete, outdated or incorrect. To see a trend, repeat the same prompts on a schedule and compare. [Serpel](https://serpel.app/features/ai-visibility) runs your prompts against ChatGPT with web search, records for each check whether the answer cited or mentioned your domain and which sources it used, and keeps the history so you can see changes after you edit a page. ```bash serpel ai add --project <id> --prompt "How do I rank in ChatGPT search?" serpel ai run --project <id> --wait serpel ai prompts --project <id> ``` ## Frequently asked questions ### Can you rank in ChatGPT? ChatGPT SEO works differently from Google SEO because there is no ranking list that you can climb. ChatGPT ranks sources internally using factors OpenAI does not publish, and placement is not guaranteed. What you control is access for OAI-SearchBot, indexable server-rendered pages and content worth quoting. ### Do I need to allow GPTBot to appear in ChatGPT search? No. GPTBot is OpenAI’s training crawler, and OAI-SearchBot handles search. OpenAI says each robots.txt setting is independent, so you can allow OAI-SearchBot and disallow GPTBot. Also confirm that your CDN does not block OpenAI’s published IP ranges. ### Does ChatGPT search use Bing? OpenAI says ChatGPT search sometimes partners with other search providers, and its help page links Microsoft’s privacy statement. Press reports at launch said third-party engines including Bing performed the searches. OpenAI does not publish how much each source contributes, so keep your site indexed in Bing as a low-cost safeguard. ### How long does it take after I change robots.txt? OpenAI says its systems can take about 24 hours to adjust to a robots.txt update for search results. Being cited also depends on a page being retrieved for a given question, so there is no fixed timeline for a first citation. ## Sources - [OpenAI Help Center: Searching the web with ChatGPT](https://help.openai.com/en/articles/9237897-searching-the-web-with-chatgpt), accessed 2026-10-10 - [OpenAI: Overview of OpenAI crawlers](https://developers.openai.com/api/docs/bots), accessed 2026-10-10 - [InfoQ: ChatGPT search release (5 Nov 2024)](https://www.infoq.com/news/2024/11/chatgpt-search-release/), accessed 2026-10-10 - [OpenAI: Published OAI-SearchBot IP ranges (JSON)](https://openai.com/searchbot.json), accessed 2026-10-10 - [IndexNow](https://www.indexnow.org/), accessed 2026-10-10 - [Vercel: The rise of the AI crawler](https://vercel.com/blog/the-rise-of-the-ai-crawler), accessed 2026-10-10 - [Google Search Central: AI features and your website](https://developers.google.com/search/docs/appearance/ai-features), accessed 2026-10-10 - [Google Search Central: Optimizing your website for generative AI features on Google Search](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide), accessed 2026-10-10 - [Aggarwal et al., GEO: Generative Engine Optimization (arXiv 2311.09735)](https://arxiv.org/abs/2311.09735), accessed 2026-10-10 --- # JavaScript SEO: how Google crawls, renders and indexes JavaScript URL: https://serpel.app/blog/javascript-seo Updated: 2026-10-10 JavaScript SEO is the work of making sure search engines can crawl, render and index content that your scripts create. Google renders JavaScript with a headless Chromium after it crawls a page, so client-side content can be indexed, just later than content that is already in the HTML. Other crawlers, including the main AI crawlers, may not run scripts at all, so the content, links and metadata that matter belong in the HTML you ship. ## Key takeaways - Google queues every page that returns a 200 status for rendering and indexes the rendered HTML, but a page can wait in the render queue for longer than a few seconds. - Crawlers only follow links that are `<a>` elements with an `href`, and they only see content that loads without scrolling, clicking or blocked scripts. - Server-side rendering, static rendering or hydration give every crawler the same HTML. Google calls dynamic rendering a workaround, not a recommended solution. - Compare the raw HTML with the rendered DOM and confirm the result with the URL Inspection tool in Search Console. ## How does Google crawl, render and index JavaScript? Google describes [three phases for JavaScript web apps](https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics): crawling, rendering and indexing. Googlebot fetches a URL after it checks `robots.txt`. Every page that returns a `200` status code is queued for rendering, and once Google’s resources allow, a headless Chromium renders the page and runs the JavaScript. Google then uses the rendered HTML to index the page and parses it again for links to crawl. Older guides talk about “two waves of indexing”. Google’s current documentation describes the three phases above and a render queue instead. A page may stay in that queue for a few seconds, but it can take longer. Client-side content can therefore be indexed, just later than content that is already in the HTML. Four rules follow from how that pipeline works: - **Blocked files are not rendered.** If `robots.txt` blocks a script or the page, Google does not run that JavaScript. - **noindex can end the process early.** When Google sees `noindex` in the HTML it may skip rendering, so a script that removes the tag later may never run. - **Only real links are discovered.** Google can only find links that are `<a>` elements with an `href` attribute. - **HTML metadata wins.** JavaScript can set the title, description and canonical, but HTML is preferred, and a canonical set by script must match the one in the HTML. ## Which rendering strategy is best for JavaScript SEO? The rendering strategy decides what the first response contains. Google copes with all of them, but many other crawlers do not. The table follows the definitions in [web.dev’s guide to rendering on the web](https://web.dev/articles/rendering-on-the-web). **Rendering strategies compared for crawlers** | Strategy | HTML in the first response | Risk for crawlers | Good fit | | --- | --- | --- | --- | | Client-side rendering (CSR) | An empty shell and script tags | High: content exists only after rendering, and a crawler that does not render sees nothing | Logged-in apps and dashboards | | Server-side rendering (SSR) | Complete HTML for every request | Low: the cost is server time and a slower first byte | Personalised or fast-changing pages | | Static rendering (SSG) | Complete HTML built once at build time | Lowest: changing content needs a rebuild | Docs, blogs and marketing pages | | Incremental static regeneration (ISR) | Prebuilt HTML that the server refreshes in the background, as [Next.js describes it](https://nextjs.org/docs/app/getting-started/caching) | Low: content can be briefly stale | Large catalogues of mostly stable pages | | Dynamic rendering | Rendered HTML for bots, a client-side app for users | Medium: two code paths to keep in sync | Avoid, see below | Google says dynamic rendering [was a workaround and not a long-term solution](https://developers.google.com/search/docs/crawling-indexing/javascript/dynamic-rendering) and recommends server-side rendering, static rendering or hydration. Hydration adds client scripts to server-rendered HTML. It keeps the HTML complete for crawlers, but web.dev notes it can have a significant negative impact on Total Blocking Time and Interaction to Next Paint. For a framework-specific walkthrough, see our guide to [Next.js SEO](https://serpel.app/blog/nextjs-seo). The choice matters beyond Google. [Vercel’s analysis from December 2024](https://vercel.com/blog/the-rise-of-the-ai-crawler) found that none of the major AI crawlers it examined rendered JavaScript, among them OpenAI’s, Anthropic’s and Perplexity’s, while Googlebot and Applebot did. Crawler behaviour changes, so treat that as a dated snapshot and check your own server logs. The safe rule is to ship the content, links and metadata that matter in the initial HTML. ## What are the most common JavaScript SEO problems? Most failures come from a handful of patterns. Each row names the symptom and the fix. **Common JavaScript SEO failures** | Problem | What goes wrong | Fix | | --- | --- | --- | | Empty HTML shell | The delivered HTML holds a root `<div>` and scripts. Text and links appear only after rendering. | Server-render or prerender indexable routes and check them with `curl`. | | Links without `href` | Buttons, `onclick` handlers and framework attributes navigate in a browser. Google [can’t reliably extract URLs](https://developers.google.com/search/docs/crawling-indexing/links-crawlable) from `<a>` elements without `href`. | Use `<a href="/path">` for every internal link. | | Fragment URLs | Routes such as `#/products` can’t be resolved reliably by Googlebot. | Use the History API with real paths. | | Soft 404s in single-page apps | Unknown routes render a “not found” view but the server answers `200`. | Redirect to a URL that returns 404 or add `noindex` to error views. See [soft 404](https://serpel.app/blog/soft-404). | | Lazy-loaded content | Google does not scroll or click, so content that loads only on interaction never appears. | Load content when it enters the viewport and give infinite scroll [paginated URLs](https://developers.google.com/search/docs/crawling-indexing/javascript/lazy-loading). | | Blocked or oversized files | `robots.txt` blocks scripts, or a bundle is large. Googlebot fetches each resource separately and [reads the first 2 MB of it](https://developers.google.com/search/docs/crawling-indexing/googlebot). | Allow script and CSS paths. Split large bundles. | | Metadata changed by script | Title, canonical or robots differ between the HTML and the rendered DOM. | Set them in the HTML. Never add `noindex` to the HTML if you want the page indexed. | | Structured data from scripts | Google can read JSON-LD that JavaScript generates, but crawlers that do not render never see it. | Put JSON-LD in the HTML where you can and test the rendered result with the Rich Results Test or by pasting the rendered HTML into the free [schema validator](https://serpel.app/tools/schema-validator). | | Stale cached scripts | Google’s renderer may ignore caching headers and run an old script. | Put a content hash in file names so a new release gets a new URL. | ## How do you test what Google actually sees? Test in three layers: the HTML your server sends, the DOM after rendering, and Google’s own view. A mismatch between the first two is the most common finding. 1. **Look at the delivered HTML** Fetch the page without a browser and count the headings and links that are already there. If the counts are zero, everything depends on rendering. ```bash curl -s https://example.com/pricing -o raw.html grep -o "<h1" raw.html | wc -l grep -o "<a [^>]*href=" raw.html | wc -l ``` 2. **Render it in a real browser** Save this small [Playwright](https://playwright.dev/docs/api/class-page) script as `renderPage.mjs`. It prints the full HTML after the scripts have run, including the doctype. ```javascript import { chromium } from 'playwright' const browser = await chromium.launch() const page = await browser.newPage() await page.goto(process.argv[2], { waitUntil: 'networkidle' }) process.stdout.write(await page.content()) await browser.close() ``` 3. **Compare the two** Run the same counts on the rendered file. Large differences in headings, links or words show what a non-rendering crawler misses. ```bash node renderPage.mjs https://example.com/pricing > rendered.html grep -o "<h1" rendered.html | wc -l grep -o "<a [^>]*href=" rendered.html | wc -l ``` 4. **Ask Google** The URL Inspection tool shows loaded resources, JavaScript console output and exceptions, and the rendered DOM, as described in [Google’s debugging guide](https://developers.google.com/search/docs/crawling-indexing/javascript/fix-search-javascript). Run a live test and open “View tested page” to see how Google [renders the URL](https://support.google.com/webmasters/answer/7440203). The Rich Results Test can also confirm that the rendered HTML contains your content. Read the comparison like an auditor. These differences matter most: - **Headings and text:** a missing H1 or a large gap in word count means the main content depends on rendering. - **Links:** internal links that appear only in the rendered file will not be followed by a crawler that does not render. - **Title, description and canonical:** values that change after rendering tell different crawlers different stories. - **Robots directives:** a `noindex` or `nofollow` that appears or disappears after rendering is a bug that belongs in the HTML. ## How does Serpel’s crawler render JavaScript pages? A JavaScript SEO audit with [Serpel’s site audit](https://serpel.app/features/site-audit) starts from the HTML the server delivers, then decides per page whether to render it. The project setting has three modes: “Automatic for empty pages” (the default), “Always render” and “Never render”. In automatic mode a page counts as a likely JavaScript shell when its delivered HTML has fewer than 50 words and shows a sign of a framework: a root container such as `#__next`, `#__nuxt`, `#root` or `#app`, Angular’s `<app-root>`, a `<noscript>` message about JavaScript, or several scripts with no links, no H1 and almost no text. Pages that qualify are rendered in headless Chromium, using a mobile or desktop viewport that matches the crawl device. The renderer waits for the network to go idle and falls back to the `load` event if that times out. A crawl renders at most a fixed number of pages and reports when it reaches the limit. After rendering, the title, headings, links and the other checks run on the rendered page, and the crawl compares it with the raw HTML. - **Content only via JavaScript:** the rendered page has at least 50 words and the raw HTML has less than half as many. - **Links only via JavaScript:** at least 3 internal links exist only after rendering, and the raw HTML holds fewer than half of the rendered links. - **Metadata changed by JavaScript:** title, description or H1 differ, or the canonical or robots directive differs between the two versions. - **Render problems:** the page could not be rendered, or the browser console reported JavaScript errors. - **Rendering recommended:** the page looks like an empty shell but was not rendered, because rendering is off, unavailable, over the limit or failed. Serpel reports this instead of false “missing title” or “missing H1” errors. ```bash serpel projects update <project-id> --render-mode always serpel crawl start --project <project-id> --wait serpel audit issues --project <project-id> --category rendering ``` > **A check, not a copy of Googlebot:** Serpel uses its own Chromium, so it can’t reproduce Google’s render queue or its resource limits. Use it to find pages that depend on JavaScript, then confirm the important ones with URL Inspection. ## What should you do first? 1. Fetch the raw HTML of your main templates and check that the H1, text and navigation links are present. 2. Replace click handlers that navigate with real `<a href>` links. 3. Return real status codes: `404` or `410` for missing content, `301` for moved content. 4. Server-render or prerender everything you want indexed. 5. Make sure `robots.txt` does not block your scripts and stylesheets. 6. Crawl the site with rendering on and review the pages where raw and rendered HTML differ. ## Frequently asked questions ### Does Google crawl and render JavaScript? Yes. Googlebot queues pages that return a 200 status for rendering, a headless Chromium runs the JavaScript, and Google indexes the rendered HTML. Rendering can wait in a queue, and blocked resources or a `noindex` tag can stop it. ### Is JavaScript bad for SEO? Not by itself. Problems start when content, links or metadata exist only after rendering that a crawler does not perform or that fails. Server-render the essentials and use JavaScript for interactivity on top. ### Do AI crawlers run JavaScript? In Vercel’s December 2024 analysis, none of the major AI crawlers it examined rendered JavaScript. Behaviour can change, so keep key content in the HTML. Our [AI crawlers guide](https://serpel.app/blog/ai-crawlers) covers which bots exist and how to control them. ### Is dynamic rendering still recommended? No. Google describes dynamic rendering as a workaround that adds complexity and resource costs, and recommends server-side rendering, static rendering or hydration instead. ### How do I check whether Google can see my JavaScript content? Open the URL Inspection tool in Search Console, run a live test and read the rendered DOM and console messages under “View tested page”. Compare it with the HTML your server sends, for example with `curl`. ## Sources - [Google Search Central: Understand JavaScript SEO basics](https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics), accessed 2026-10-10 - [Google Search Central: Fix Search-related JavaScript problems](https://developers.google.com/search/docs/crawling-indexing/javascript/fix-search-javascript), accessed 2026-10-10 - [Google Search Central: Make your links crawlable](https://developers.google.com/search/docs/crawling-indexing/links-crawlable), accessed 2026-10-10 - [Google Search Central: Fix lazy-loaded content](https://developers.google.com/search/docs/crawling-indexing/javascript/lazy-loading), accessed 2026-10-10 - [Google Search Central: Dynamic rendering as a workaround](https://developers.google.com/search/docs/crawling-indexing/javascript/dynamic-rendering), accessed 2026-10-10 - [Google Search Central: Googlebot](https://developers.google.com/search/docs/crawling-indexing/googlebot), accessed 2026-10-10 - [Search Console Help: Page indexing report](https://support.google.com/webmasters/answer/7440203), accessed 2026-10-10 - [web.dev: Rendering on the web](https://web.dev/articles/rendering-on-the-web), accessed 2026-10-10 - [Vercel: The rise of the AI crawler (17 December 2024)](https://vercel.com/blog/the-rise-of-the-ai-crawler), accessed 2026-10-10 - [Next.js documentation: Caching and Incremental Static Regeneration](https://nextjs.org/docs/app/getting-started/caching), accessed 2026-10-10 - [Playwright documentation: Page](https://playwright.dev/docs/api/class-page), accessed 2026-10-10 --- # LLM SEO: how to get found and cited by AI assistants URL: https://serpel.app/blog/llm-seo Updated: 2026-10-10 LLM SEO is the practice of making your site easy for assistants built on large language models, such as ChatGPT, Claude, Gemini and Perplexity, to find, read and cite. It works through two paths: models learn from training data collected before release, and assistants with search fetch live pages to ground their answers. Crawler access, server-rendered content and quotable facts decide the second path, and this guide explains both, the honest status of llms.txt and how to measure the result. ## Key takeaways - LLM SEO has two paths. Training data is fixed when a model is built, so you can only influence future versions. Live retrieval, where an assistant searches or fetches pages at answer time, is the path you can influence now. - Vendors separate their bots by purpose. OpenAI uses GPTBot for training, OAI-SearchBot for search and ChatGPT-User for user requests. Anthropic uses ClaudeBot, Claude-SearchBot and Claude-User. Perplexity uses PerplexityBot and Perplexity-User. Google-Extended is a robots.txt token, not a crawler. - Allow the search bots and decide on the training bots separately. User-triggered fetchers may ignore robots.txt, so use each vendor’s published IP ranges to tell real requests from fakes. - Serve key content in the HTML. Vercel’s 2024 analysis found that major AI crawlers fetched JavaScript files but did not execute them. - llms.txt is a proposal. Google says Google Search ignores it, and we found no statement from OpenAI, Anthropic or Perplexity that their crawlers read it. Treat it as optional. ## What is LLM SEO? LLM SEO is search engine optimisation for the assistants that sit on top of large language models (LLMs). When someone asks ChatGPT, Claude, Gemini or Perplexity a question, the answer comes from the model’s training, from pages the assistant retrieves while it answers, or from both. LLM SEO is the work of making sure your site is in the second group, readable by the bots that retrieve it, and worth quoting. You will also hear it called “SEO for AI”, AI SEO, GEO and AEO. The labels overlap, and Google’s [optimisation guide](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide) says optimising for generative AI search is still SEO. Our [GEO vs SEO](https://serpel.app/blog/geo-vs-seo) comparison explains how the terms relate, and the [AI SEO](https://serpel.app/blog/ai-seo) guide covers using AI as a tool for SEO work. This article is about being found and cited by LLM assistants. ## How do LLM assistants find and cite sources? An LLM on its own is not a search engine. It generates text from patterns it learned in training. To cite a page, the product around the model has to retrieve that page at answer time. The vendors describe this in their documentation. - **ChatGPT:** OpenAI says ChatGPT [may search the web automatically](https://help.openai.com/en/articles/9237897-searching-the-web-with-chatgpt) when a question benefits from current information, typically rewrites the question into targeted queries and may attach citations. It also warns that results and citations can be incomplete, outdated or incorrect. - **Claude:** Anthropic’s [web search tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool) lets Claude decide when to search, run the searches and answer with cited sources. Claude answers directly for stable knowledge and searches for current or changing information. The page does not say which index it searches. - **Gemini:** Google’s [grounding with Google Search](https://ai.google.dev/gemini-api/docs/google-search) lets the model decide whether a search helps, generate queries, run them and answer with citations to the sources. - **Perplexity:** its [crawler documentation](https://docs.perplexity.ai/docs/resources/perplexity-crawlers) says PerplexityBot is designed to surface and link websites in Perplexity search results, and that Perplexity-User visits a page to answer a user’s question and includes a link. ### Training data versus live retrieval **The two ways an LLM assistant can know about your site** | Path | How it works | What you can do | When it takes effect | | --- | --- | --- | --- | | Training data | Crawlers collect web content that may be used to train future models. What a released model learned is fixed | Allow or disallow training bots such as GPTBot and ClaudeBot in robots.txt, which is the control the vendors document | Only for future model versions. A block applies to future crawls, not to what a released model already learned | | Live retrieval | The assistant searches or fetches pages while it answers and cites them | Allow the search bots, stay indexed, serve content as text and write quotable passages | As soon as crawlers revisit. OpenAI says search systems can take about 24 hours to adjust to a robots.txt change | | User-requested fetch | A user asks about a specific page and the assistant fetches it | Make sure the page loads quickly and returns status 200 to the user-triggered bots | Immediately, on each request | The practical consequence is that a citation needs retrieval. If an assistant answers from memory it will not link to you, and what it says about your product depends on what was written about it before its training cut-off. That is why most LLM SEO effort goes into retrieval. ## Which crawlers should you allow for LLM SEO? Each vendor splits its bots by purpose, so you can allow search and block training. This table shows the documented behaviour. Our guide to [AI crawlers](https://serpel.app/blog/ai-crawlers) lists the rest. **Crawlers of the main LLM assistants and what the vendors say about them** | Vendor | Bot | Purpose | robots.txt | | --- | --- | --- | --- | | OpenAI | OAI-SearchBot | Surfaces websites in ChatGPT’s search features | Applies. Sites that opt out are not shown in ChatGPT search answers | | OpenAI | GPTBot | Crawls content that may be used to train foundation models | Applies. Disallow it to opt out of training | | OpenAI | ChatGPT-User | Visits pages for certain user actions | OpenAI says the rules may not apply to user-initiated requests | | Anthropic | Claude-SearchBot | Improves the quality of Claude’s search results | Honours robots.txt. Disabling it may reduce visibility in search results | | Anthropic | ClaudeBot | Collects content that could contribute to model training | Honours robots.txt. A block excludes future material from training | | Anthropic | Claude-User | Retrieves pages when a user asks a question | Honours robots.txt. Disabling it may reduce visibility for user-directed search | | Perplexity | PerplexityBot | Surfaces and links websites in Perplexity results. Not used to train foundation models | Applies. Perplexity recommends allowing it | | Perplexity | Perplexity-User | Visits a page to answer a user’s question | Generally ignores robots.txt, because a user requested the fetch | | Google | Google-Extended | A robots.txt token that controls use of crawled content for Gemini training and grounding | A token only, with no separate user agent. Does not affect inclusion in Search or ranking | Sources: the documentation of [OpenAI](https://developers.openai.com/api/docs/bots), [Anthropic](https://privacy.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler), [Perplexity](https://docs.perplexity.ai/docs/resources/perplexity-crawlers) and [Google](https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers). OpenAI and Perplexity each say their settings work independently, and Anthropic documents its three bots separately, so the file below allows search and user requests and opts out of training. ```text User-agent: OAI-SearchBot User-agent: Claude-SearchBot User-agent: PerplexityBot Allow: / User-agent: ChatGPT-User User-agent: Claude-User User-agent: Perplexity-User Allow: / User-agent: GPTBot User-agent: ClaudeBot Disallow: / ``` Google-Extended is left out on purpose. Google says the token covers both Gemini training and grounding, which means serving the Search index to the model at prompt time, so decide on it separately. Whether to opt out of training at all is a business decision, not an SEO one, and it does not decide whether you can be cited in live answers. Allowing the crawler is not enough on its own. A firewall or bot protection can block a bot that robots.txt allows, so check your CDN rules against the vendors’ published IP ranges, such as [OpenAI’s OAI-SearchBot list](https://openai.com/searchbot.json) and [Anthropic’s list](https://claude.com/crawling/bots.json). To see how your file reads, use the free [robots.txt checker](https://serpel.app/tools/robots-txt-checker). It tests your robots.txt against 13 AI crawlers, as every Serpel crawl does. ## Do LLM crawlers run JavaScript? Mostly not, according to the best public evidence. In a [December 2024 analysis](https://vercel.com/blog/the-rise-of-the-ai-crawler) of its network, Vercel found that none of the major AI crawlers it measured rendered JavaScript, with the exceptions of Gemini, which uses Googlebot’s infrastructure, and Applebot. ChatGPT and Claude fetched JavaScript files (11.50% and 23.84% of their requests) but did not execute them. The data is almost two years old, comes from Vercel’s own network and predates later crawler changes, so read it as a strong hint, not a current guarantee. The safe rule is to put everything an assistant should quote in the initial HTML. A quick test is to fetch the page without a browser and search for a sentence from it. If the count is zero, the text only exists after JavaScript runs. Our [JavaScript SEO](https://serpel.app/blog/javascript-seo) guide covers the fixes. ```bash curl -s https://www.example.com/pricing | grep -c "a sentence from your page" ``` ## What is the status of llms.txt for LLM SEO? [llms.txt](https://llmstxt.org/) is a proposal by Jeremy Howard, published in September 2024: a Markdown file at /llms.txt with a project name, a short summary and links to the pages an LLM should read, meant for use when a model is answering. The honest status is that it is optional and unproven. Google’s guide says Google Search ignores such files and that creating them will neither harm nor help your visibility there. We found no statement from OpenAI, Anthropic or Perplexity that their crawlers read your llms.txt. That does not make the file useless. It costs little, it is a tidy index of your best pages, and a coding agent that already works on your documentation can use it. Just do not expect it to change citations in ChatGPT or Google. Our [llms.txt examples](https://serpel.app/blog/llms-txt-examples) show formats that work, and the free [llms.txt generator](https://serpel.app/tools/llms-txt-generator) builds a valid file. ## Which content and entity signals help LLM assistants cite you? No vendor publishes a ranking recipe for assistants, so stay with what is documented or measured. - **Original, specific content.** Google says unique, non-commodity content will likely influence your presence in AI search more than any other suggestion in its guide. - **Facts with numbers, quotes and sources.** In the [GEO research](https://serpel.app/blog/generative-engine-optimization), adding statistics, quotations and source citations raised visibility in the authors’ test setup, while keyword stuffing did not. Use them only where true. - **Clear entities.** Use the same name for your company, product and people everywhere. Say what you are in the first paragraph of your home page and About page, and keep structured data consistent with the visible text. - **Answer-first structure.** A heading that states the question and a direct answer in the first sentences give a retrieved passage context. See [answer engine optimization](https://serpel.app/blog/answer-engine-optimization). - **Genuine mentions.** Google says seeking inauthentic mentions across the web is not as helpful as it might seem. Documentation links, honest reviews and independent comparisons are the mentions worth earning. - **Visible dates.** OpenAI tells users to check when a cited source was published or updated, so show your update date. > **Do not hide instructions for LLMs in your pages:** Hidden text that tries to steer an AI system is a risk, not a shortcut. Microsoft’s Bing Webmaster Guidelines list prompt injection among the practices they treat as abuse. Write for readers and let the assistants quote what they find. ## What should an LLM SEO tool or ChatGPT SEO tool do? Be sceptical of any tool that promises a position inside an assistant, because there is no ranking list to climb. Google also says no third-party tool has access to its internal ranking or AI systems. A useful LLM SEO tool does five things: 1. Runs real prompts on named assistants and tells you which ones it covers. 2. Stores every answer and its sources, so you can see changes over time. 3. Separates a citation (a link to your domain) from a mention (your name in the text). 4. Tests whether the assistants’ crawlers can reach your site. 5. Never promises guaranteed citations. [Serpel](https://serpel.app/features/ai-visibility) does this for ChatGPT with web search and for Google AI Overviews, records the sources of each answer and tests your robots.txt against the AI crawlers. It does not query Claude, Gemini or Perplexity, so for those assistants you need their own checks. Our comparison of the [best AI visibility tools](https://serpel.app/blog/best-ai-visibility-tools) covers the alternatives. ## How do you measure LLM SEO? 1. **Build a prompt set** Write 20 to 50 questions your customers ask an assistant, in their words. Include a few comparison and recommendation prompts. 2. **Record a baseline** Run the prompts and note, for each answer, whether you were cited, mentioned or missing, and which domains were cited instead. Repeat the run, because answers vary. ```bash serpel ai add --project <project-id> --prompt "Which SEO tool has a CLI and an MCP server?" serpel ai run --project <project-id> --wait serpel ai status --project <project-id> ``` 3. **Read your logs** Count requests from OAI-SearchBot, Claude-SearchBot, PerplexityBot and the user-triggered bots, and check that they receive status 200. 4. **Add the engine reports** Read Search Console’s Generative AI performance report and Bing’s AI Performance report, which count impressions and citations in the products they cover. 5. **Change one thing, then repeat** Edit a page, wait for the crawlers to return and run the same prompts again. Compare against the baseline, not against a single lucky answer. ## Frequently asked questions ### What is LLM SEO? LLM SEO is optimising your site so that assistants built on large language models, such as ChatGPT, Claude, Gemini and Perplexity, can find, read and cite it. It covers crawler access, server-rendered content, quotable facts and measurement of citations, and it builds on classic SEO. ### Is LLM SEO different from SEO? Mostly it builds on SEO, and Google says optimising for generative AI search is still SEO. The differences are the extra crawlers to manage, the need for passages that are easy to quote and the way you measure success, which is citations per prompt rather than a ranking position. ### Should I block GPTBot and ClaudeBot? Blocking them opts your content out of future model training and does not remove you from live search. OpenAI says its settings are independent, so you can allow OAI-SearchBot and disallow GPTBot. Whether to opt out of training is a business decision. ### Does llms.txt help with LLM SEO? There is no proof that it does. Google says Google Search ignores llms.txt, and we found no statement from OpenAI, Anthropic or Perplexity that their crawlers read it. The file is cheap to publish and can help coding agents that read your documentation, but do not expect more citations. ### How do I know if an LLM cites my website? Run a fixed set of real prompts on each assistant, record whether the answer links to your domain or only names you, and repeat the checks over time. Add your server logs for AI crawlers and the AI reports in Search Console and Bing Webmaster Tools. ## Sources - [OpenAI: Overview of OpenAI crawlers](https://developers.openai.com/api/docs/bots), accessed 2026-10-10 - [OpenAI Help Center: Searching the web with ChatGPT](https://help.openai.com/en/articles/9237897-searching-the-web-with-chatgpt), accessed 2026-10-10 - [Anthropic: Does Anthropic crawl data from the web, and how can site owners block the crawler?](https://privacy.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler), accessed 2026-10-10 - [Anthropic: Web search tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool), accessed 2026-10-10 - [Perplexity: Perplexity crawlers](https://docs.perplexity.ai/docs/resources/perplexity-crawlers), accessed 2026-10-10 - [Google for Developers: Google’s common crawlers](https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers), accessed 2026-10-10 - [Google AI for Developers: Grounding with Google Search](https://ai.google.dev/gemini-api/docs/google-search), accessed 2026-10-10 - [Google Search Central: Optimizing your website for generative AI features on Google Search](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide), accessed 2026-10-10 - [llms.txt: A proposal to standardise on using an /llms.txt file](https://llmstxt.org/), accessed 2026-10-10 - [Vercel: The rise of the AI crawler](https://vercel.com/blog/the-rise-of-the-ai-crawler), accessed 2026-10-10 - [Aggarwal et al., GEO: Generative Engine Optimization (arXiv 2311.09735, KDD 2024)](https://arxiv.org/abs/2311.09735), accessed 2026-10-10 - [Bing Webmaster Guidelines](https://www.bing.com/webmasters/help/webmaster-guidelines-30fba23a), accessed 2026-10-10 - [OpenAI: OAI-SearchBot IP ranges (JSON)](https://openai.com/searchbot.json), accessed 2026-10-10 - [Anthropic: Crawler IP ranges (JSON)](https://claude.com/crawling/bots.json), accessed 2026-10-10 --- # llms.txt example: 3 complete files for docs, shops and blogs URL: https://serpel.app/blog/llms-txt-examples Updated: 2026-10-10 An llms.txt file is a Markdown file at /llms.txt that gives AI agents a short, curated map of your site. This guide shows three complete llms.txt examples for a SaaS docs site, an online shop and a blog, built to the llmstxt.org format. Google says its Search ignores the file and we found no major AI provider that confirms reading it to answer questions, so treat it as a low-cost aid for coding agents, not a ranking factor. ## Key takeaways - An llms.txt file is Markdown with one required element, an H1 with the site name. A blockquote summary, short notes and H2 sections of links with descriptions make it useful to agents. - Jeremy Howard proposed llms.txt in September 2024, and llmstxt.org is now at version 2, last modified in August 2026. The file can sit at the site root or at any subpath such as /docs/llms.txt. - Google says its Search does not use llms.txt and that having one neither helps nor harms rankings. We found no statement from OpenAI, Anthropic or Perplexity that their search products read it, so do not expect AI citations from it. - The file does help where an agent is pointed at it. Docs platforms generate it, MCP servers such as mcpdoc serve it to coding agents, and Chrome’s Lighthouse can check for it. - llms-full.txt, a single file with the full content, is a convention outside the specification. Validate your file by checking its structure and that every link returns status 200. ## What is llms.txt? llms.txt is a proposal for a Markdown file, placed at `/llms.txt`, that gives language models and AI agents a curated overview of a website and links to the pages that matter. [Jeremy Howard](https://www.answer.ai/posts/2024-09-03-llmstxt.html) published the idea on 3 Sept 2024, and the [llmstxt.org specification](https://llmstxt.org/), now at version 2, was last modified on 10 Aug 2026. It addresses a practical problem. HTML pages wrap their information in navigation, ads and scripts, and context windows are still too small for most whole sites. A short file that points to clean Markdown pages lets an agent find what it needs and fetch only that. The specification says the file is meant to be used on demand, when an agent needs information while helping a user, and mainly for inference rather than training. It does not replace the files you already have. robots.txt tells automated tools what access is acceptable, and sitemap.xml lists all indexable pages, which is usually far too much for a context window. Our guide to [AI crawlers](https://serpel.app/blog/ai-crawlers) covers robots.txt for AI bots. ## What is the llms.txt format? A file that follows the specification contains these parts, in this order: 1. An optional byte-order mark. 2. **An H1 with the name of the project or site.** This is the only required part. 3. **A blockquote with a short summary** that holds the key information needed to understand the rest of the file. 4. **Zero or more paragraphs or lists** with more detail on how to interpret the files. These sections must not contain headings. 5. **Zero or more H2 sections that hold file lists.** Each entry is a Markdown list item with a required link in the form `[name](url)`, optionally followed by a colon and notes. By convention an H2 section called Optional holds secondary links that an agent can skip when it needs a shorter context. The file can sit at the site root or at any path, for example `/docs/llms.txt`. It covers the URLs under its path, and where several files apply, agents should use the most specific one. The proposal also recommends a clean Markdown version of each important page at the same URL, with `.md` appended or replacing the extension, and your links should point to those versions where you have them. ```markdown # Site name > One or two sentences that say what the site is and who it is for. Optional notes that help an agent interpret the links below. ## Section name - [Page title](https://www.example.com/page.md): What the page covers ## Optional - [Secondary page](https://www.example.com/secondary.md): Safe to skip when context is short ``` ## llms.txt example for a SaaS docs site Documentation is where llms.txt is used most, because coding agents follow it to find API references and tutorials. This example is for a fictional analytics product, and all names, URLs and numbers in the three examples are made up. It opens with the facts an agent would otherwise get wrong, then groups links by task. ```markdown # Northwind Metrics > Northwind Metrics is a product analytics platform with a REST API and SDKs for JavaScript, Python and Go. This file lists the pages an agent needs to send events, query reports and manage API keys. Notes for agents: - The API base URL is https://api.example.com/v1 and every request needs a bearer token. - The rate limit is 600 requests per minute per project. Retry with exponential backoff on HTTP 429. - Event names are case-sensitive and limited to 64 characters. ## Getting started - [Quickstart](https://docs.example.com/quickstart.md): Create a project, copy an API key and send a first event - [Authentication](https://docs.example.com/authentication.md): API keys, scopes and key rotation - [Core concepts](https://docs.example.com/concepts.md): Projects, events, properties and users ## API reference - [Events API](https://docs.example.com/api/events.md): Send single and batched events, with the full JSON schema - [Query API](https://docs.example.com/api/query.md): Run funnel, retention and trend queries - [Errors and rate limits](https://docs.example.com/api/errors.md): Status codes, error bodies and retry guidance ## SDKs - [JavaScript SDK](https://docs.example.com/sdks/javascript.md): Install, initialise and track in browsers and Node.js - [Python SDK](https://docs.example.com/sdks/python.md): Install, async client and batching - [Go SDK](https://docs.example.com/sdks/go.md): Client setup and context handling ## Optional - [Changelog](https://docs.example.com/changelog.md): Release notes for the API and the SDKs - [Migration guide from v0](https://docs.example.com/migrate-from-v0.md): Breaking changes and the upgrade checklist - [Status page](https://status.example.com/): Current incidents and uptime history ``` - **The summary names the product and the audience**, so an agent knows whether the file is relevant. - **The notes hold rules agents get wrong**, such as authentication, rate limits and naming limits. - **Every link has a one-line description** and points to a Markdown page. - **Secondary material sits under Optional**, so an agent with limited context can skip the changelog and migration guide. ## llms.txt example for an e-commerce store A shop has more pages than any context window can hold, so link to category pages, buying guides and policies rather than individual products. Policies matter most, because customers ask assistants about delivery and returns. ```markdown # Fernwood Outdoor > Fernwood Outdoor is an online shop for hiking and camping gear that ships from Leeds to the UK and Ireland. This file points to category pages, buying guides and the policies customers ask about most. Key facts: - Orders over £60 ship free in the UK, and standard delivery takes 2 to 4 working days. - Unused items can be returned within 30 days. - Prices are in GBP and include VAT. ## Shop by category - [Tents](https://www.example.com/tents.md): Backpacking, family and ultralight tents with weight and season ratings - [Sleeping bags](https://www.example.com/sleeping-bags.md): Down and synthetic bags, listed by comfort temperature - [Rucksacks](https://www.example.com/rucksacks.md): Day packs and trekking packs, listed by capacity in litres - [Footwear](https://www.example.com/footwear.md): Boots and trail shoes with width and waterproofing details ## Buying guides - [How to choose a tent](https://www.example.com/guides/choose-a-tent.md): Season ratings, weight and packed size explained - [Sleeping bag temperature ratings](https://www.example.com/guides/sleeping-bag-ratings.md): What comfort and limit ratings mean - [Boot sizing guide](https://www.example.com/guides/boot-sizing.md): How to measure your foot and compare brands ## Policies - [Delivery and shipping](https://www.example.com/policies/delivery.md): Costs, delivery times and countries served - [Returns and refunds](https://www.example.com/policies/returns.md): Return window, condition rules and how refunds are paid - [Warranty and repairs](https://www.example.com/policies/warranty.md): What is covered and how to make a claim ## Support - [Contact us](https://www.example.com/contact.md): Email, phone hours and live chat - [Order tracking](https://www.example.com/orders/tracking.md): How to track a parcel ## Optional - [About Fernwood](https://www.example.com/about.md): The company, its stores and its sustainability commitments - [Sitemap](https://www.example.com/sitemap.xml): Every product URL, for tools that need the full catalogue ``` Put the facts people ask about, such as the free shipping threshold, the return window and the currency, in the notes, and link the sitemap under Optional for tools that need the full catalogue. Keep these numbers in sync with your real policies, because an agent will repeat them. ## llms.txt example for a blog A blog benefits from a short list of the posts you most want read, a topic map and pages that show who writes it and how it is maintained. ```markdown # Ledgerline Engineering Blog > Practical articles on payments infrastructure, written by the engineers at Ledgerline. Posts include working code and are updated when the underlying APIs change. About this blog: - New posts appear every Tuesday, and each post shows its publication and last updated dates. - Code samples use TypeScript and Postgres unless a post says otherwise. - Author pages list the role and expertise of each writer. ## Start here - [Idempotency keys explained](https://blog.example.com/idempotency-keys.md): How to make payment retries safe, with a complete Node.js example - [Designing a double-entry ledger](https://blog.example.com/double-entry-ledger.md): Schema, constraints and queries for a Postgres ledger - [Webhook reliability checklist](https://blog.example.com/webhook-reliability.md): Signatures, retries and replay protection ## Topics - [Payments](https://blog.example.com/topics/payments.md): All posts about charges, refunds and payouts - [Databases](https://blog.example.com/topics/databases.md): Schema design, migrations and performance - [Reliability](https://blog.example.com/topics/reliability.md): Incident reviews and resilience patterns ## About the authors - [Authors](https://blog.example.com/authors.md): Who writes here and what they work on - [Editorial policy](https://blog.example.com/editorial-policy.md): How posts are reviewed, corrected and updated ## Optional - [Full archive](https://blog.example.com/archive.md): Every post, newest first - [RSS feed](https://blog.example.com/feed.xml): Subscribe to new posts ``` Curate rather than dump. List the pillar posts instead of the archive, and link the archive under Optional. State how often posts are updated and how corrections are handled, because that is what a reader or an agent needs to judge reliability. ## What is llms-full.txt? llms-full.txt is not part of the llmstxt.org specification, which does not mention it. It is a convention in which a site publishes the full text of its documentation as one Markdown file next to llms.txt, so an agent can load everything in a single request. [Anthropic’s developer docs](https://platform.claude.com/llms.txt) show the pattern: the llms.txt lists pages as links and ends with a pointer to llms-full.txt. Cloudflare uses the subpath approach that the specification allows, with a [top-level llms.txt](https://developers.cloudflare.com/llms.txt) that points to a separate llms.txt for each product. Use a full file only when the whole text fits comfortably in a model’s context, for example for a small API. Large sites are better served by the index file and clean per-page Markdown, because a multi-megabyte file defeats the purpose. ## Do AI systems read llms.txt? Nobody has confirmed it for search. Google’s [guide to generative AI features](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide) says you don’t need llms.txt to appear in Google Search, that Google Search itself does not use such files, and that creating one will neither help nor harm your visibility there. Google’s John Mueller wrote in [June 2025](https://www.seroundtable.com/google-ai-llms-txt-39607.html) that no AI system currently uses llms.txt, pointing to server logs, and in [June 2026](https://www.searchenginejournal.com/google-says-llms-txt-is-purely-speculative-for-now/577576/) called it purely speculative for now. We found no statement from OpenAI, Anthropic or Perplexity that their search products read it to choose sources. Where it does have a use is with tools you point at it. The proposal lists documentation platforms such as Mintlify and GitBook that generate the file, and [mcpdoc](https://github.com/langchain-ai/mcpdoc) is an MCP server that serves a list of llms.txt files to hosts including Cursor, Windsurf, Claude Desktop and Claude Code. Chrome’s Lighthouse [includes an optional llms.txt audit](https://developer.chrome.com/docs/lighthouse/agentic-browsing/llms-txt) that flags a server error and treats a missing file as not applicable. You can collect your own evidence instead of trusting claims. This command lists which user agents request your llms.txt. If no AI crawler shows up after a few months, that fits the statements above, and the file still costs almost nothing to keep. For what does move AI visibility, read our guide to [generative engine optimization](https://serpel.app/blog/generative-engine-optimization). ```bash grep "GET /llms.txt" access.log | awk -F'"' '{print $6}' | sort | uniq -c | sort -rn | head ``` ## How do you validate an llms.txt file? llmstxt.org defines the format but does not provide an llms.txt validator. A useful check covers three things: the structure follows the format (one H1 first, a blockquote, entries written as `- [name](url): notes`), every link is absolute and returns status 200, and each link has a short description. The specification’s own advice is to test the file by giving an agent only your llms.txt and asking it questions about your content. This small Node script (Node 18 or later) checks structure and links and warns about missing notes. Run it with a URL or a file path. It exits with status 1 when it finds a problem, so you can add it to CI after each deploy. ```javascript import { readFile } from "node:fs/promises"; const location = process.argv[2]; if (!location) { console.error("Usage: node validateLlmsTxt.mjs <url-or-file>"); process.exit(2); } async function loadText() { if (!/^https?:\/\//i.test(location)) return readFile(location, "utf8"); const response = await fetch(location); if (!response.ok) throw new Error("HTTP " + response.status + " for " + location); return response.text(); } async function readStatus(url) { try { const head = await fetch(url, { method: "HEAD" }); return head.status === 405 || head.status === 403 ? (await fetch(url)).status : head.status; } catch (error) { return error instanceof Error ? error.message : String(error); } } const lines = (await loadText()).replace(/^\uFEFF/, "").split(/\r?\n/); const problems = []; const warnings = []; const headings = lines.filter((line) => /^#\s+\S/.test(line)); if (headings.length !== 1) problems.push("Expected one H1, found " + headings.length + "."); if (lines.find((line) => line.trim()) !== headings[0]) problems.push("The H1 must be the first line."); if (!lines.slice(1).find((line) => line.trim())?.startsWith(">")) warnings.push("No blockquote summary after the H1."); const entryPattern = /^\s*[-*]\s+\[([^\]]+)\]\((\S+?)\)(?::\s*(.+))?\s*$/; const entries = lines.map((line) => entryPattern.exec(line)).filter(Boolean); if (entries.length === 0) problems.push("No entries like - [name](url): notes found."); const statuses = await Promise.all(entries.map(([, , url]) => (/^https?:\/\//i.test(url) ? readStatus(url) : "relative"))); entries.forEach(([, name, url, notes], index) => { if (statuses[index] !== 200) problems.push('Link "' + name + '" returned ' + statuses[index] + ": " + url); if (!notes) warnings.push('Link "' + name + '" has no notes.'); }); console.log(entries.length + " links, " + problems.length + " problem(s), " + warnings.length + " warning(s)."); problems.forEach((text) => console.log("PROBLEM " + text)); warnings.forEach((text) => console.log("WARNING " + text)); process.exitCode = problems.length > 0 ? 1 : 0; ``` To create a file, write it by hand from the examples above, use your docs platform’s generator, or start from Serpel’s free [llms.txt generator](https://serpel.app/tools/llms-txt-generator). Serpel’s site audit also reports a missing llms.txt as a notice, and `serpel ai status` shows whether one is reachable from your last crawl, next to the AI crawlers your robots.txt allows. See [AI visibility in Serpel](https://serpel.app/features/ai-visibility) for the rest. ## Frequently asked questions ### What is an llms.txt file? An llms.txt file is a Markdown file at /llms.txt that gives AI agents a short, curated overview of a website: an H1 with the site name, a blockquote summary and lists of links with descriptions. Jeremy Howard proposed it in September 2024. It is a proposal, not a formal standard. ### Does ChatGPT or Google use llms.txt? Google says its Search does not use llms.txt and that having one neither helps nor harms visibility. We found no statement from OpenAI, Anthropic or Perplexity that their search products read it to choose sources. Some tools, such as MCP servers for coding agents, do read it when you point them at it. ### Where do I put llms.txt? At the root of your site, so it is served at /llms.txt. The specification also allows the file at any subpath, such as /docs/llms.txt, where it covers the URLs under that path. If several files apply, agents should use the most specific one. ### What is the difference between llms.txt, robots.txt and sitemap.xml? robots.txt tells crawlers which access is acceptable, and sitemap.xml lists all indexable pages for search engines. llms.txt is a short, curated overview for agents that need information on demand. It does not allow or block anything. ### Do I need an llms-full.txt file? No. It is a convention, not part of the specification. It only makes sense for small documentation sets whose full text fits in a model’s context, as one file an agent can load in a single request. ## Sources - [llmstxt.org: The /llms.txt file, v2 (Jeremy Howard)](https://llmstxt.org/), accessed 2026-10-10 - [Answer.AI: The /llms.txt file (Jeremy Howard, 3 Sept 2024)](https://www.answer.ai/posts/2024-09-03-llmstxt.html), accessed 2026-10-10 - [Google Search Central: Optimizing your website for generative AI features on Google Search](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide), accessed 2026-10-10 - [Search Engine Roundtable: Google says no AI system currently uses llms.txt (June 2025)](https://www.seroundtable.com/google-ai-llms-txt-39607.html), accessed 2026-10-10 - [Search Engine Journal: Google says llms.txt is purely speculative for now (June 2026)](https://www.searchenginejournal.com/google-says-llms-txt-is-purely-speculative-for-now/577576/), accessed 2026-10-10 - [Chrome for Developers: Lighthouse llms.txt audit](https://developer.chrome.com/docs/lighthouse/agentic-browsing/llms-txt), accessed 2026-10-10 - [Anthropic: Developer documentation llms.txt](https://platform.claude.com/llms.txt), accessed 2026-10-10 - [Cloudflare: Developer documentation llms.txt](https://developers.cloudflare.com/llms.txt), accessed 2026-10-10 - [LangChain: mcpdoc, an MCP server for llms.txt files](https://github.com/langchain-ai/mcpdoc), accessed 2026-10-10 --- # Next.js SEO: the App Router guide to metadata, sitemaps and rendering URL: https://serpel.app/blog/nextjs-seo Updated: 2026-10-10 Next.js SEO comes down to using the App Router’s built-in features correctly: the Metadata API for titles, descriptions and canonicals, `sitemap.ts` and `robots.ts` for crawling, and prerendered or server-rendered HTML so every crawler sees your content. This guide shows working code for Next.js 15 and 16, the mistakes that cause most indexing problems and how to check the result on the live site. ## Key takeaways - Next.js sends complete HTML for prerendered and server-rendered routes. The Metadata API, `sitemap.ts` and `robots.ts` cover most technical SEO. - Set `metadataBase` once, define a title template in the root layout and canonicals per page. Page-level objects such as `openGraph` replace the layout’s object instead of merging with it. - Prerender indexable pages with `generateStaticParams` so titles and canonicals sit in the HTML head for every crawler, and return a real 404 with `notFound()` for unknown slugs. - Use `next/image` and `next/font` for Core Web Vitals, then check the live site with a crawl. ## Is Next.js good for SEO? Yes, when you use it for what it is good at. In the App Router, pages are Server Components by default, and routes that do not need request-time data are prerendered at build time. The crawler then receives full HTML, with the content, links and metadata already in place. Google recommends [server-side rendering, static rendering or hydration](https://developers.google.com/search/docs/crawling-indexing/javascript/dynamic-rendering) over workarounds, and many other crawlers do not run scripts at all. Our guide to [JavaScript SEO](https://serpel.app/blog/javascript-seo) explains why. Next.js does not do SEO for you. It gives you the tools, and the mistakes are usually configuration mistakes. The code below follows the Next.js 16 documentation. In Next.js 15 and 16, `params` is [a Promise](https://nextjs.org/docs/app/guides/upgrading/version-15), so you await it. ## How do you set titles, descriptions and canonicals with the Metadata API? Export a `metadata` object for static values or a `generateMetadata` function for data-driven ones. Both are only supported in [Server Components](https://nextjs.org/docs/app/getting-started/metadata-and-og-images). Put the shared parts in the root layout. ```typescript import type { Metadata } from 'next' export const metadata: Metadata = { metadataBase: new URL('https://example.com'), title: { default: 'Example', template: '%s | Example', }, description: 'Example builds invoicing software for freelancers.', } export default function RootLayout({ children }: { children: React.ReactNode }) { return ( <html lang="en"> <body>{children}</body> </html> ) } ``` ```typescript import type { Metadata } from 'next' import { notFound } from 'next/navigation' import { getPost } from '@/lib/posts' type PageProps = { params: Promise<{ slug: string }> } export async function generateMetadata({ params }: PageProps): Promise<Metadata> { const { slug } = await params const post = await getPost(slug) if (!post) { return {} } return { title: post.title, description: post.summary, alternates: { canonical: `/blog/${slug}` }, openGraph: { type: 'article', title: post.title, description: post.summary }, } } export default async function Page({ params }: PageProps) { const { slug } = await params const post = await getPost(slug) if (!post) { notFound() } return <h1>{post.title}</h1> } ``` - **Set metadataBase once.** It lets you use relative URLs in fields such as `alternates` and `openGraph`. Without it, a relative URL causes a build error, as the [generateMetadata reference](https://nextjs.org/docs/app/api-reference/functions/generate-metadata) explains. - **A title template applies to child segments only.** `title.template` does not affect a title in a `page.tsx` next to the layout that defines it, and it needs a `title.default`. - **Metadata is merged shallowly.** A nested object such as `openGraph` or `robots` defined in a page replaces the one from the layout. Repeat every field you still want. - **A canonical in the root layout is inherited.** Every page without its own `alternates` would canonicalise to that URL. Set canonicals per page. - **Share data fetching.** Wrap `getPost` in React’s `cache` so `generateMetadata` and the page do not fetch twice. ## How do you generate a sitemap and robots.txt in Next.js? Add `app/sitemap.ts` and `app/robots.ts`. Both are special route handlers that are cached by default unless they use request-time APIs, according to the [sitemap](https://nextjs.org/docs/app/api-reference/file-conventions/metadata/sitemap) and [robots](https://nextjs.org/docs/app/api-reference/file-conventions/metadata/robots) references. ```typescript import type { MetadataRoute } from 'next' import { getAllPosts } from '@/lib/posts' export default async function sitemap(): Promise<MetadataRoute.Sitemap> { const posts = await getAllPosts() return [ { url: 'https://example.com', lastModified: '2026-10-01' }, ...posts.map((post) => ({ url: `https://example.com/blog/${post.slug}`, lastModified: post.updatedOn, })), ] } ``` ```typescript import type { MetadataRoute } from 'next' export default function robots(): MetadataRoute.Robots { return { rules: { userAgent: '*', allow: '/', disallow: ['/api/', '/admin/'] }, sitemap: 'https://example.com/sitemap.xml', } } ``` - **Use real dates.** Google [ignores the priority and changefreq values](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap) and uses `<lastmod>` only if it is consistently and verifiably accurate. `new Date()` for every URL fails that test. - **Stay within the limits.** A sitemap may hold 50,000 URLs or 50 MB uncompressed. Split larger sites with `generateSitemaps`. - **List canonical, indexable URLs only.** Leave out redirects, `noindex` pages and error pages. - **Do not use robots.txt to hide pages.** Google says it is [not a mechanism for keeping a page out of Google](https://developers.google.com/search/docs/crawling-indexing/robots/intro). Use `noindex` or authentication. Run the generated file through the free [sitemap checker](https://serpel.app/tools/sitemap-checker), then submit the sitemap URL in Search Console. Google treats a submitted sitemap as a suggestion and does not guarantee it will download or use it, so strong internal links still matter. ## How do you add JSON-LD and Open Graph images? Next.js recommends rendering [JSON-LD](https://nextjs.org/docs/app/guides/json-ld) as a plain `<script>` tag in a layout or page. `JSON.stringify` does not sanitise strings, so replace `<` with its unicode escape to prevent injection. Validate the result in Google’s Rich Results Test or with the free [schema validator](https://serpel.app/tools/schema-validator). ```typescript import { notFound } from 'next/navigation' import { getPost } from '@/lib/posts' export default async function Page({ params }: { params: Promise<{ slug: string }> }) { const { slug } = await params const post = await getPost(slug) if (!post) { notFound() } const jsonLd = { '@context': 'https://schema.org', '@type': 'BlogPosting', headline: post.title, datePublished: post.publishedOn, dateModified: post.updatedOn, } return ( <article> <script type="application/ld+json" dangerouslySetInnerHTML={{ __html: JSON.stringify(jsonLd).replace(/</g, '\\u003c'), }} /> <h1>{post.title}</h1> </article> ) } ``` For social previews, add an [opengraph-image file](https://nextjs.org/docs/app/api-reference/file-conventions/metadata/opengraph-image) to a route segment. A static `.png` or `.jpg` works, and so does a `.tsx` file that returns an `ImageResponse`. Generated images are statically optimised at build time unless they use request-time APIs. A more specific segment overrides the one above it, so a file in `app/blog/[slug]` wins over the one in `app`. ```typescript import { ImageResponse } from 'next/og' export const alt = 'Example, invoicing for freelancers' export const size = { width: 1200, height: 630 } export const contentType = 'image/png' export default function Image() { return new ImageResponse( ( <div style={{ width: '100%', height: '100%', display: 'flex', alignItems: 'center', justifyContent: 'center', fontSize: 96, background: 'white', }} > Example </div> ), { ...size }, ) } ``` ## Which rendering mode should each page use? **Rendering choices in the App Router** | Page type | Approach | Notes | | --- | --- | --- | | Blog, docs and marketing pages | Prerender at build with `generateStaticParams` | Complete HTML. With `dynamicParams = false`, unknown slugs return 404 | | Large catalogue | Prerender a subset and render the rest on first visit | Return a partial list from `generateStaticParams`. Other pages are rendered when first requested, as [documented](https://nextjs.org/docs/app/api-reference/functions/generate-static-params) | | Personalised pages such as carts and dashboards | Dynamic rendering | Keep them out of the sitemap and add `noindex` | | Pages with a few dynamic parts | Cache Components with `<Suspense>` | With `cacheComponents: true`, static and cached content ships in the [initial HTML](https://nextjs.org/docs/app/getting-started/caching) and runtime data streams in behind fallbacks | ```typescript import { getAllPosts } from '@/lib/posts' export const dynamicParams = false export async function generateStaticParams() { const posts = await getAllPosts() return posts.map((post) => ({ slug: post.slug })) } ``` Watch metadata on dynamic pages. For dynamically rendered pages, Next.js [streams metadata](https://nextjs.org/docs/app/api-reference/functions/generate-metadata) and appends it to the body once `generateMetadata` resolves. Bots that run JavaScript, such as Googlebot, read it correctly. For crawlers on a built-in list of HTML-limited bots, which includes Bingbot, Twitterbot and Slackbot, the metadata still blocks rendering and lands in `<head>`. Prerendered pages resolve metadata at build time and do not stream. If you need metadata in the head for every crawler, set the [htmlLimitedBots option](https://nextjs.org/docs/app/api-reference/config/next-config-js/htmlLimitedBots) to `/.*/`. The documentation warns that overriding it can lead to longer response times. ## How do canonical URLs and hreflang work in Next.js? Use `alternates.canonical` and `alternates.languages`. Google requires that each language version [lists itself and all other versions](https://developers.google.com/search/docs/specialty/international/localized-versions), with fully qualified URLs. Annotations without matching return links are ignored. `x-default` is the fallback for visitors who match no language. Set the same alternates on every version, and use `alternates.languages` in `sitemap.ts` if you prefer the sitemap method. ```typescript import type { Metadata } from 'next' export const metadata: Metadata = { alternates: { canonical: '/en/pricing', languages: { 'en-GB': '/en/pricing', 'de-DE': '/de/pricing', 'x-default': '/en/pricing', }, }, } ``` ## How do you keep images and fonts from hurting Core Web Vitals? `next/image` uses `width` and `height` to reserve space, which avoids layout shift. Images load lazily by default, so the hero image needs the opposite. Since Next.js 16 the `priority` prop is deprecated in favour of `preload`, but the documentation says that in most cases `loading="eager"` or `fetchPriority="high"` is the better choice. `next/font` downloads Google fonts at build time and self-hosts them, and `adjustFontFallback` is on by default to reduce layout shift. See our guide to the [Core Web Vitals test](https://serpel.app/blog/core-web-vitals-test) for the thresholds. ```typescript import Image from 'next/image' import { Inter } from 'next/font/google' const inter = Inter({ subsets: ['latin'], display: 'swap' }) export default function Hero() { return ( <section className={inter.className}> <h1>Invoices that send themselves</h1> <Image src="/hero.webp" alt="Product dashboard" width={1200} height={630} loading="eager" fetchPriority="high" /> </section> ) } ``` ## What are the most common Next.js SEO mistakes? **Mistakes, effects and fixes** | Mistake | Effect | Fix | | --- | --- | --- | | Fetching page content in `useEffect` inside a Client Component | The HTML is empty for crawlers that do not render | Fetch in a Server Component | | Exporting `metadata` from a Client Component | Not supported | Move the export to a server `layout` or `page` | | Overriding `openGraph` or `robots` in a page | Fields from the layout disappear | Repeat the full object or build it with a helper | | A canonical in the root layout | Every page canonicalises to one URL | Set `alternates.canonical` per page | | A staging `noindex` shipped to production | The site drops out of the index | Drive `robots` from an environment variable and check after each deploy | | Unknown slugs render a “not found” message with 200 | [Soft 404](https://serpel.app/blog/soft-404) pages | Call `notFound()` before streaming starts | | Navigation with `router.push` on a button | Crawlers find no link | Use `next/link` or `<a href>`, which Google can [crawl](https://developers.google.com/search/docs/crawling-indexing/links-crawlable) | | `new Date()` as `lastModified` for every URL | Google stops trusting `lastmod` | Use the real update date | ## How does Serpel’s codebase scan map Next.js routes to live pages? The Serpel CLI can link your source to what is live. `serpel scan` runs locally and needs `next` in your `package.json`. It reads the routes in `app/` and `pages/`, also under `src/`, and drops route groups such as `(marketing)`. Dynamic segments like `[slug]` and catch-all segments stay in the route. Parallel routes, intercepted routes and private folders are not routes. It also reads static metadata, meaning `title`, `description`, `alternates.canonical` and `robots` written as strings or template literals without expressions, and it applies `title.template` from parent layouts. Anything computed, including `generateMetadata`, counts as unknown unless build output exists. If a `.next` or `out` folder holds HTML files, the scan also uses their title, description, headings and structured data types. The scan never reads `.env` files, keys, `node_modules`, binary files or anything in `.gitignore`, and it sends extracted metadata, never source code. Use `--dry-run` to see exactly what would be sent. After an upload, Serpel matches each route with the pages of your latest completed crawl and your Search Console page data. A static route matches its exact path, and a dynamic route matches by segment with the most specific route winning. ```bash serpel scan --dry-run serpel scan --project <project-id> --wait ``` The result lists routes without a live page, live pages without a route, routes without metadata and titles or descriptions that differ between code and live site. An agent can read the same summary through the `get_codebase_overview` tool of the [MCP server](https://serpel.app/developers/mcp), which links a crawl finding to the file that renders it. ## Next.js SEO best practices at a glance 1. Set `metadataBase`, a title template and a description in the root layout. 2. Give every page a unique title, description and canonical. 3. Prerender indexable pages and return a real 404 for unknown slugs. 4. Generate `sitemap.ts` with real `lastModified` dates and `robots.ts` without hiding pages. 5. Add JSON-LD as a plain script tag and an `opengraph-image` per route. 6. Use `next/image` and `next/font`, and keep the LCP image out of lazy loading. 7. Submit the sitemap in Search Console and keep a staging noindex out of production. 8. Crawl the live site after each release. ## Frequently asked questions ### Is Next.js good for SEO? Yes, when pages are prerendered or server-rendered, because crawlers then receive complete HTML with content, links and metadata. Problems come from client-side data fetching, wrong canonicals, missing 404 responses and metadata that is overridden by mistake. ### How do I add meta tags in the Next.js App Router? Export a `metadata` object or a `generateMetadata` function from a server `layout.tsx` or `page.tsx`. Next.js builds the head tags from them, and `metadataBase` lets you use relative URLs for canonicals and images. ### Do I need next-sitemap or another package for a sitemap? No. The App Router supports `app/sitemap.ts` and `app/robots.ts` natively, and `generateSitemaps` splits large sitemaps. A package is only worth adding if you need features the built-in files do not offer. ### Why does my Next.js page show a 200 status for a missing slug? The page probably renders a not-found message without calling `notFound()`, or calls it after streaming has started. Next.js then keeps the 200 status and relies on a `noindex` tag. Check for the slug before the page streams. ### How can I check my Next.js SEO on the live site? Crawl the site and compare it with your code. A site audit finds missing titles, wrong canonicals and soft 404s, and `serpel scan` shows routes without a live page or without metadata. ## Sources - [Next.js documentation: Metadata and OG images](https://nextjs.org/docs/app/getting-started/metadata-and-og-images), accessed 2026-10-10 - [Next.js documentation: generateMetadata](https://nextjs.org/docs/app/api-reference/functions/generate-metadata), accessed 2026-10-10 - [Next.js documentation: sitemap.xml](https://nextjs.org/docs/app/api-reference/file-conventions/metadata/sitemap), accessed 2026-10-10 - [Next.js documentation: robots.txt](https://nextjs.org/docs/app/api-reference/file-conventions/metadata/robots), accessed 2026-10-10 - [Next.js documentation: opengraph-image and twitter-image](https://nextjs.org/docs/app/api-reference/file-conventions/metadata/opengraph-image), accessed 2026-10-10 - [Next.js documentation: How to implement JSON-LD](https://nextjs.org/docs/app/guides/json-ld), accessed 2026-10-10 - [Next.js documentation: generateStaticParams](https://nextjs.org/docs/app/api-reference/functions/generate-static-params), accessed 2026-10-10 - [Next.js documentation: Caching and Cache Components](https://nextjs.org/docs/app/getting-started/caching), accessed 2026-10-10 - [Next.js documentation: htmlLimitedBots](https://nextjs.org/docs/app/api-reference/config/next-config-js/htmlLimitedBots), accessed 2026-10-10 - [Next.js documentation: Upgrading to version 15](https://nextjs.org/docs/app/guides/upgrading/version-15), accessed 2026-10-10 - [Next.js documentation: Image component](https://nextjs.org/docs/app/api-reference/components/image), accessed 2026-10-10 - [Next.js documentation: Font module](https://nextjs.org/docs/app/api-reference/components/font), accessed 2026-10-10 - [Google Search Central: Build and submit a sitemap](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap), accessed 2026-10-10 - [Google Search Central: Introduction to robots.txt](https://developers.google.com/search/docs/crawling-indexing/robots/intro), accessed 2026-10-10 - [Google Search Central: Localized versions of your pages](https://developers.google.com/search/docs/specialty/international/localized-versions), accessed 2026-10-10 - [Google Search Central: Dynamic rendering as a workaround](https://developers.google.com/search/docs/crawling-indexing/javascript/dynamic-rendering), accessed 2026-10-10 - [Google Search Central: Make your links crawlable](https://developers.google.com/search/docs/crawling-indexing/links-crawlable), accessed 2026-10-10 --- # SEO audit report example: Serpel’s own crawl of serpel.app URL: https://serpel.app/blog/seo-audit-report-example Updated: 2026-10-10 An SEO audit report lists what a crawl found on your site, ranks the findings by severity and says what to fix first. This example is real: Serpel crawled its own site, serpel.app, twice on 10 Oct 2026 and scored it 90 and then 93 out of 100. Below you see every part of the report with the actual numbers, an honest note on what we fixed and a format you can copy. ## Key takeaways - An SEO audit report has five core parts: a summary with a score, findings grouped by severity and category, Core Web Vitals, indexability and crawl health, and a prioritised fix list. Add a comparison with the previous crawl so progress is visible. - In this real example, Serpel’s crawl of serpel.app scored 93 out of 100 with 0 errors, 2 warnings and 71 notices across 52 pages. A first crawl earlier the same day had scored 90 with 3 warnings and 87 notices. - Treat findings as leads, not verdicts. Of the 43 redirecting link targets the crawl reported, 31 answered 200 without a redirect when we re-checked them, and the mobile LCP of the home page moved from 3.6 s to 4.1 s between two crawls an hour apart. - Serpel’s score is its own calculation, not a Google metric: 100 points minus a deduction per issue type, weighted by severity and by the share of affected pages. - Crawl again and compare after every round of fixes. Four issue types were resolved between our two crawls, and the score rose by 3 points. ## What is an SEO audit? An SEO audit is a structured check of everything that affects how search engines find, understand and show your pages. A technical audit crawls the site the way a search engine does and reports problems such as broken links, pages blocked from the index, missing titles, invalid structured data and slow loading. The result is an SEO audit report: a document that turns hundreds of raw findings into a short list of decisions. A full audit has three layers: technical (the crawl), content (does each page answer a real query) and off-page (links and mentions). A crawler automates the first layer and supplies data for the second. It cannot judge search intent or quality on its own. Google’s [SEO starter guide](https://developers.google.com/search/docs/fundamentals/seo-starter-guide) is a good yardstick for the basics, because it covers unique titles and descriptions, descriptive URLs, links, images and structured data. ## SEO audit report example: Serpel’s crawl of serpel.app We ran Serpel’s [site audit](https://serpel.app/features/site-audit) on our own marketing site, serpel.app, on 10 Oct 2026. We crawled it twice: once before a round of fixes and once after. Every number below was read from the real reports with the Serpel CLI (`serpel crawl show`, `serpel audit issues`, `serpel audit web-vitals` and `serpel crawl compare`). Nothing is mocked up. **The two crawls of serpel.app on 10 Oct 2026** | Measure | First crawl | Second crawl | | --- | --- | --- | | Pages crawled | 52 (limit 60) | 52 (limit 200) | | Crawl device | Mobile | Mobile | | JavaScript rendering | Always render, 49 pages rendered | Automatic, 0 pages rendered | | Duration | 112 s | 108 s | | Average response time | 40 ms | 37 ms | | External link targets checked | 173 | 173 | | Issue types found | 10 | 7 | | Errors, warnings, notices | 0, 3, 87 | 0, 2, 71 | | Score | 90/100 (Excellent) | 93/100 (Excellent) | One detail about rendering. The first crawl was set to always render, so all 49 HTML pages were loaded in a headless browser. The second used the default automatic mode, which renders only pages that arrive almost empty, and it rendered none. The findings that stayed had identical counts in both crawls, 46 render-blocking pages for example, which is what you expect from a site that sends its content in the HTML. If your pages depend on JavaScript, the mode matters a lot. Our guide to [JavaScript SEO](https://serpel.app/blog/javascript-seo) explains why. ## What goes in the summary of an SEO audit report? The summary is the only part many readers will see. It must answer three questions: how much was checked, how healthy is the site and what changed since last time. ```text Pages: 52 Score: 93/100 (Excellent) Issues: 0 errors, 2 warnings, 71 notices Since the last crawl: 1 new, 4 resolved issue types ``` The counts are per affected URL. The 2 warnings are two pages with the same warning, and the 71 notices are 46 + 13 + 4 + 3 + 3 + 2 pages across six issue types. The score is Serpel’s own calculation, not a Google metric. It starts at 100 and subtracts a deduction for each issue type. An error costs 6 to 20 points, a warning 2 to 8 and a notice 0.5 to 2, depending on the share of HTML pages affected. The table shows how the second crawl lost its points. **Score deductions in the second crawl (49 HTML pages)** | Issue type | Severity | Pages affected | Points deducted | | --- | --- | --- | --- | | Largest Contentful Paint is poor | Warning | 2 of 49 | 2.2 | | Render-blocking resources in the head (heuristic) | Notice | 46 of 49 | 1.9 | | External links point to redirecting URLs | Notice | 13 of 49 | 0.9 | | Page tells search engines not to follow links | Notice | 4 of 49 | 0.6 | | Largest Contentful Paint needs improvement | Notice | 3 of 49 | 0.6 | | Page is set to noindex | Notice | 3 of 49 | 0.6 | | Structured data without recommended fields | Notice | 2 of 49 | 0.6 | Together the deductions add up to 7.4 points, so the score is 92.6, which Serpel rounds to 93. Two things stand out. A warning on only 2 pages costs more than a notice on 46 pages, because severity sets the base cost. And a high score is not the goal: a site can score 93 and still have a poor mobile LCP on its home page, as ours does. ## How should an audit report group issues by severity and category? Group findings twice. Severity tells you how urgent a finding is: errors break something, warnings are likely to cost you, notices are worth knowing. Category tells you who has to act: a developer for Core Web Vitals, an editor for descriptions, an SEO for canonicals. Serpel’s catalogue has 94 checks in 18 categories, split into 11 error, 43 warning and 40 notice checks. Each finding carries the same four facts: what was found, why it matters, how to fix it and which URLs are affected. **The seven issue types in the second crawl, with our verdict** | Issue | Severity | Category | URLs | Verdict | | --- | --- | --- | --- | --- | | Largest Contentful Paint is poor | Warning | Core Web Vitals | 2 | Real. Mobile lab test of / and /bot. Fix first, after a re-test. | | Render-blocking resources in the head (heuristic) | Notice | Response time | 46 | Heuristic. Same scripts on every page. Check a browser trace first. | | External links point to redirecting URLs | Notice | Linking | 13 | Partly real. 12 of 43 redirecting targets had moved. | | Page tells search engines not to follow links | Notice | Indexing | 4 | Intended on /cancel, /login and /signup. A mistake on the public /bot page. | | Largest Contentful Paint needs improvement | Notice | Core Web Vitals | 3 | Real. /about, /privacy and /terms on mobile. Same metric as the warning. | | Page is set to noindex | Notice | Indexing | 3 | Intended. /cancel, /login and /signup should stay out of the index. | | Structured data without recommended fields | Notice | Structured data | 2 | Valid markup. Optional fields missing on / and /about. | The verdict column is the part a raw export lacks. It records that someone looked at each finding and decided what it means, and that is what turns an export into an audit report. Serpel marks heuristic checks, such as render-blocking resources, so you can tell a rule of thumb from a confirmed error. ### What does a single finding look like? Open any issue and Serpel shows the detail behind the count. This is the one warning of the second crawl. ```text Largest Contentful Paint is poor Severity: Warning Category: Core Web Vitals Affected: 2 URLs Why it matters The Largest Contentful Paint (LCP) is over 4 seconds. Google rates that as poor. Visitors wait too long for the main content and leave more often. Fix Optimize the largest visible element, usually an image or a text block: compress and resize images, load the LCP image with high priority (fetchpriority="high") and without lazy loading, and reduce server response time and blocking resources. Affected URLs https://serpel.app/ value=4051 ms, source=lab https://serpel.app/bot value=5437 ms, source=lab ``` ## Which Core Web Vitals numbers belong in an audit report? Report all three metrics for mobile and for desktop, and say whether each value is lab or field data. Google’s [Web Vitals guidance](https://web.dev/articles/vitals) sets the good thresholds at 2.5 seconds for Largest Contentful Paint (LCP), 200 milliseconds for Interaction to Next Paint (INP) and 0.1 for Cumulative Layout Shift (CLS), assessed at the 75th percentile of page loads. Lab data is one simulated load. Field data comes from real Chrome users over 28 days. The [PageSpeed Insights documentation](https://developers.google.com/speed/docs/insights/v5/about) says the two can differ and that measurements vary between runs. **Lab LCP of the five measured pages, from serpel audit web-vitals** | Page | Mobile, first crawl | Mobile, second crawl | Desktop, second crawl | | --- | --- | --- | --- | | / | 3.6 s (needs improvement) | 4.1 s (poor) | 0.8 s (good) | | /about | 3.0 s (needs improvement) | 2.6 s (needs improvement) | 0.7 s (good) | | /bot | 3.6 s (needs improvement) | 5.4 s (poor) | 1.0 s (good) | | /privacy | 3.1 s (needs improvement) | 3.2 s (needs improvement) | 0.7 s (good) | | /terms | 3.2 s (needs improvement) | 2.7 s (needs improvement) | 0.7 s (good) | Three readings of this table. First, desktop is good everywhere and every lab CLS value was 0.00, so the problem is mobile loading. Second, the field columns show a dash for every page, meaning no real-user data was available, so INP, which exists only as a field value, could not be reported. Third, the lab values moved a lot between two crawls about an hour apart: the home page on mobile went from 3.6 s to 4.1 s and /bot from 3.6 s to 5.4 s, although none of our fixes touched loading. That is why the fix list says to re-test first. The crawl also records the home page HTML at 1,076,839 bytes, just over 1 MB, which makes the document itself a first suspect for the slow mobile load. We have not tested that yet, so it is a hypothesis, not a finding. Our [Core Web Vitals test](https://serpel.app/blog/core-web-vitals-test) guide explains each metric. ## What does an audit report say about indexability and crawl health? Include what passed. A report that lists only problems cannot tell a reader whether the rest was checked. These are the passes in the second crawl. - **Status codes:** all 52 URLs answered 200, with no client errors, server errors or redirects among the crawled pages. - **Indexability:** 46 of the 49 HTML pages are indexable. The other three, /cancel, /login and /signup, are set to [noindex](https://developers.google.com/search/docs/crawling-indexing/block-indexing) on purpose. The three non-HTML URLs are the RSS feed, llms.txt and llms-full.txt. - **Metadata:** every HTML page has a title, one H1, a meta description and itself as canonical. No title is repeated and no image lacks alt text. - **Crawl access and speed:** robots.txt answers 200 and the sitemap at /sitemap.xml is found. An llms.txt file is present, all 13 AI crawlers Serpel checks are allowed, and the average response time was 37 ms. ## How do you read structured data and link findings? ### Structured data: errors fixed, recommendations left The first crawl reported a warning on 3 pages. The home page declared SoftwareApplication markup that lacked a price and either an aggregateRating or a review, and two tool pages declared WebApplication markup that lacked one of those two. We have no customer ratings and will not make any up, so we described the home page as a Product with a price range and the tool pages as plain web pages. The second crawl found no errors. Google’s [introduction to structured data](https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data) explains the markup. Two recommendations remain: the Organization markup on / and /about has no sameAs property, and the Product on / has none of the identifiers (sku, gtin or mpn) that Google recommends. Both are optional, so they are notices. Check your own markup with the free [schema validator](https://serpel.app/tools/schema-validator). ### Links: check a finding before you fix it The link check followed 173 external targets and reported that 13 of our pages link to URLs that redirect. Those pages point to 43 different targets, which looks like a long to-do list. We requested every one of them again with curl and an English Accept-Language header. - **31 targets were fine.** The crawler had been redirected to the same address on developers.google.com, developer.chrome.com or web.dev with ?hl=de appended, and when we asked again each one answered 200 without a redirect. - **12 targets had really moved.** Four were Google crawling documents that now live under a new path, and eight were pages of Perplexity, the RFC Editor, the Model Context Protocol, InfoQ, Anthropic, OpenAI and Cursor. The lesson for your own report is that a finding is a lead. Without the re-check we would have changed 31 links that were already correct. With it, the real task is short: replace 12 targets with their final addresses. Our [redirect checker](https://serpel.app/tools/redirect-checker) shows the chain for a single URL. ## Which fixes come first? A prioritised list End the report with a short, ordered list: what to fix, why, how much work it is. Order by impact on visitors and search engines first, then by effort. This is ours. **Prioritised fixes for serpel.app after the second crawl** | Priority | Fix | Why | Effort | | --- | --- | --- | --- | | 1 | Re-test mobile LCP on / and /bot, then reduce it, starting with the size of the home page HTML | The only warning, and it affects the first page visitors see | Medium | | 2 | Let search engines follow links on /bot | A public page that passes no signals to the pages it links to | One line | | 3 | Replace the 12 moved link targets with their final URLs | Each one costs a redirect, and old addresses can disappear | Low | | 4 | Add sameAs to the Organization markup where we have profiles to list | Completes optional fields on / and /about | Low | | 5 | Revisit render-blocking resources after the LCP re-test | A heuristic that is identical on every page | Low, later | | 6 | Leave noindex on /cancel, /login and /signup | Intended | None | ### What we fixed between the two crawls, and what is still open After the first crawl we fixed four things, and the comparison between the crawls confirms each of them. - **Structured data errors on 3 pages:** resolved, and the structured data notices fell from 7 pages to 2. - **A missing canonical tag on /bot:** resolved. - **A skipped heading level on 2 pages:** resolved. An H3 had followed the H1 on /pricing and on the llms.txt generator page. - **Six meta descriptions that were probably too long:** shortened and resolved. The score rose from 90 to 93. One new warning appeared, the poor mobile LCP described above. None of our changes touched loading, so we read it as measurement variation. At the time of the second crawl everything on the prioritised list was still open. ### The third crawl, after working through the list Later on 10 Oct 2026 we shipped fixes for the top of the list and crawled again. The third crawl scored 95 out of 100 with 0 errors, 0 warnings and 71 notices across 61 pages. - **Mobile LCP of the home page:** 2.0 s in the lab test, rated good, after we stopped preloading a font subset the site did not need, stopped preloading the monospace font and removed a prefetch of the home page from its own logo link. - **/bot:** now indexable and followed. Its mobile LCP was 3.6 s, so it stays on the list as needs improvement. - **Redirecting external links:** down from 13 pages to 3. Serpel’s crawler now sends the project’s language instead of a fixed German Accept-Language header, which removed the language redirects, and we updated the links that had really moved. Two of the three remaining pages link to documentation that sends visitors to its current version with a temporary redirect, which is how that site is meant to work, so we kept the stable address. The third was an RFC link on the schema validator page, which we have since pointed at the final address. ## SEO audit report format: a checklist you can reuse Copy these sections, fill them from your own crawl and add a verdict to every finding. The table shows where each section comes from in Serpel. **Sections of the report and the command that produces each one** | Section | What to include | Serpel command | | --- | --- | --- | | Summary | Date, scope (pages, device, rendering mode), score, issue counts, change since the last crawl | `serpel crawl show <crawl-id>` | | Findings | Issue, severity, category, number of URLs, your verdict | `serpel audit issues --project <project-id>` | | Finding detail | What was found, why it matters, the fix and the affected URLs | `serpel audit issue <code> --project <project-id>` | | Core Web Vitals | LCP, INP and CLS per page, mobile and desktop, lab or field | `serpel audit web-vitals <crawl-id>` | | Comparison | New, resolved and changed issues, and the score before and after | `serpel crawl compare --project <project-id>` | | Export | Issues as CSV or JSON for tickets and spreadsheets | `serpel export --project <project-id> --type issues --format csv` | ```text Site audit report: <domain> Crawled on: <date> Pages: <number> Device: <mobile or desktop> Rendering: <mode> Score: <score>/100 Errors: <number> Warnings: <number> Notices: <number> Change since the last crawl: <score difference> 1. Findings: severity, issue, category, URLs, verdict 2. Core Web Vitals: page, device, LCP, INP, CLS, lab or field 3. Passed checks: status codes, indexability, metadata, canonicals, robots.txt, sitemap, AI crawler access 4. Fix list: priority, fix, reason, effort, owner 5. Changes since the last crawl: new, resolved, changed ``` ## How do you do a website audit, step by step? 1. **Set the scope** Choose the start URL and the page limit. Serpel crawls as a mobile browser and follows internal links up to the limit you set, within your plan’s maximum. 2. **Choose the rendering mode** Automatic renders only pages that arrive almost empty, always renders every page and never audits the delivered HTML only. A server-rendered site like ours needs no rendering. 3. **Run the crawl** A crawl is priced at 1 credit per 20 pages, so a 52-page crawl like ours is priced at 3 credits. Reading the results costs nothing. ```bash serpel crawl start --project <project-id> --wait ``` 4. **Read the summary and the errors first** Start with the score and the issue counts, then list the errors. ```bash serpel audit issues --project <project-id> --severity error ``` 5. **Triage every finding** Open the detail, check heuristics by hand and decide: fix it, keep it because it is intended or re-test it. Write the verdict down. 6. **Fix, crawl again and compare** After your fixes, crawl again and compare the two crawls, so the report shows what is resolved and what is new. ```bash serpel crawl compare --project <project-id> ``` ## What can an SEO audit report not tell you? - **Whether a page deserves to rank.** A crawler checks mechanics, not intent or quality. Read it next to your [Search Console and Bing data](https://serpel.app/features/search-data). - **Anything a heuristic guesses.** Title length, thin content and render-blocking resources are rules of thumb, so check them before you act. - **How real visitors experience speed.** Lab values are one run. Only field data shows what users see. - **Anything after the crawl.** A crawl is a snapshot, so run one after each release. ## Frequently asked questions ### What is an SEO audit? An SEO audit is a structured check of the technical, content and link factors that affect how search engines find, understand and show your pages. A technical audit uses a crawler to find issues such as broken links, noindex pages and slow loading, and the report ranks them so you know what to fix first. ### What should an SEO audit report include? A summary with a score and issue counts, findings grouped by severity and category, Core Web Vitals, indexability and crawl health, structured data, links and a prioritised fix list. Add a comparison with the previous crawl and a verdict for every finding, so a reader can see what you decided and why. ### How do you do a website audit? Crawl the site with a tool that reports issues by severity, read the summary and the errors first, check each finding by hand, fix the highest-impact items and crawl again to confirm. The example above follows these steps on a 52-page site. ### How often should you run an SEO audit? Google gives no schedule, so choose one that matches how often your site changes. Run a crawl after every release and keep a scheduled crawl running, so regressions show up quickly. Serpel can run crawls weekly on a schedule. ### Is a score of 100 the goal of an SEO audit? No. The score is a summary, not a ranking factor, and Serpel’s is its own calculation. A site with a high score can still have a serious problem, such as a poor mobile LCP. Use the score to track progress and the findings to decide what to fix. ## Sources - [Google Search Central: SEO starter guide](https://developers.google.com/search/docs/fundamentals/seo-starter-guide), accessed 2026-10-10 - [web.dev: Web Vitals](https://web.dev/articles/vitals), accessed 2026-10-10 - [Google for Developers: About PageSpeed Insights](https://developers.google.com/speed/docs/insights/v5/about), accessed 2026-10-10 - [Google Search Central: Block search indexing with noindex](https://developers.google.com/search/docs/crawling-indexing/block-indexing), accessed 2026-10-10 - [Google Search Central: Introduction to structured data markup in Google Search](https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data), accessed 2026-10-10 --- # Soft 404 errors: what they are and how to fix them URL: https://serpel.app/blog/soft-404 Updated: 2026-10-10 A soft 404 is a page that tells visitors the content does not exist, or that has almost no content, but still returns a 200 status code. Google excludes these URLs from Search and reports them as “Soft 404” in Search Console. The fix depends on the case: a real 404 or 410 for removed pages, a 301 redirect for moved pages, and real content or a rendering fix for pages that should exist. ## Key takeaways - A soft 404 shows an error message, an empty page or almost no content, but answers with a 200 status. Google excludes such pages from Search. - Common causes are empty category and search pages, out-of-stock products, single-page apps that answer 200 for every route, and broken back ends or blocked scripts. - Fix by case: 404 or 410 for removed content, 301 for moved content, and real content or a rendering fix when the page should exist. - Serpel flags likely soft 404s by comparing pages with the site’s own error page and looking for error phrases, thin text and homepage redirects. Treat the result as a lead, and confirm in Search Console. ## What is a soft 404? Google defines a [soft 404](https://developers.google.com/search/docs/crawling-indexing/troubleshoot-crawling-errors) as a URL that displays a “content doesn’t exist” page but is served with a 200 success status. Some soft 404 pages have no main content at all. When Google’s systems recognise an error page from its content, Search Console lists the URL as “Soft 404” in the Page indexing report. A real 404 is different because the status code itself tells crawlers that the page is gone. **How Google treats the common responses** | Response | Status code | What the visitor sees | What Google does | | --- | --- | --- | --- | | Real 404 | 404 | A “not found” page | Ignores any content. A URL that was indexed is [removed from the index](https://developers.google.com/crawling/docs/troubleshooting/http-status-codes), and Google stops using the URL over time | | Real 410 | 410 | A “gone” page | Treats it like a 404. Google handles all 4xx codes except 429 the same way | | Soft 404 | 200 | A “not found” message, an empty page or almost no content | Detects the problem from the content, reports it in Search Console and excludes the page from Search | | Permanent redirect | 301 | The new page | Ignores the content of the old URL and treats the redirect as a strong signal to process the target | ## Why do soft 404s matter for SEO? - **The page cannot rank.** Pages that Google classes as soft 404 are excluded from Search. That is harmless for a page that should not exist and costly for one that should. - **They waste crawling.** Google’s [crawl budget guide](https://developers.google.com/crawling/docs/crawl-budget) says soft 404 pages continue to be crawled and waste budget. The guide is aimed at very large sites, so for a small site the first point matters more. - **They hide real faults.** A broken database connection, a missing include file or a script that fails to load can turn a good page into a soft 404. - **Monitoring misses them.** A check that only watches status codes sees a healthy `200` while visitors see an error. ## What causes soft 404 errors? Google lists [typical causes](https://developers.google.com/search/docs/crawling-indexing/troubleshoot-crawling-errors): a missing server-side include file, a broken database connection, an empty internal search results page and an unloaded or missing JavaScript file. In practice they show up in six places: - **Thin or empty pages.** A category, tag or filter page with no items renders a template and nothing else. - **Empty internal search results.** Every query gets a page, even one with “no results”. - **Out-of-stock and discontinued products.** The page stays up with an “unavailable” message and little else. - **JavaScript apps that answer 200 for everything.** Client-side routing renders a “page not found” view, but the server already sent a success status. See our guide to [JavaScript SEO](https://serpel.app/blog/javascript-seo). - **Catch-all redirects.** Every missing URL is redirected to the homepage. - **Broken or blocked resources on a page that should exist.** The page renders blank or with an error for Googlebot. ## How does Google Search Console report soft 404s? Open the [Page indexing report](https://support.google.com/webmasters/answer/7440203) and look for “Soft 404” among the reasons pages are not indexed. Google describes it as a request that returns what it thinks is a soft 404 response: a user-friendly “not found” message without a 404 status code. The recommended fix is to return a 404 for pages that are truly not found. If the page is not meant to be a 404, add more information so Google can tell it is a real page. Google’s advice for a flagged URL is to run a live URL Inspection test and open “View tested page” to see how it renders. Check the status code your server sends as well. A random URL that cannot exist should return 404. ```bash curl -s -o /dev/null -w "%{http_code}\n" https://example.com/this-page-does-not-exist-4821 ``` ## How do you fix a soft 404 in each case? Choose the fix from the state of the page and the result you want. The table follows Google’s guidance for [removed, moved and existing pages](https://developers.google.com/search/docs/crawling-indexing/troubleshoot-crawling-errors) and its documents on [e-commerce URLs](https://developers.google.com/search/docs/specialty/ecommerce/designing-a-url-structure-for-ecommerce-sites), [JavaScript](https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics) and [pausing a business](https://developers.google.com/search/docs/crawling-indexing/pause-online-business). **The right response per situation** | Situation | Right response | Why | | --- | --- | --- | | Content removed for good, no replacement | Return `404` or `410`. A custom error page is fine if the server still sends the status | It tells search engines the page does not exist | | Content moved or clearly replaced | Return a `301` to the new URL | A permanent redirect passes the signal to the target | | Product temporarily out of stock | Keep the page and mark it as out of stock | Google’s guidance for pausing a business says it is better to keep the page and mark the product out of stock | | Product discontinued for good | Return `404` or `410`, or `301` to a close replacement | The same rules as for removed and moved content | | Empty category | Add `noindex` while it is empty, or return `404` if the site removes empty categories from browsing and search | Google suggests `noindex` for categories with no items and a 404 when the category is removed | | Internal search with no results | Return `404` for queries without results, or add `noindex` to those pages | Google names empty internal search result pages as a typical cause of soft 404s | | Single-page app route that does not exist | Redirect with JavaScript to a URL that returns `404`, or add `noindex` to the error view | Client-side error views often return `200`, which can get them indexed | | Thin page that should exist | Add real content and a specific title | More information lets Google tell it is not a soft 404 | | Good page flagged by mistake | Check the rendered page in URL Inspection and fix blocked, failing, slow or oversized resources | A blank or error rendering usually comes from resources that did not load | Redirecting every removed URL to the homepage fits none of these cases. Google asks for a 301 when there is a clear replacement and a 404 or 410 when there is none. The free [redirect checker](https://serpel.app/tools/redirect-checker) shows the status code of every hop for a URL you changed. Frameworks need care. In Next.js, the [notFound function](https://nextjs.org/docs/app/api-reference/functions/not-found) renders the 404 page and adds a `noindex` tag. If you call it before streaming starts, the response is a real 404. If it runs after streaming has started, for example inside a Suspense boundary, the response keeps its 200 status and relies on the `noindex` tag. Do the existence check before the page starts streaming. Our [Next.js SEO guide](https://serpel.app/blog/nextjs-seo) covers the rest. ```typescript import { notFound } from 'next/navigation' import { getProduct } from '@/lib/products' export default async function ProductPage({ params }: { params: Promise<{ slug: string }> }) { const { slug } = await params const product = await getProduct(slug) if (!product) { notFound() } return <h1>{product.name}</h1> } ``` ## How can you find soft 404s on your own site? Do not wait for Search Console to report them. A short routine catches most cases: 1. Request a random URL that cannot exist and confirm the status code is 404 or 410. 2. Crawl the site and list pages that answer 200 but contain an error phrase or only a few words. 3. Open an empty category, a search for nonsense and a discontinued product, and read the status code of each response. 4. Check your server logs for 200 responses to URLs that were never part of the site. 5. Review the Page indexing report regularly for new “Soft 404” entries. ## How does Serpel detect likely soft 404s? Serpel can’t see Google’s verdict, so its crawler looks for the same signs from the outside and labels the result as likely. During a crawl it first requests a random URL that cannot exist on the same site, as long as `robots.txt` allows it. That response shows what the site’s real error page looks like: its status, title, first heading and a fingerprint of its text. Then [Serpel’s site audit](https://serpel.app/features/site-audit) checks every page that answers with a 2xx status. It skips the homepage, pages with `noindex` and pages that look like an empty JavaScript shell, because their real content is unknown and they get a rendering note instead. For the remaining pages it looks for five signals: - **Content matches the error page.** The text is practically identical to the response for the random URL. - **Error message in the page.** A phrase such as “not found”, “404”, “no longer available” or “does not exist”, in English or German, appears in the title or first heading of a short page, or in the text of a very short one. - **Same title as the error page.** The title equals the one the random URL returned. - **Very little text.** The page has only a handful of words. - **Redirect to the homepage.** A URL at least two path levels deep redirects to the homepage. Each signal has a weight, and the weights add up to a confidence score. A score of 0.85 or more produces a `soft_404` error, and a score of 0.5 or more produces a `soft_404_possible` warning. A near-identical match with the error page reaches the first level on its own. An error message alone, or a redirect to the homepage alone, reaches the second. The same title and very little text only count together with another signal. Read “possible” findings as leads, not verdicts. The details name the signals and the confidence, so you can see why a page was flagged. ```bash serpel audit issue soft_404 --project <project-id> serpel audit issue soft_404_possible --project <project-id> serpel crawl page <page-id> --crawl <crawl-id> ``` ## Frequently asked questions ### What is the difference between a 404 and a soft 404? A 404 is an HTTP status code that tells crawlers the page does not exist. A soft 404 shows the visitor a “not found” message, an empty page or almost no content while the server still answers with 200, so Google has to work out from the content that the page is an error. ### Do soft 404s hurt SEO? Google excludes pages it classes as soft 404 from Search, so a page that should rank will not. They can also waste crawling, which matters mostly on very large sites. Soft 404s on pages that should not exist are harmless to rankings but still worth cleaning up. ### How do I fix soft 404 errors in Google Search Console? Open the Page indexing report, inspect a flagged URL and check the status code and rendered page. Return 404 or 410 for removed content, 301 for moved content, and add real content or fix blocked resources if the page should exist. ### Should I redirect deleted pages to the homepage? No, unless the homepage is a relevant replacement. Google asks for a 301 when a clear replacement exists and a 404 or 410 when it does not. A blanket redirect is neither, which is why Serpel flags deep URLs that redirect to the homepage as a possible soft 404. ### Is a 410 better than a 404? For Google Search there is no practical difference. Its documentation says all 4xx status codes except 429 are treated the same, and indexed URLs that return them are removed from the index. Use 410 if you want to state clearly that content is gone for good. ## Sources - [Google Search Central: Troubleshoot Google Search crawling errors](https://developers.google.com/search/docs/crawling-indexing/troubleshoot-crawling-errors), accessed 2026-10-10 - [Google Search Central: HTTP status codes, network and DNS errors, and Google Search](https://developers.google.com/crawling/docs/troubleshooting/http-status-codes), accessed 2026-10-10 - [Search Console Help: Page indexing report](https://support.google.com/webmasters/answer/7440203), accessed 2026-10-10 - [Google Search Central: Managing crawl budget for large sites](https://developers.google.com/crawling/docs/crawl-budget), accessed 2026-10-10 - [Google Search Central: Understand JavaScript SEO basics](https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics), accessed 2026-10-10 - [Google Search Central: Ecommerce URL structure best practices](https://developers.google.com/search/docs/specialty/ecommerce/designing-a-url-structure-for-ecommerce-sites), accessed 2026-10-10 - [Google Search Central: Temporarily pause or disable a website](https://developers.google.com/search/docs/crawling-indexing/pause-online-business), accessed 2026-10-10 - [Next.js documentation: notFound](https://nextjs.org/docs/app/api-reference/functions/not-found), accessed 2026-10-10 --- # About Serpel URL: https://serpel.app/about Updated: 2026-10-10 Serpel is an SEO tool for developers. It tracks Google rankings, audits every page of a website, reads Search Console and analytics data, and checks whether ChatGPT and Google AI Overviews cite you. Use it from the dashboard, the CLI, the REST API or your coding agent over MCP. ## What is Serpel? Serpel gives developers and coding agents the SEO data that marketing teams usually get from a dashboard. It covers: - **Rank tracking** on Google in 43 countries, on desktop and mobile, down to position 100. - **Site audits** with 94 checks and JavaScript rendering. - **AI visibility:** whether ChatGPT and Google AI Overviews cite you, and which of 13 AI crawlers can reach your site. - **Search data** from Google Search Console, Bing Webmaster Tools and web analytics. ## Why does Serpel exist? Developers now do much of their SEO work in the editor and the terminal, and more of it is delegated to coding agents. That work needs data it can call, script and budget, not only charts in a browser tab. Search is changing as well. More questions are answered directly by ChatGPT or Google’s AI Overviews, so a Google rank no longer tells the whole story. Serpel tracks both. ## What can you check for yourself? - **Pay per credit, no subscription needed.** A credit is worth €0.01 including VAT, and new accounts start with 100. A rank check right now costs 5 credits per keyword, a ChatGPT answer check 2 and a crawl 1 per 20 pages. Every price is on the public [price list](https://serpel.app/pricing), reading your data is free, and the History page shows what each job cost. Starter and Pro are optional plans with a monthly allowance. - **Data you can take with you.** Export issues, keywords, rankings and tasks as CSV from the dashboard, or as CSV or JSON through the CLI and the API. In the settings you can also download your account data or delete your account. - **The same capabilities everywhere.** The dashboard, the REST API and the CLI work on the same data, and the CLI covers almost every API endpoint with JSON output. The [MCP server](https://serpel.app/developers/mcp) gives coding agents 13 tools. Paid tools run only within a daily budget you approve, on a Starter or Pro plan. - **European data storage.** Your account and project data sit in a database in Frankfurt, and the application runs there too. The background worker that runs crawls and checks sits on a server in Germany. Some providers are US companies, so the [privacy policy](https://serpel.app/privacy), which is in German, lists each provider with its location and safeguards. - **A crawler that identifies itself.** SerpelBot only visits sites that someone added as a project, follows robots.txt and crawl-delay, and keeps its request rate low. The [crawler page](https://serpel.app/bot) explains what it fetches and how to block it. ## Who runs Serpel? Serpel is operated by Kerem Sinecek, as stated in the [imprint](https://serpel.app/imprint) (Impressum). The imprint also gives the postal address and the other provider details that German law requires. Serpel is in public beta. Features can change, and there is no promise of a specific availability. The [changelog](https://serpel.app/changelog) records what changed and when. The legal pages are in German, and the German version is the binding one. ## How can you contact Serpel? Write to [support@serpel.app](mailto:support@serpel.app) with questions about Serpel, your account, billing, the crawler or your data. This is the contact address given in the [imprint](https://serpel.app/imprint). ## Frequently asked questions ### Who is behind Serpel? Serpel is operated by Kerem Sinecek, as stated in the [imprint](https://serpel.app/imprint), which also gives the postal address and contact details. Serpel is operated from Germany. ### Is Serpel free to use? You can start without a subscription: new accounts get 100 credits, and you top up when you need more. Reading your data is free, and every paid action is on the public [price list](https://serpel.app/pricing). Starter and Pro are optional plans with a monthly credit allowance. ### Where is my data stored? Your account and project data are stored in a database in Frankfurt, Germany, and the background worker runs on a server in Germany. Some providers are US companies. The privacy policy, which is in German, lists every provider with its location and safeguards. ### How do I recognise the SerpelBot crawler? Its user agent contains SerpelBot. It only visits sites that someone added as a project, and it follows robots.txt. To block it, add `User-agent: SerpelBot` and `Disallow: /` to your robots.txt, as the [crawler page](https://serpel.app/bot) explains. --- # Serpel changelog URL: https://serpel.app/changelog ## 2026-10-10: English everywhere, a section for coding agents and serpel.app The whole product now speaks English, serpel.app is the single address for everything, and the website gains a section for coding agents. - new: serpel.app is the address of Serpel: website, dashboard, API and MCP server share one domain, and links to earlier addresses redirect there. - new: The website has a new “For coding agents” section with three example terminal sessions in Claude Code, Cursor and Codex, plus the facts on MCP sign-in and budgets. - improved: The dashboard, emails, API messages and CLI are fully in English, with British spelling and dates such as 10 Oct 2026. The legal pages and the statutory cancellation label “Verträge hier kündigen” stay in German. - fixed: Menus, the project switcher, the command palette, tooltips and the save bar are opaque instead of see-through, so the sidebar no longer shows through them. - fixed: Dropdown lists inside dialogs, such as the schedule pickers for report emails, open on top of the dialog instead of behind it. - fixed: The links “Set up competitors” and “Email settings” in report emails lead to the right pages. Both used to point at sections that did not exist. - improved: The Serpel logo on sign-in and standalone pages leads back to the home page, and the sidebar group above History is titled Activity instead of leaving a gap. - improved: “Verträge hier kündigen” sits in the website footer as a quiet text link next to the copyright line and stays reachable from every page. ## 2026-10-09: Deeper audits, an MCP server for coding agents and plan options The crawler renders JavaScript and runs far more checks, coding agents can connect over MCP, and the pricing gains Starter and Pro plans with a monthly credit allowance. - new: JavaScript rendering: when a server returns an empty app shell, the crawler renders the page in a headless browser and audits what visitors actually see. - improved: The site audit grows from 43 to 94 checks, including hreflang, structured data, soft 404 pages, link checks and Core Web Vitals measured through PageSpeed. - new: The MCP server lets Claude Code, Cursor, Codex and other agents sign in with OAuth and use 12 tools for rankings, search data, crawl issues, keyword research and competitors. - new: Paid agent tools run only within a budget you approve, with a daily credit limit that you control. - new: The CLI can scan a Next.js project and send a snapshot of its routes and metadata to Serpel, which agents can read over MCP. - new: Recommendations come with evidence, data sources and a next action, and you can accept, dismiss or complete each one. - new: Starter and Pro plans with a monthly credit allowance are defined next to pay-per-credit use. Online payment opens later in the public beta. - improved: Scheduled monitoring runs in a queue at a lower price than a rank check right now. ## 2026-10-08: A new dashboard with activity history and competitor comparison The dashboard is rebuilt with a new sidebar and command palette, and the History page shows what every job cost. - new: The dashboard is rebuilt with a new sidebar and a command palette that offers commands for every page. - new: History lists jobs, sent report emails and keyword researches with the credits each one cost. Open an entry to see its details, cancel it or run it again. - new: Competitors are compared keyword by keyword. - improved: Search data shows a daily trend and compares it with the previous period, and web analytics adds entry pages and devices. - improved: Slack and Discord are separate webhooks with a channel name, and History shows how long researches and emails took. - new: A new audit finding flags a missing llms.txt file. - improved: Crawl progress shows an estimated page count while the crawl is queued and running. ## 2026-10-07: Monitoring, AI visibility and email reports Serpel can now monitor rankings on a schedule, check ChatGPT and Google AI Overviews, and send you reports. - new: Monitoring runs produce reports that arrive by email or webhook, and every report email links to an unsubscribe page. - new: AI visibility checks whether ChatGPT with web search and Google AI Overviews cite your site for the questions you track. - new: Local rankings, search volume per keyword, saved search results and competitor tracking. - new: Connections for Google Search Console, Bing Webmaster Tools and web analytics. - new: Every crawl gets an audit score, and you can compare two crawls. - improved: The CLI now covers almost the whole API. - fixed: Live rank checks are no longer cut off after 60 seconds, and partial Google results are used instead of failing the check. - fixed: The CLI says that nothing matches your search instead of claiming that you have no projects or rankings yet. ## 2026-10-06: Project icons and steadier page loads Projects show the icon of their website, and pages load reliably when many people use the dashboard at once. - new: Each project shows the icon of its website in the project switcher, in search and in the project overview. - improved: The crawler picks the best icon from the homepage after every crawl, and projects without one show a globe. - fixed: Pages could fail with a database connection error when many people navigated at the same time. Serpel now limits connections and retries them. ## 2026-10-05: First working version The first build of Serpel combines a dashboard, a REST API, a CLI and a crawler. - new: A dashboard for keyword research, website audits, rank tracking and optimisation tasks. - new: A crawler that audits a website against 43 checks. - new: A REST API under /api/v1 that you call with API tokens. - new: A command-line tool that signs in through your browser and prints JSON with the --json option. - new: A background queue that runs crawls and rank checks.