AgentFriendlyRankBot

How it identifies itself

Every request carries this user agent, unmodified:

AgentFriendlyRankBot/1.0 (+https://agentfriendlyrank.com/bot)

We never impersonate a browser or another crawler to get around a block. If you have blocked us, we stay blocked.

What it fetches

Your homepage plus up to five additional pages chosen from your sitemap and navigation. We prefer product, pricing, docs, contact and about routes. Alongside those we probe for robots.txt, sitemap.xml, llms.txt, llms-full.txt, agents.json and /.well-known/security.txt.

Pages are fetched twice, once as raw HTML and once rendered in a headless browser, and once more under the GPTBot, ClaudeBot and PerplexityBot user agents so we can record what those receive. That is the whole crawl. We do not follow every link, we do not spider your site, and we do not return unless someone asks for a re-scan.

Deeper crawls are a paid engagement, not a default

The six-page cap is a design decision rather than a limitation. A free scanner has no business consuming your bandwidth at depth. If you want a full-site audit, that is the paid audit, and we agree the scope with you first.

How to block us

We obey robots.txt for our own user agent, checked before every crawl. To block us entirely:

User-agent: AgentFriendlyRankBot
Disallow: /

That is honoured automatically and immediately: no form, no email, no waiting. To remove an existing report as well, claim the site with an address on your domain and hide it yourself, or email hello@agentfriendlyrank.com and we will take it down within 48 hours.

If we get something wrong

Report a scoring error or crawler misbehaviour to hello@agentfriendlyrank.com. We publish corrections rather than quietly amending them, and the rubric changelog records every change to how scoring works.