Rubric v1.0.1
How we score agent readiness
The whole rubric, published. 27 checks across four pillars worth 25 points each. Every check is automatable from an unauthenticated crawl and returns evidence we captured, never an opinion.
We publish this for a simple reason: a score you cannot audit is a score you should not trust.
Discovery & access
25 pointsCan agents find and reach the content at all?
| ID | Check | Points | Fix effort | How it is evaluated |
|---|---|---|---|---|
| A1 | robots.txt exists and parses | 2 | Low | Returns 200 with valid directives. A 404 scores partial credit — the site is open by default, but the omission is sloppy. |
| A2 | Major AI crawlers allowed | 6 | Low | Allow/deny tested for GPTBot, OAI-SearchBot, ClaudeBot, Claude-User, PerplexityBot and Google-Extended. Scored proportionally to how many are permitted. A blanket Disallow for all six scores zero. |
| A3 | XML sitemap valid and referenced | 3 | Low | One point for existing, one for parsing with at least one URL, one for being referenced from robots.txt. |
| A4 | llms.txt present | 4 | Low | An llms.txt at the root with three or more described links scores three points; an llms-full.txt adds one. An empty or boilerplate file scores one. |
| A5 | Bot user-agent HTTP health | 3 | Low | All three bot user agents receive a 200 rather than a 403 or 429, plus one point for a median TTFB under 1.5 seconds. This is where edge and CDN blocking surfaces, regardless of what robots.txt says. |
| A6 | No hard interstitial | 2 | Medium | The rendered page is not dominated by a CAPTCHA, challenge screen or cookie wall — detected from known challenge markup and from any overlay covering more than half the viewport. |
| A7 | Server-rendered content parity | 5 | High | Visible text in the raw HTML compared against the fully rendered DOM. A ratio at or above 0.8 scores full marks; below 0.5 scores zero, because an agent using a plain fetch sees essentially nothing. |
Understanding
25 pointsOnce fetched, can a machine parse what this site is and sells?
| ID | Check | Points | Fix effort | How it is evaluated |
|---|---|---|---|---|
| B1 | JSON-LD present and valid | 5 | Medium | At least one parseable application/ld+json block on the homepage, and on half or more of the crawled pages. Invalid JSON scores zero for that page. |
| B2 | Correct schema types for the site type | 5 | Medium | Organization or WebSite sitewide, plus type-specific markup: Product and Offer with price and availability for commerce; Article, FAQPage or SoftwareApplication for SaaS and content sites. |
| B3 | Semantic HTML structure | 4 | Medium | Exactly one h1 per page, no skipped heading levels, and real landmarks — main, nav and footer, or their ARIA equivalents. |
| B4 | Meta and Open Graph | 3 | Low | A unique title of 15–70 characters, a meta description of 50–160, and og:title, og:description and og:image all present. |
| B5 | Clean text ratio | 3 | Medium | Visible text as a proportion of HTML bytes, excluding the contents of script and style elements. At or above 0.10 scores full marks, 0.05–0.10 scores two. This penalises div soup and megabyte pages. Script contents are excluded deliberately: server-rendering frameworks inline a serialised payload that duplicates the page text, and counting it would penalise the server-rendered architecture check A7 rewards. |
| B6 | Alt text coverage | 2 | Low | Content images — icons under 50px excluded — carrying non-empty alt text. 80% or better scores full marks. |
| B7 | Canonical and duplication hygiene | 3 | Low | A self-consistent rel=canonical, plus www/non-www and http/https all resolving to one canonical host. |
Actionability
25 pointsCan an agent actually do things — navigate, search, fill forms, transact?
| ID | Check | Points | Fix effort | How it is evaluated |
|---|---|---|---|---|
| C1 | Form usability | 5 | Medium | Every input carries a label or aria-label, input types are correct (email, tel, number), and submit is a real button. A site with no forms at all scores partial credit. |
| C2 | Navigable link graph | 4 | Medium | Navigation uses real anchor elements rather than click-handled divs, fewer than 5% of links use contextless text like 'click here', and there are breadcrumbs or a clear URL hierarchy. |
| C3 | Interactive element semantics | 4 | Medium | Buttons are button elements rather than clickable divs, interactive elements are focusable with sane tab order, and no keyboard traps appear under render. |
| C4 | Machine endpoints and agent manifests | 5 | High | Any of an agents.json, an MCP or WebMCP manifest, a published API documentation link, or a discoverable OpenAPI spec. More than one scores full marks. Most sites score zero here today — which is precisely why it separates the leaders. |
| C5 | Commerce actionability | 4 | High | Commerce sites only. Product pages within two clicks of the homepage, price and availability in schema matching the page, a cart or checkout path returning 200 without login, and a guest or express checkout signal. Redistributed across C1–C4 for non-commerce sites. |
| C6 | On-site search | 3 | Medium | A search form exists, and it uses GET with a query parameter so an agent can construct search URLs directly rather than having to drive a form. |
Trust & freshness
25 pointsShould an agent — or the model behind it — trust and cite this site?
| ID | Check | Points | Fix effort | How it is evaluated |
|---|---|---|---|---|
| D1 | Transport security | 4 | Low | Valid TLS, an HSTS header, and no mixed content under render. |
| D2 | Verifiable identity | 4 | Low | A contact page or discoverable contact details, plus Organization schema with sameAs links to at least one social or registry profile. |
| D3 | Policies discoverable | 4 | Low | A linked privacy policy, plus shipping and returns pages for commerce sites or a terms page otherwise. |
| D4 | Freshness signals | 4 | Medium | Sitemap lastmod present, at least 20% of URLs modified in the last 90 days, and visible dates on articles or a changelog. |
| D5 | Identity consistency | 3 | Low | The site name agrees across the title element, og:site_name and the schema name, and both favicon and og:image resolve. |
| D6 | security.txt | 2 | Low | A valid /.well-known/security.txt per RFC 9116, including a future-dated Expires field. |
| D7 | Multi-page stability | 4 | Medium | Every crawled page returns 200, no page exceeds 4 seconds TTFB, and template and navigation stay consistent across pages. |
Grading
| Grade | Score |
|---|---|
| A+ | 95–100 |
| A | 85–94 |
| B | 70–84 |
| C | 55–69 |
| D | 40–54 |
| F | 0–39 |
How fixes are prioritised
Every failed check carries an impact (roughly its point weight) and an effort: low for a file you can upload, medium for template and schema work, high for rendering architecture or checkout. Reports rank the top five by impact descending and effort ascending, so the cheap, high-value work is obvious.
Versioning and disputes
Every scan stores its rubric version. Standards are moving: llms.txt adoption, WebMCP, agentic commerce protocols. So the rubric will change, but historical scores are never rewritten. If you think a check scored your site wrongly, tell us and we will publish the correction.
Rubric changelog
- v1.0.1
B5 (clean text ratio) now excludes the contents of script and style elements from the denominator. Found by auditing our own site: modern server-rendering frameworks inline a serialised payload duplicating the page text, which on our homepage was 60% of the document. Measured against total bytes, the check penalised precisely the server-rendered architecture that check A7 rewards — so it was measuring framework choice rather than markup quality. Scores issued under v1.0.0 are not restated.
- v1.0.0
Initial published rubric. Four pillars, 25 points each, 27 checks.