Method and limits
How Zyntlox reaches its results
The Zyntlox Generative Index (ZGI) is a number from 0 to 100 for how reachable and machine-readable a site is for AI search and answer engines. It is computed by fixed arithmetic from test results. No AI model writes a score or a finding. This page is current for ZGI version 2.7.
How the score is calculated
- Every test returns one of six states: pass, warn (partial credit), fail, inconclusive (could not be determined), not applicable or not tested.
- A test counts toward the score only if it has a weight above zero, applies to the site, and produced pass, warn or fail. Inconclusive, not tested and not applicable tests are left out and shown as coverage. They are never counted as passed.
- Inside a category, the category score is the weighted share of credit earned: Σ(test weight × credit) ÷ Σ(test weight), over the tests that counted.
- The ZGI is Σ(category weight × category score). If a whole category has nothing to score, the others are rescaled to 100.
- The points shown next to each test add up to the ZGI.
Every report states the formula version and the time the tests ran. Any change to a weight, severity, test, threshold or crawler list bumps the version, so a score can always be traced to the formula that produced it. Scores from different versions may not be comparable.
Categories and weights
The category weights are product decisions, not measurements. A site that only leaves crawlers allowed earns most of Crawler access; that is intended, because access is the precondition, not the whole score.
| Category | Weight | Scored tests | What it covers |
|---|---|---|---|
| Crawler access | 30 | 15 | Whether AI and search crawlers are permitted and able to fetch the site. |
| Entity graph | 35 | 5 | Whether the site states, in machine-readable form, who or what it is. |
| Machine context | 20 | 7 | Files and page elements that help machines discover and understand the site. |
| Citability | 15 | 1 | How much concrete, quotable data the page text contains. |
Every test (61)
28 tests are scored and 33 are reported only. Severity is fixed per test and is never chosen by a model. Open a category to see what each test expects and what counts as a pass.
Crawler access25 tests · 30% of the score
Googlebot is permitted to fetch "/".
Pass: The matching robots.txt group allows "/" for Googlebot, or the site has no robots.txt (HTTP 4xx).
OAI-SearchBot is permitted to fetch "/".
Pass: The matching robots.txt group allows "/" for OAI-SearchBot, or the site has no robots.txt (HTTP 4xx).
Bingbot is permitted to fetch "/".
Pass: The matching robots.txt group allows "/" for Bingbot, or the site has no robots.txt (HTTP 4xx).
The server answers crawler user agents the way it answers a browser.
Pass: Every considered crawler request returns 2xx (redirects allowed).
The final response is HTTP 2xx with an HTML content type.
Pass: HTTP 2xx and a text/html (or XHTML) content type.
Neither the HTML nor the headers tell crawlers to keep the page out of their index.
Pass: No noindex/none directive was found.
Claude-SearchBot is permitted to fetch "/".
Pass: The matching robots.txt group allows "/" for Claude-SearchBot, or the site has no robots.txt (HTTP 4xx).
PerplexityBot is permitted to fetch "/".
Pass: The matching robots.txt group allows "/" for PerplexityBot, or the site has no robots.txt (HTTP 4xx).
The page is served over HTTPS with TLS 1.2 or 1.3.
Pass: TLS 1.2 or 1.3 was negotiated.
The certificate chain is trusted and valid for at least 14 more days.
Pass: Trusted certificate with 14 or more days left.
Plain HTTP requests are permanently redirected (301 or 308) to HTTPS.
Pass: http:// redirects to https:// with a permanent redirect.
The page is reached with at most 1 redirect and no loop.
Pass: 0 or 1 redirect.
ChatGPT-User is permitted to fetch "/".
Pass: The matching robots.txt group allows "/" for ChatGPT-User, or the site has no robots.txt (HTTP 4xx).
Claude-User is permitted to fetch "/".
Pass: The matching robots.txt group allows "/" for Claude-User, or the site has no robots.txt (HTTP 4xx).
The page declares a canonical URL that points to itself (or to another page of the same site, deliberately).
Pass: A canonical link is present and matches the page URL.
GPTBot is permitted to fetch "/".
Pass: The matching robots.txt group allows "/" for GPTBot, or the site has no robots.txt (HTTP 4xx).
ClaudeBot is permitted to fetch "/".
Pass: The matching robots.txt group allows "/" for ClaudeBot, or the site has no robots.txt (HTTP 4xx).
Google-Extended is permitted to fetch "/".
Pass: The matching robots.txt group allows "/" for Google-Extended, or the site has no robots.txt (HTTP 4xx).
The median time to first byte is at most 800 ms.
Pass: Median at most 800 ms.
The HTML is served compressed (gzip, br, zstd or deflate).
Pass: A compression Content-Encoding is present.
The response carries at least one cache validator or directive.
Pass: At least one of Cache-Control, ETag, Last-Modified.
The file is no larger than 500 KiB.
Pass: The file is 500 KiB or smaller.
A page that does not exist answers 404 or 410.
Pass: The made-up path answers HTTP 404 or 410.
The page's main text is present in the HTML as the server sends it.
Pass: At least half of the rendered text is already in the served HTML (or, with no rendering to compare, the served HTML holds a substantial amount of text and no empty app shell).
Neither the HTML nor the headers forbid text snippets.
Pass: No nosnippet or max-snippet:0 directive was found.
Entity graph11 tests · 35% of the score
The page declares the organization behind the site in JSON-LD.
Pass: An Organization/Corporation entity is present in valid JSON-LD.
The entity links to its other public profiles.
Pass: At least one sameAs reference is present in the JSON-LD.
Every JSON-LD block is syntactically valid JSON.
Pass: All JSON-LD blocks parse.
Question-and-answer content is marked up when the page has any.
Pass: FAQPage markup is present.
The Organization declares a name, a url and a logo.
Pass: name, url and logo are all present.
The Organization has an @id (a stable identifier other entities can refer to).
Pass: An @id is present.
The page describes one organization.
Pass: At most one distinct Organization entity.
Reported for information.
Pass: At least one schema.org type was found.
The article names an author and links to a profile.
Pass: Every article declares an author with a name and a url or sameAs link.
The page links to an About page and a Contact page.
Pass: Both links were found.
The Organization name appears in the title, og:site_name or H1.
Pass: The name appears in at least one of them.
Machine context24 tests · 20% of the score
robots.txt points crawlers to the site's sitemap.
Pass: At least one Sitemap: line is present.
The sitemap returns HTTP 200, is well-formed XML with a <urlset> or <sitemapindex> root, and lists at least one URL (at most 50,000 per file).
Pass: HTTP 200, well-formed XML, known root element, 1 to 50,000 URLs.
Sitemap URLs return HTTP 2xx directly (no redirect, no 404/410).
Pass: Every sampled URL with a decisive answer returned HTTP 2xx.
The page has a non-empty <title>.
Pass: A non-empty title is present.
Every <lastmod> is a valid W3C Datetime (YYYY, YYYY-MM, YYYY-MM-DD or a full timestamp with a time zone).
Pass: All lastmod values present are valid.
The page has a non-empty meta description.
Pass: A non-empty meta description is present.
The page has exactly one non-empty H1.
Pass: Exactly one H1.
A /llms.txt file is published.
Pass: /llms.txt returns HTTP 200.
Reported for information. No AI provider documents a size threshold.
Pass: The page HTML was measured.
Every hreflang code is well formed, the page lists itself, and no code points to two different URLs.
Pass: All codes are well formed, the page references itself, and there are no conflicts.
No empty headings and no skipped levels.
Pass: No empty headings and no skipped levels.
Every image has an alt attribute.
Pass: Every image has an alt attribute.
The page uses <main> or <article> to mark its main content.
Pass: <main> or <article> is present.
Links are described by their text.
Pass: No generic or empty link text.
Reported for information.
Pass: At least 150 words of readable text.
No stretch of text runs longer than 300 words without a heading.
Pass: Every stretch between headings is 300 words or fewer.
A paragraph of 20 to 100 words follows the H1.
Pass: The first paragraph after the H1 is 20 to 100 words long.
Internal links lead to pages that exist.
Pass: No crawled internal link answered 404 or 410.
Internal links point straight at the final URL.
Pass: No crawled internal link redirects.
Each page declares itself as its canonical URL.
Pass: Every crawled page with a canonical points to itself.
Each page is linked under a single URL.
Pass: No page is linked under more than one spelling.
Every page in the sitemap is linked from another page.
Pass: Every sitemap URL was found as a link.
Pages in the sitemap can be indexed.
Pass: No crawled sitemap page carries noindex.
Few internal links carry query parameters, or those pages declare the clean URL as canonical.
Pass: Fewer than 5 internal URLs, or fewer than 20% of them, carry query parameters.
Citability1 tests · 15% of the score
The page contains concrete figures that can be quoted.
Pass: 5 or more figures. Fewer earns proportional credit (n ÷ 5).
The crawlers Zyntlox simulates
For each crawler below, the audit sends one request with that crawler's user-agent string and compares the answer with a request from a normal browser. The details come from each provider's own documentation, last checked 2026-11-07.
- GPTBot · OpenAIProvider documentation
Crawls content that may be used to train OpenAI's generative AI foundation models.
User-agent string follows an example in the provider's documentation.
- OAI-SearchBot · OpenAIProvider documentation
Surfaces websites in ChatGPT's search features. Controls search eligibility, separately from GPTBot.
The exact string was not published, so it is built from the provider's token and naming convention; the real crawler's version number may differ.
- ChatGPT-User · OpenAIProvider documentation
Fetches a page when a ChatGPT user or Custom GPT action asks for it. Not an automatic crawler.
User-agent string follows an example in the provider's documentation. This fetcher acts on a person's request and may ignore robots.txt by design.
- ClaudeBot · AnthropicProvider documentation
Collects web content that could contribute to model training.
The exact string was not published, so it is built from the provider's token and naming convention; the real crawler's version number may differ.
- Claude-SearchBot · AnthropicProvider documentation
Navigates the web to improve the quality of Claude search results.
The exact string was not published, so it is built from the provider's token and naming convention; the real crawler's version number may differ.
- Claude-User · AnthropicProvider documentation
Retrieves a page when a Claude user asks a question that needs it. User-initiated.
The exact string was not published, so it is built from the provider's token and naming convention; the real crawler's version number may differ.
- Googlebot · GoogleProvider documentation
Google Search crawler; its index also serves Google's AI search features.
User-agent string follows an example in the provider's documentation.
- Bingbot · MicrosoftProvider documentation
Bing's crawler; discovers and indexes pages for Bing Search, which also feeds Copilot.
The exact string was not published, so it is built from the provider's token and naming convention; the real crawler's version number may differ.
- PerplexityBot · PerplexityProvider documentation
Surfaces and links websites in Perplexity search results. Perplexity states it is not used to crawl content for foundation models.
User-agent string follows an example in the provider's documentation.
What the score does not say
- Crawler access is what robots.txt permits, plus a simulation. It does not show that any AI engine visits, indexes or cites the site.
- The simulation cannot detect firewall rules that check a crawler's IP address against the provider's published ranges, so a real crawler may be treated differently from a simulated request.
- Structured data does not guarantee citations or rankings.
- When a page could only be fetched without running scripts, structured data added by scripts is invisible. In that case a missing Organization or FAQ entry is reported as inconclusive, not as a failure.
- Signals that no major AI engine has confirmed it uses, such as /llms.txt, are checked and shown but never scored.
- Blocking training crawlers (GPTBot, ClaudeBot, Google-Extended) is reported but does not lower the score. It is a site owner's choice.
Judgement calls we made
These are Zyntlox's choices, not facts about any AI engine. They are stated so you can disagree with them.
- The category weights, each test's weight and each severity.
- Full credit for quantitative figures at 5 figures in the page text.
- A text block is flagged above 300 words without a heading.
- A short summary after the main heading should run 20 to 100 words.
- Query-parameter links are flagged at 5 URLs or 20% of the pages tested.
How Zyntlox scans a site
- The audit requests the page you enter, /robots.txt, /llms.txt, the sitemap, and a sample of linked pages (25 by default, 50 at most), fetched without running scripts. Where possible the main page is loaded in a headless browser so that content added by scripts is visible; if that is slow or unavailable, it is fetched without scripts instead.
- Its own checks identify themselves with the user agent
Mozilla/5.0 (compatible; ZyntloxAudit/x.y; +https://zyntlox.com), where x.y is the version of that check. The simulated-access test sends each crawler's own string, listed above. - Each audit is bounded: a fixed number of requests and a time budget, and audits are rate limited per visitor. Private, internal and local addresses are refused, including when a redirect leads to one. Zyntlox does not log in to sites.
- If you run a site and want us to stop scanning it, write to hello@zyntlox.com.