Can AI answer engines read your website?
Zyntlox checks what your site permits AI crawlers to read and how clearly it describes itself: robots.txt rules, simulated crawler requests, sitemaps and structured data. Every finding comes with its evidence and a way to check it yourself. No AI model writes the results.
What Zyntlox checks
Four scored categories, and signals we only report
28 tests move the score, each with a fixed weight and severity. A further 33 are reported for your information and never change it.
Crawler access
Whether search and retrieval crawlers are allowed by robots.txt and actually served when they ask: a simulated request for each one, HTTPS and certificate, redirects, and whether the page is marked noindex.
15 scored tests
Entity graph
Whether the site says clearly who it is in structured data: Organization details, sameAs links to your other profiles, FAQPage markup, and whether the JSON-LD is valid.
5 scored tests
Machine context
How easy the site is to map and read: a valid sitemap that is declared in robots.txt and lists healthy URLs, and a page title, meta description and a single main heading.
7 scored tests
Citability
Whether the page text contains concrete, quotable figures such as percentages, amounts and counts, which are easier to cite than general statements.
1 scored tests
Reported, never scored
33 signalsBlocking training crawlers (GPTBot, ClaudeBot, Google-Extended), /llms.txt, page weight, response time and compression, heading and image-alt structure, and checks across the pages Zyntlox crawls. They are shown with the same evidence as every other test, but they stay out of the score because they are a site owner's choice or have not been confirmed by AI engines.
How it works
Three steps, no setup
From an address to a list of findings you can verify.
Enter a public address
Paste the address of any website a visitor can open. Zyntlox refuses private and internal addresses.
Zyntlox runs the tests
It reads robots.txt, sends a simulated request as each crawler, and checks the sitemap, the server's responses, structured data and a sample of linked pages (25 by default). No AI model is involved.
Review the evidence
Every finding shows what was expected, what was observed and how to check it yourself. Download the findings as CSV or JSON, or export a PDF report.
Example report
What a finished audit looks like
This is an example for a fictional business, northwind-studio.example. It was produced by the same engine and tests as a live audit, from fixed inputs, and it is not a real site. Open any test to see its evidence.
ZGI™ Master Score
Category Score Distribution (% of each category)
How this score was calculated
ZGI v2.7 · tested 2026-11-01| Category | Category score | Points earned |
|---|---|---|
| Crawler access | 99% | 29.6 / 30.0 |
| Entity graph | 25% | 8.8 / 35.0 |
| Machine context | 97% | 19.4 / 20.0 |
| Citability | 60% | 9.0 / 15.0 |
| Total | 67 / 100 |
Each test has a fixed weight inside its category; tests that do not apply to this site, or could not be verified, are excluded from the score and shown in coverage instead. Blocking training crawlers (GPTBot, ClaudeBot, Google-Extended) is reported but does not lower the score. Full method: docs/zgi-methodology.md.
AI crawler access by engine
OAI-SearchBot: allowed · ChatGPT-User: allowed · GPTBot: blocked
PerplexityBot: allowed
Googlebot: allowed. Google-Extended is a separate control token for Gemini training and is not part of this status
Claude-SearchBot: allowed · Claude-User: allowed · ClaudeBot: allowed
Simulated crawler access
Each row is one request sent to https://northwind-studio.example/ with that crawler's user-agent string, compared with a normal browser request. It shows how the server answers a request that looks like the crawler. It does not prove the real crawler visits, and real crawlers may be treated differently (for example, firewalls that check provider IP ranges).
| Crawler | Result | HTTP | Time | robots.txt | Evidence |
|---|---|---|---|---|---|
| GPTBot | Blocked by robots.txt | 200 | 360 ms | blocked | robots.txt disallows this crawler. For comparison the simulated request got HTTP 200. (server: cloudflare) |
| OAI-SearchBot | Accessible | 200 | 360 ms | allowed | HTTP 200 for the simulated crawler request. (server: cloudflare) |
| ChatGPT-User | Accessible | 200 | 360 ms | allowed | HTTP 200 for the simulated crawler request. (server: cloudflare) |
| ClaudeBot | Accessible | 200 | 360 ms | allowed | HTTP 200 for the simulated crawler request. (server: cloudflare) |
| Claude-SearchBot | Accessible | 200 | 360 ms | allowed | HTTP 200 for the simulated crawler request. (server: cloudflare) |
| Claude-User | Accessible | 200 | 360 ms | allowed | HTTP 200 for the simulated crawler request. (server: cloudflare) |
| Googlebot | Accessible | 200 | 180 ms | allowed | HTTP 200 for the simulated crawler request. (server: cloudflare) |
| Google-Extended | Not tested | — | — | allowed | This is a robots.txt control token, not a crawler. It sends no requests, so only its robots.txt permission applies. |
| Bingbot | Accessible | 200 | 180 ms | allowed | HTTP 200 for the simulated crawler request. (server: cloudflare) |
| PerplexityBot | Blocked by server/CDN | 403 | 180 ms | allowed | The crawler user agent got HTTP 403 while a normal browser user agent got HTTP 200. Response carried: server: cloudflare. (server: cloudflare) |
Test results
52 tests in total: 26 are scored (these match the score breakdown) and 26 are reported but never change the score. Click a row for the full explanation.
- Passed: 44
- Warning: 3
- Failed: 5
- Not verified: 0
Emerging signals
Not scoredConventions that no major AI engine has confirmed it uses yet. They are checked so you can prepare, and they never change the score.
Page weight for AI
Estimate · not scoredThe rendered page is about 423 tokens (0.3% of an illustrative 128,000-token context window). The readable text in it is about 234 tokens, 55% of the HTML.
Tokens are estimated as characters / 4; real tokenizers differ by model and language. Many AI tools extract the text before the model sees it, so the HTML figure is what an agent reading the raw page would load, not a measured bill. No AI provider documents a size threshold, so this is never scored.
Site crawl
Sample · not scoredStarting from the audited page, Zyntlox followed internal links and tested up to 26 pages without running scripts. Every page it found was tested. Account, login and cart links are left out.
| Page tested | Type | HTTP | Issues |
|---|---|---|---|
| https://northwind-studio.example/old-services | Other | 404 | missing page |
URL-level findings (1)
| URL | Type | HTTP | Issue | Test | Evidence |
|---|---|---|---|---|---|
| https://northwind-studio.example/old-services | Other | 404 | Link leads to a missing page Informational | GEO-801-INTERNAL-LINKS | HTTP 404. Linked from https://northwind-studio.example/services. |
Page type is guessed from the URL path alone (for example /blog/, /products/), so a page whose address gives no hint is shown as Other. Redirecting links are recorded, not followed. Pages are fetched without a browser, so links that scripts add are not seen. Tested 2026-11-01T10:00:00.000Z.
Priority fixes
Issues found on this site that lower its ZGI score.
[GEO-120-SIMULATED-ACCESS] Server refuses some crawler user agents (High). PerplexityBot (HTTP 403 vs browser HTTP 200) refused while a browser was served.
[GEO-301-ORG-SCHEMA] No Organization structured data (Medium). No Organization structured data found.
[GEO-204-SITEMAP-URLS] Some sitemap URLs redirect or do not exist (Medium). Sampled 6 of 6 URL(s): 5 OK, 0 redirected, 1 not found.
[GEO-401-STAT-DENSITY] Few quantitative figures in page text (Low). 3 quantitative figure(s) found in the page text (full credit at 5).
[GEO-303-FAQ-SCHEMA] No FAQPage structured data (Low). No FAQPage structured data found.
Detailed findings
9 findingsEvery finding states what was observed, what was expected, the evidence, and how to check it yourself. Severity is fixed per test by published rules.
HighServer refuses some crawler user agentsGEO-120-SIMULATED-ACCESS · −0.4 pts
- Test
- OAI-SearchBot access simulation
- Requested URL
- https://northwind-studio.example/
- HTTP response
- HTTP 200 (browser control: HTTP 200)
- Result
- Accessible
- Timestamp
- 2026-11-01T10:00:00.000Z
- Test
- ChatGPT-User access simulation
- Requested URL
- https://northwind-studio.example/
- HTTP response
- HTTP 200 (browser control: HTTP 200)
- Result
- Accessible
- Timestamp
- 2026-11-01T10:00:00.000Z
- Test
- Claude-SearchBot access simulation
- Requested URL
- https://northwind-studio.example/
- HTTP response
- HTTP 200 (browser control: HTTP 200)
- Result
- Accessible
- Timestamp
- 2026-11-01T10:00:00.000Z
- Test
- Claude-User access simulation
- Requested URL
- https://northwind-studio.example/
- HTTP response
- HTTP 200 (browser control: HTTP 200)
- Result
- Accessible
- Timestamp
- 2026-11-01T10:00:00.000Z
- Test
- Googlebot access simulation
- Requested URL
- https://northwind-studio.example/
- HTTP response
- HTTP 200 (browser control: HTTP 200)
- Result
- Accessible
- Timestamp
- 2026-11-01T10:00:00.000Z
- Test
- Bingbot access simulation
- Requested URL
- https://northwind-studio.example/
- HTTP response
- HTTP 200 (browser control: HTTP 200)
- Result
- Accessible
- Timestamp
- 2026-11-01T10:00:00.000Z
- Test
- PerplexityBot access simulation
- Requested URL
- https://northwind-studio.example/
- HTTP response
- HTTP 403 (browser control: HTTP 200)
- Result
- Blocked by server/CDN
- Timestamp
- 2026-11-01T10:00:00.000Z
MediumNo Organization structured dataGEO-301-ORG-SCHEMA · −21.9 pts
- Test
- Organization structured data
- Requested URL
- https://northwind-studio.example/
- Result
- No Organization structured data found.
- Timestamp
- 2026-11-01T10:00:02.580Z
MediumSome sitemap URLs redirect or do not existGEO-204-SITEMAP-URLS · −0.6 pts
- Test
- Sitemap URL request (broken)
- Requested URL
- https://northwind-studio.example/old-services
- HTTP response
- HTTP 404
- Result
- broken
- Timestamp
- 2026-11-01T10:00:00.000Z
LowFew quantitative figures in page textGEO-401-STAT-DENSITY · −6 pts
- Test
- Quantitative figures in page text
- Requested URL
- https://northwind-studio.example/
- Result
- 3 quantitative figure(s) found in the page text (full credit at 5).
- Timestamp
- 2026-11-01T10:00:02.580Z
LowNo FAQPage structured dataGEO-303-FAQ-SCHEMA · −4.4 pts
- Test
- FAQPage structured data
- Requested URL
- https://northwind-studio.example/
- Result
- No FAQPage structured data found.
- Timestamp
- 2026-11-01T10:00:02.580Z
InformationalGPTBot (training) is disallowed in robots.txtGEO-101-GPTBOT · not scored
- Test
- robots.txt evaluation for GPTBot
- Requested URL
- https://northwind-studio.example/robots.txt
- HTTP response
- HTTP 200
- Matched rule
- Disallow: / (line 6, under User-agent: GPTBot)
- Result
- fail
- Timestamp
- 2026-11-01T10:00:02.580Z
- Test
- GPTBot access simulation
- Requested URL
- https://northwind-studio.example/
- HTTP response
- HTTP 200 (browser control: HTTP 200)
- Result
- Blocked by robots.txt: robots.txt disallows this crawler. For comparison the simulated request got HTTP 200.
- Timestamp
- 2026-11-01T10:00:00.000Z
Informational/llms.txt is not publishedGEO-201-LLMS-TXT · not scored
- Test
- /llms.txt
- Requested URL
- https://northwind-studio.example/llms.txt
- HTTP response
- HTTP 404
- Result
- /llms.txt returned HTTP 404.
- Timestamp
- 2026-11-01T10:00:02.580Z
InformationalMissing pages do not return a real 404GEO-510-404-HANDLING · not scored
- Test
- Missing pages return a real 404
- Requested URL
- https://northwind-studio.example/
- Result
- A made-up path answered HTTP 200 instead of 404. Google calls this a soft 404.
- Timestamp
- 2026-11-01T10:00:02.580Z
InformationalSome internal links lead to missing pagesGEO-801-INTERNAL-LINKS · not scored
- Test
- Internal link target
- Requested URL
- https://northwind-studio.example/old-services
- HTTP response
- HTTP 404
- Result
- linked from https://northwind-studio.example/services
- Timestamp
- 2026-11-01T10:00:00.000Z
How to read this report
- It shows access, not outcomes. Zyntlox reports what a site permits and what a simulated request receives. It does not show that ChatGPT, Claude, Perplexity, Gemini or any other AI engine visits, indexes or cites the site.
- You can check every finding. Each one lists what was expected, what was observed and the evidence. No AI model writes a score or a finding.
- Unverified tests are not counted as passed. A test that could not be run is left out of the score and lowers the coverage figure instead.
- Some limits are our choices. Thresholds such as word counts or how many figures earn full credit are judgement calls, not facts about any AI engine.
ZGI v2.7 · tested 2026-11-01 · Read the full method and limits
How we keep it honest
Four rules every report follows
Zyntlox reports what a site permits and what a simulated request receives. It does not claim to know whether any AI engine will visit, index or cite you.
No AI model in the audit
Every result is a fact recorded by a test, and every explanation is written documentation. The score comes from fixed arithmetic, so the same observations always give the same score.
Evidence for every finding
Each finding shows what was expected, what was observed, the requested URL, the HTTP response, the rule that matched and a timestamp, plus how to check it yourself and the limits of the test.
Unverified is never “passed”
A test that could not be run is left out of the score and shown as lower coverage. Nothing is counted as passed because it could not be checked.
New conventions stay out of the score
Signals such as /llms.txt are checked and shown, but never scored, because no major AI engine has confirmed that it uses them.
Questions
Frequently asked questions
Plain answers about what Zyntlox checks, what it cannot tell you, and what happens to your data.
What does Zyntlox check?
Whether a website can be reached and understood by AI search and answer engines. That covers robots.txt rules for named crawlers, a simulated request as each crawler, HTTPS and redirects, the sitemap, structured data (Organization, sameAs, FAQPage), basic page structure and the quantitative figures in the text. The Method page lists every test with its weight and severity.
Does it show whether ChatGPT, Perplexity or Claude cite my site?
No, and it does not claim to. Zyntlox shows what your site permits and what a simulated request receives. Whether an AI engine visits, indexes or cites a site is decided by that engine, and Zyntlox cannot observe it.
Is an AI model involved in the audit?
No. Every result is a fact recorded by a test, and every explanation is written documentation. The score is fixed arithmetic over the test results, so the same observations always give the same score.
How is the score calculated?
The Zyntlox Generative Index (ZGI) runs from 0 to 100. Four categories carry fixed weights: Crawler access 30, Entity graph 35, Machine context 20 and Citability 15. Tests that could not be verified are left out of the score and shown as lower coverage, never counted as passed. Every report states the formula version it used.
What about /llms.txt and other new conventions?
They are checked and shown in a separate Emerging signals section, but never scored, because no major AI engine has confirmed that it uses them. If that changes and an engine documents it, the weight can be raised and the formula version updated.
How does Zyntlox scan a website?
It sends ordinary page requests. Its own checks identify themselves as ZyntloxAudit (+https://zyntlox.com). For the simulated-access test it also sends one request per crawler using that crawler's published user-agent string, so you can see how your server answers. Requests are rate limited and private addresses are refused. If you would like us to stop scanning a site you run, write to hello@zyntlox.com.
Do you store my audits?
An audit run without an account is not saved: the result goes to your browser. If you sign in, your reports are saved to your account history so you can reopen them. Server logs keep the audited domain, the score and timings, and rate limiting uses a hashed form of your IP address. The Privacy page has the details.
Do I need an account?
No. You can run an audit or a comparison without signing in during the public beta. Runs are limited per hour to keep the service available for everyone.
Can I compare my site with a competitor's?
Yes. The Compare page runs the same tests on both sites and shows, test by test, where each one stands. It compares what each site permits and how clearly it describes itself, and says nothing about how either one ranks or is cited.
Can I export the results for a client?
Yes. You can download the findings as CSV, the complete result as JSON, or a PDF report with your agency name and logo. Every file carries the formula version and the time the tests ran.