HTTP/1.1 200 OK for you · HTTP/1.1 403 Forbidden for them
Your site answers browsers. It might be hanging up on agents.
We request your page as five different clients, compare what each one gets back, and hand you the exact files that fix the gaps.
As a browser
GET / HTTP/1.1Host: linear.appUser-Agent: Mozilla/5.0 (Macintosh) … Chrome/141.0Accept: text/html HTTP/1.1 200 OKcontent-type: text/html; charset=utf-8server: cloudflare Linear is a purpose-built tool for planning andbuilding products. Streamline issues, projects,and product roadmaps. Pricing: Free, Basic $8,Business $14, Enterprise …
9,240 readable characters
As a reading agent
GET / HTTP/1.1Host: linear.appUser-Agent: ClaudeBot/1.0Accept: text/html HTTP/1.1 403 Forbiddencontent-type: text/htmlserver: cloudflarecf-mitigated: challenge (nothing readable)
312 readable characters
Three passes, about thirty seconds
What the check actually does
GET / × 5 clients
The same URL, five ways
- Once as Chrome and once as each of ClaudeBot, GPTBot, PerplexityBot and Google-Extended, recording the status, headers and byte count each got back. Same URL, same second.
raw HTML vs rendered DOM
Text with and without JavaScript
- The readable text is extracted from the plain response and from a headless render by the same algorithm. The ratio between them is how much of your page only exists once a script has run.
GET /robots.txt, /llms.txt, /.well-known/…
The paths agents look for first
- robots.txt, sitemap.xml, llms.txt, llms-full.txt and the agent manifests, checked for existence, parseability, and links that actually resolve.
6 pages at most, sequential, 1 second apart. We identify as BotreadyBot/1.0 and obey your robots.txt. If your site refuses us, we record it as refused and show it that way rather than working around it. The weights are published.