Is Your Site Agent-Ready?

Scan your website to see how ready it is for AI agents. We check multiple emerging standards — from robots.txt and Markdown negotiation to MCP, OAuth, Agent Skills and agentic commerce.

What do we check?

Multiple checks across 5 categories:

Discoverability — robots.txt, Sitemap, Link response headers
Content Accessibility — Markdown content negotiation
Bot Access Control — AI bot rules in robots.txt, Content Signals, Web Bot Auth
Protocol Discovery — MCP Server Card, Agent Skills, WebMCP, API Catalog, OAuth discovery, OAuth Protected Resource
Commerce — x402, MPP, UCP, ACP

Discoverability

Score: 2/3

robots.txt

Goal: Publish /robots.txt with clear crawl rules
Result: robots.txt exists with valid format

Audit details


content-type	text/plain; charset=utf-8
content-length	5628

#    .__________________________.
#    | .___________________. |==|
#    | | ................. | |  |
#    | | ::[ Dear robot ]: | |  |
#    | | ::::[ be nice ]:: | |  |
#    | | ::::::::::::::::: | |  |
#    | | ::::::::::::::::: | |  |
#    | | ::::::::::::::::: | |  |
#    | | ::::::::::::::::: | | ,|
#    | !___________________! |(c|
#    !_______________________!__!
#   /                            \
#  /  [][][][][][][][][][][][][]  \
# /  [][][][][][][][][][][][][][]  \
#(  [][][][][_____...

Sitemap

Goal: Publish a sitemap and reference it from robots.txt
Result: sitemap.xml exists with valid structure

Audit details


content-type	application/xml; charset=utf-8

Content

Score: 1/1

Goal: Return HTML responses as markdown when agents request it
Result: Site supports Markdown for Agents

Audit details


accept	text/markdown
content-type	text/markdown; charset=utf-8

Bot Access Control

Score: 2/2

Goal: Add User-agent rules for AI crawlers like GPTBot, Claude-Web, and others
Result: No AI-specific bot rules; wildcard rules apply to all crawlers including AI bots

Audit details


content-type	text/plain; charset=utf-8
content-length	5628

Conclusion

No AI-specific bot rules; wildcard rules apply to all crawlers including AI bots.

Where can I learn more?

Check out the Cloudflare Agents documentation to learn how to build and deploy AI agents that can browse, interact, and transact on the web.