training crawlers
ClaudeBot
Operated by Anthropic · Training crawlers
Anthropic crawler for public content that may be used to improve Claude.
When ClaudeBot shows up in your logs
Anthropic is collecting public content that may improve Claude.
Crawlers that collect public content that may be used to train or ground models.
Allow or block it in robots.txt
A robots.txt entry is a request. Well-behaved crawlers honor it; it is not an enforcement mechanism.
More from Anthropic
Is ClaudeBot on your site right now?
Voris AI visibility answers that: verified AI agent traffic, split from your human visitors, on every plan. Cookieless, with a free plan and a 14-day trial that needs no card.