training crawlers
Google-Extended
Operated by Google · Training crawlers
Robots token controlling whether Google may use crawled content for Gemini model training and grounding; it is not a separate HTTP crawler user agent.
When Google-Extended shows up in your logs
This is a robots.txt token, not live traffic. It controls Gemini training and grounding permission.
Crawlers that collect public content that may be used to train or ground models.
Allow or block it in robots.txt
Google-Extended is a policy token: robots.txt is exactly where it lives, and these lines are the whole mechanism.
More from Google
Is Google-Extended on your site right now?
Voris AI visibility answers that: verified AI agent traffic, split from your human visitors, on every plan. Cookieless, with a free plan and a 14-day trial that needs no card.