training crawlers
Google-CloudVertexBot
Operated by Google · Training crawlers
Google's crawler for on-demand fetches a site owner requests while building a Vertex AI agent.
When Google-CloudVertexBot shows up in your logs
A Vertex AI builder requested this specific crawl. If you run Vertex, it's likely your own.
Crawlers that collect public content that may be used to train or ground models.
Allow or block it in robots.txt
A robots.txt entry is a request. Well-behaved crawlers honor it; it is not an enforcement mechanism.
More from Google
Is Google-CloudVertexBot on your site right now?
Voris AI visibility answers that: verified AI agent traffic, split from your human visitors, on every plan. Cookieless, with a free plan and a 14-day trial that needs no card.