blog

Are agents glorified bots?

Sometime in 2024, the machines quietly took the lead. Not in the science-fiction sense. In the traffic-log sense.

Imperva's 2025 Bad Bot Report found that automated traffic reached 51% of the web in 2024, the first time in a decade that bots outnumbered people. Bad bots alone were 37%, up from 32% the year before. Imperva points at AI for lowering the barrier to building them.

So more than half your visitors are not human. Fine. The more interesting question is what the machines are actually doing when they arrive, because "AI traffic" has become a flattering label for a lot of things, and most of them are not agents in any meaningful sense. They are bots. Some of them, to borrow the phrase, are peeping toms without purpose. They read everything and knock on no doors.

Let's sort out which is which.

First, what is an agent, and what is just a bot

The words get used loosely, so here is an honest definition.

A bot is any automated visitor. A crawler is a bot that fetches pages in bulk, usually to build a dataset or an index. An agent, in the sense people get excited about, is a bot doing something on behalf of a person, right now: fetching a page because a human just asked a question and is waiting for the answer.

That last distinction is the whole game. One is reading your site for its own reasons. The other is standing in for a person who wants something from you. They can hit the same URL a second apart and mean completely different things.

Most of it is not an agent, it's a crawler with an appetite

Cloudflare, which sees a large slice of the web's traffic, breaks AI crawling down by purpose. As of July 2025, about 79% of it was for model training, up from 72% a year earlier. Search indexing was around 17%. Live, user-triggered fetching was roughly 3%.

Read that again. Roughly four out of five AI bot visits exist to absorb content for training. Not to answer anyone. Not to send anyone back. Just to read, and leave.

Bar chart: about 79% of AI crawler requests are for model training, 17% for search indexing, and about 3% from live user queries. The user-query slice is the only one that fetches a page for a real person. Source: Cloudflare, July 2025.

The user-query slice is the only one that fetches a page for a real person. Source: Cloudflare, July 2025.

Training crawls, by definition, refer no one. Content goes in, no human comes out. If you are looking for the peeping tom, this is it. And its share of AI crawling grew over the year, not shrank, so the imbalance is widening, not closing.

The gap between what they take and what they give

Here is the uncomfortable part.

While AI crawling has exploded, the actual human traffic that AI sends back to websites is still tiny. Semrush, looking at billions of visits across more than 50,000 sites, put AI referrals at about 0.14% of total traffic in 2025. That was up 66% on the year, which sounds dramatic until you set it next to organic search at roughly 16% and direct at roughly 65%.

Bar chart of web traffic by channel: direct 64.69%, organic search 16.04%, and AI referrals just 0.14%. The traffic AI assistants actually send back is a rounding error. Source: Semrush, 2025.

AI referrals are the visits an AI assistant actually sends. Still a rounding error. Source: Semrush, 2025.

Definitions vary, so other trackers land a little higher or lower. But every serious estimate puts AI referrals in the same neighborhood, and that neighborhood is a rounding error. A lot of taking, very little giving. That is the tension underneath this whole moment.

The part that genuinely is an agent

Now the honest counterweight, because "all AI traffic is purposeless" is the wrong conclusion.

That small sliver of user-triggered fetching, the roughly 3%, is the fastest-growing category of the three. Cloudflare measured it climbing more than 15x across 2025. Tiny base, steep curve. When you ask ChatGPT something and it goes and reads a page to answer you, that is a real agent doing real work for a real person who is still sitting there waiting.

And the industry now takes the difference seriously enough to name it. Look at the robots tokens the vendors publish. OpenAI runs GPTBot for training and ChatGPT-User for the live, someone-just-asked fetch. Anthropic separates ClaudeBot from Claude-User. Perplexity separates PerplexityBot from Perplexity-User. Different names, different jobs, spelled out in their own documentation. The line between an agent and a crawler is not something a marketing team invented. The people building these systems drew it themselves.

Diagram: background crawlers like GPTBot, ClaudeBot, and PerplexityBot read your page and send no visitor back, while purposeful agents like ChatGPT-User, Claude-User, and Perplexity-User fetch your page for a waiting person and can send a real visitor to you. Sources: OpenAI, Anthropic, and Perplexity bot docs.

Sources: OpenAI, Anthropic, and Perplexity bot documentation.

It is not a settled peace, though. Perplexity's user fetcher is documented to generally ignore robots.txt on the logic that a person asked for the page, and in August 2025 Cloudflare stopped treating Perplexity as a verified bot, citing disguised user agents and rotating IP addresses. So the distinction is real, and it is also contested at the edges. Worth keeping an eye on.

When an agent does send a human, that human is different

The visitors AI does send tend to behave unlike search visitors, because they have already done their thinking inside the chat and arrive further along.

One case study from Seer Interactive makes the point vividly: on a single site, traffic from ChatGPT converted at 15.9% against 1.76% for Google organic, with more pages per session, while AI made up just 0.07% of the volume. One site, one B2B account, so treat it as an illustration and not a law. But the shape of it shows up repeatedly. Fewer visitors, further down the funnel.

So the small purposeful stream behaves nothing like the giant purposeless one. Same "AI traffic" label. Opposite value.

So, are agents glorified bots?

Mostly, today, yes. If you average all the automated AI traffic hitting your site, the typical visitor is a crawler taking content for training and giving nothing back. The word "agent" is doing a lot of flattering work.

But not entirely, and less so every month. A small, fast-growing share genuinely acts on a person's behalf, and those visits are worth more than their count suggests. The real mistake is treating "AI traffic" as one thing. It is at least two things, and they point in opposite directions.

Which is exactly why you have to tell them apart

You cannot manage what your analytics lumps together. If a training crawler, an agent fetching for a waiting human, and that human's own later visit all land in the same "AI" bucket, you learn nothing you can act on.

That separation is the job Voris was built for. Humans, AI referrals, and AI agents are three different signals, kept apart on purpose. Filter the bots out of your reports, see which named assistants actually send converting visitors, and verify the purposeful agents instead of guessing. The peeping toms and the genuine visitors stop sharing a row.

The question worth asking about any AI visit is not whether it is good or bad. It is simpler than that. Which kind was this, and what did it do next. Answer that, and "are agents glorified bots" stops being a debate and starts being something you can just look up.

The numbers, and where they come from

Anna van Bergeijk, Head of Brand. Writes the blog and reads the replies.

read next

ChatGPT vs Perplexity for B2B discovery

More B2B buyers now start in a chatbot than on Google. ChatGPT and Perplexity surface your brand in very different ways, and only one shows its sources.