Cloudflare CEO is lying to you about the bot traffic jump • Hacker News Board

Re: Cloudflare CEO is lying to you about the bot traffic jump

#21

Post by gonzalohm » Fri, Jun 05, 2026, 6:57 PM UTC

kordlessagain wrote: post #2

I concur and have been talking about this for a while. The fact is, Cloudflare is a man-in-the-middle. That's their focus, that's their purpose. They will limit your local crawler from accessing pages. They will demand you use their crawler. They will decrypt your traffic if they get a warrant. They always decrypt your traffic anyway, but they will give it to state actors if they demand it. That's not to say anyone s…

Careful I posted something similar in another Cloudfare thread and people threw at me like lions.

They don't see anything wrong with one entity controlling most of the internet traffic

Re: Cloudflare CEO is lying to you about the bot traffic jump

#22

Post by csomar » Fri, Jun 05, 2026, 6:58 PM UTC

The article is a bit too strong, aggressive I’d even say. Content is loaded only if the bot executes JavaScript and loads all content willingly. These do exist, but they are more expensive to run than a basic curl bot.

It’d make sense as you might not want your bot to load everything a real human would do (ie: analytics, ads, unrelated files, etc..) and only focus on the content.

Also, am I the only one surprised that bot traffic is not the majority already? For my site, it’s x100 bots for every human.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#23

Post by a1o » Fri, Jun 05, 2026, 6:58 PM UTC

DevKoala wrote: post #13

Not sure if the Cloudflare CEO is lying, but I have a pixel deployed on tens of thousands of sites offering B2B solutions, and bot traffic overtook human traffic this year.

What does pixel means in this context?

Re: Cloudflare CEO is lying to you about the bot traffic jump

#24

Post by autoexec » Fri, Jun 05, 2026, 7:01 PM UTC

thm wrote: post #14

Why did we start treating Cloudflare (a public, for-profit company) as the undisputed authority on anything related to the network layer of the internet in the first place?

Because they inserted themselves into almost everything we do online and basically managed to take control over it. Cloudflare should never have been allowed to man in the middle the entire internet, but now that they have they're the only man on earth with a dataset that size.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#25

Post by yubblegum » Fri, Jun 05, 2026, 7:01 PM UTC

nothrows wrote: post #20

Cloudflare is junk. Their entire billion dollar service can't distinguish my (DAILY) GET request to mainstream news sites from bot traffic, nothing they say or do is of any value. I've had the same IP for decades.

That could be plausible deniability. I mean, CF is in fact keeping a tab on who is visiting which websites. Between them and Google, these two companies know everything about everyone.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#26

Post by supriyo-biswas » Fri, Jun 05, 2026, 7:01 PM UTC

nothrows wrote: post #20

Cloudflare is junk. Their entire billion dollar service can't distinguish my (DAILY) GET request to mainstream news sites from bot traffic, nothing they say or do is of any value. I've had the same IP for decades.

It can be worse; they randomly block my uptime monitoring with 4xx and 5xx status codes once in two months or something like that, despite nothing changing.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#27

Post by supriyo-biswas » Fri, Jun 05, 2026, 7:04 PM UTC

a1o wrote: post #23

DevKoala wrote: post #13

Not sure if the Cloudflare CEO is lying, but I have a pixel deployed on tens of thousands of sites offering B2B solutions, and bot traffic overtook human traffic this year.

What does pixel means in this context?

https://en.wikipedia.org/wiki/Web_beacon, aka "tracking pixel", though these days it probably means a JS-based analytics reporting script.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#28

Post by Xirdus » Fri, Jun 05, 2026, 7:04 PM UTC

1vuio0pswjnm7 wrote: post #4

There is a unfortunate incentive created when a "business" (MiTM) depends on "bot traffic", i.e., the continued nuisance of bot traffic, to make money If the "bot traffic" declines, then the "bot protection business" goes down with it Cloudflare communication are sometimes careful to refer to traffic _labeled as_ bot traffic versus actual bot traffic Because the "business" relies on the existance of "bot traffic", th…

> Originally "bot traffic" was based on behaviour, e.g., exceeding acceptable rates of usage, making too many requests in a given time period, exceeding rate limits

> Now it's available to exclude traffic based on criteria such as what browser someone is using

I'm pretty sure user-agent-based bot detection predates every request-rate-based method by quite a few years.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#29

Post by jimrandomh » Fri, Jun 05, 2026, 7:05 PM UTC

I deal with scrapers that sometimes border on DDoSes for LessWrong. The amount of bot traffic varies greatly between sites; if you have more URLs you get more bot traffic (regardless of whether those URLs represent a deep content catalog, or useless URL parameter permutations). It's bad for LW because of the content-catalog depth.

It's easy to drastically underestimate the amount of bot traffic, because bots make efforts (of varying sophistication) to look human enough to evade blocking. That includes using fake user-agent strings corresponding to real browsers (often but not always with implausibly old version numbers), proxying through residential IPs, and sometimes using full headless browsers. In my own data, traffic from badly behaved browser-impersonation bots exceeds traffic from named scrapers like GPTBot by something like 10x.

The measured percentage of bot traffic is higher for HTML than for other content types because many bots will load an HTML page, and then not load the JS/CSS/image/etc resources it references. But these are the least-sophisticated and most-detectable bots.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#30

Post by JimDabell » Fri, Jun 05, 2026, 7:05 PM UTC

simonw wrote: post #11

"Cloudflare CEO is lying" is a bit of an aggressive take when he linked to the exact data so you can see it for yourself - and that's how this article was able to analyze it: https://radar.cloudflare.com/traffic#bot-vs-human Update: I see the problem. Here's the full tweet: https://x.com/eastdakota/status/2062212701414187452 "Thought it would be end of 2027, then early 2027, but agentic traffic growing so fast that b…

> It looks to me like the data supports "bots passed human traffic"

I think you are missing the fact that the dashboard has HTML pre-selected as a filter. Once you change that to all content types, you’ll see humans account for twice as much traffic as bots.

Note this part of the article:

> The CEO ignored the all-traffic number, on his own dashboard, and instead published the HTML-only number as a fact about the whole internet.