Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare CEO is lying to you about the bot traffic jump

flyingpenguin.com

Post reply on HN
Print view View original

31–40 of 149 posts

Re: Cloudflare CEO is lying to you about the bot traffic jump

#31

Post by jimrandomh »

nothrows wrote: post #20

Cloudflare is junk. Their entire billion dollar service can't distinguish my (DAILY) GET request to mainstream news sites from bot traffic, nothing they say or do is of any value. I've had the same IP for decades.

Have you checked your IP address's reputation with a service such as ipqualityscore.com? If cloudflare thinks your traffic is bot traffic, it's likely that there is bot traffic you don't know about coming from your IP, either from a compromised device on your network or a sketchy VPN product.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#32

Post by onei »

a1o wrote: post #23
DevKoala wrote: post #13

Not sure if the Cloudflare CEO is lying, but I have a pixel deployed on tens of thousands of sites offering B2B solutions, and bot traffic overtook human traffic this year.

What does pixel means in this context?

It's a tracking tool. You have a bunch of sites embed an image, and requests to those sites also make requests to said image, which you can use to start tracking a client. A single pixel is merely the cheapest image.

I recall Facebook doing it years ago, I imagine they still do.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#33

Post by letmevoteplease »

bobjordan wrote: post #12

The article strikes me as quite uncharitable to characterize it as "a lie". I doubt this CEO just sat down and calculated he was going to write lies. While it's fine to call out he's wrong per his own fuller data set, it's quite a different thing than calling the person out as a "liar" in a rage-bait fashion.

The author is also not very good at interpreting data himself. He claims "the AI number is padded by counting Googlebot twice" and links to [1], but there is nothing on that page that could support that assertion. It looks like he misinterpreted this part: "Googlebot crawls for both search indexing and AI training and is included as a separate entry due to its crawl volume" (Googlebot was NOT included in the "AI bot" count, so it was not counted twice.)

[1] https://radar.cloudflare.com/year-in-review/2025

Re: Cloudflare CEO is lying to you about the bot traffic jump

#34

Post by majke »

autoexec wrote: post #24
thm wrote: post #14

Why did we start treating Cloudflare (a public, for-profit company) as the undisputed authority on anything related to the network layer of the internet in the first place?

Because they inserted themselves into almost everything we do online and basically managed to take control over it. Cloudflare should never have been allowed to man in the middle the entire internet, but now that they have they're the only man on earth with a dataset that size.

(I'm ex CF) This is backwards. Nobody "allowed" anyting. CF serves a customers need. You can argue with the solution but you can't argue with the core problem. It's more healthy to start the conversation of _why_ CF services are valuable.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#35

Post by autoexec »

nothrows wrote: post #20

Cloudflare is junk. Their entire billion dollar service can't distinguish my (DAILY) GET request to mainstream news sites from bot traffic, nothing they say or do is of any value. I've had the same IP for decades.

Some people think they're providing value. They do block some bot traffic (at the cost of many many false positives) and I'll bet that they're providing endless amounts of data to one or more three letter agencies. They also provide lots of value to scammers who use cloudflare to protect their phishing and malware pushing webpages from automated systems that would detect/report them while also making sure visitors have JS enabled so their browser exploits work.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#36

Post by simonw »

JimDabell wrote: post #30
simonw wrote: post #11

"Cloudflare CEO is lying" is a bit of an aggressive take when he linked to the exact data so you can see it for yourself - and that's how this article was able to analyze it: https://radar.cloudflare.com/traffic#bot-vs-human Update: I see the problem. Here's the full tweet: https://x.com/eastdakota/status/2062212701414187452 "Thought it would be end of 2027, then early 2027, but agentic traffic growing so fast that b…

> It looks to me like the data supports "bots passed human traffic" I think you are missing the fact that the dashboard has HTML pre-selected as a filter. Once you change that to all content types, you’ll see humans account for twice as much traffic as bots. Note this part of the article: > The CEO ignored the all-traffic number, on his own dashboard, and instead published the HTML-only number as a fact about the who…

Personally I think that limiting the stats to just HTML pages is a credible stat on which you could claim that "bots passed human traffic". I don't care about bots downloading other types of content.

The most important thing is to link to your methodology, which Cloudflare's CEO did in this case.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#37

Post by reconnecting »

taeric wrote: post #16
reconnecting wrote: post #9

Earlier quoted context omitted.

I don't understand what difference bots make. For me, a website (the public part) is a storefront. People walk down the street and see what's inside — that's the purpose. If something should not be available immediately, that's the private part of the store. I've been monitoring bot traffic on digital platforms for over 10 years. Sure, the crawler share is growing, some even with malicious intentions, and those I det…

For ad views, the concern is specifically that people pay for clicks and views. That that can be so heavily influenced by bot traffic greatly undermines their value. Same general idea goes for any of the algorithmic driven platforms. The algorithms are ostensibly intended to surface organically discovered things by watching how people interact with things. That they are so susceptible to distortion through bot farms…

For ads, I believe this must be a problem for ad platform owners.

If the digital platform's storefront is their business, they could afford to spend some budget on bot detection. Bots still come from data center networks, sometimes render pages incompletely, request resources in bulk, and show enough patterns to be flagged internally.

If we look at a medium website, most random crawlers will come from Amazon, Microsoft, DigitalOcean, Hetzner, OVH, and a few other DC networks — these can be blocked easily without harming real users. The rest can be detected and cleaned up, even manually.

The math is simple: 20,000 visits a day at 15 seconds each = ~83 hours a day lost watching a Cloudflare logo, just because someone doesn't want to dig into the logs. I don't buy it.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#38

Post by Terretta »

a1o wrote: post #23
DevKoala wrote: post #13

Not sure if the Cloudflare CEO is lying, but I have a pixel deployed on tens of thousands of sites offering B2B solutions, and bot traffic overtook human traffic this year.

What does pixel means in this context?

https://advertising.amazon.com/resources/ad-policy/pixeling-...

A 'pixel' is an unobtrusive (as in, not seen by the user like a banner ad is seen) asset* served on a web page that can cause the user's user agent to make an affirmative web request from you, a third party, so you know someone was at the site serving your pixel.

Typically used for:

- tracking in general, as well as more specifically:

- retargeting

- conversion

* Note: Doesn't have to be a literal pixel, but a literal transparent pixel is least likely to get blocked. Serve your pixels from the end of a parameterized path (/some/param/or/other/pixel.gif) and it's not seen as query string tracking either.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#39

Post by EnergyAmy »

majke wrote: post #34
autoexec wrote: post #24

Earlier quoted context omitted.

Because they inserted themselves into almost everything we do online and basically managed to take control over it. Cloudflare should never have been allowed to man in the middle the entire internet, but now that they have they're the only man on earth with a dataset that size.

(I'm ex CF) This is backwards. Nobody "allowed" anyting. CF serves a customers need. You can argue with the solution but you can't argue with the core problem. It's more healthy to start the conversation of _why_ CF services are valuable.

No, it's more healthy to start the conversation on why we allow corporations to do bad things with excuses like "just serving customer's needs"

Re: Cloudflare CEO is lying to you about the bot traffic jump

#40

Post by eli »

"Lying" is not supported by the evidence. In the context of bot traffic on the web, looking at only GETs for HTML is a reasonable approach. If you're counting all requests for all assets then a single page view of nytimes.com would count 100x as much as one for HN.

I would assume a lot of people running websites tend to think in pageviews, especially when dealing with bots because images and CSS files tend to be "cheap" static content but HTML requests are often dynamically generated.

It's also a single tweet that links to the data used to "disprove" it. Would be a weird way to lie.

Post reply on HN