Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare CEO is lying to you about the bot traffic jump

flyingpenguin.com

Post reply on HN
Print view View original

61–70 of 149 posts

Re: Cloudflare CEO is lying to you about the bot traffic jump

#61

Post by tmpz22 »

majke wrote: post #34
autoexec wrote: post #24

Earlier quoted context omitted.

Because they inserted themselves into almost everything we do online and basically managed to take control over it. Cloudflare should never have been allowed to man in the middle the entire internet, but now that they have they're the only man on earth with a dataset that size.

(I'm ex CF) This is backwards. Nobody "allowed" anyting. CF serves a customers need. You can argue with the solution but you can't argue with the core problem. It's more healthy to start the conversation of _why_ CF services are valuable.

The fent dealer just serves the customers needs too

Re: Cloudflare CEO is lying to you about the bot traffic jump

#62

Post by gruez »

kordlessagain wrote: post #2

I concur and have been talking about this for a while. The fact is, Cloudflare is a man-in-the-middle. That's their focus, that's their purpose. They will limit your local crawler from accessing pages. They will demand you use their crawler. They will decrypt your traffic if they get a warrant. They always decrypt your traffic anyway, but they will give it to state actors if they demand it. That's not to say anyone s…

>They will limit your local crawler from accessing pages. They will demand you use their crawler.

Source? According to cloudflare, their crawling service don't get any special treatment from their WAF/CDN.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#63

Post by LorenPechtel »

reconnecting wrote: post #9
taeric wrote: post #3

I confess a sad assumption that bot traffic is far higher than we have admitted for a long time. Though, maybe we would see different stats specifically to social media sights to astroturf like counts? Certainly feels that we have known for a long time that bots were larger in ad viewing than ad companies wanted to admit.

I don't understand what difference bots make. For me, a website (the public part) is a storefront. People walk down the street and see what's inside — that's the purpose. If something should not be available immediately, that's the private part of the store. I've been monitoring bot traffic on digital platforms for over 10 years. Sure, the crawler share is growing, some even with malicious intentions, and those I det…

When most of your server capacity is going to answering the scrapers it matters. It's not that the stuff is hidden, it's that storefront being flooded with 10x as many customers as the fire code allows. And some of them go around asking your employees mindless questions. (Small forum I help moderate: we were getting hammered with what was probably some sort of AI that was taking search queries and feeding them into the forum search. Search is now registered users only.)

Re: Cloudflare CEO is lying to you about the bot traffic jump

#64

Post by simonw »

burnte wrote: post #44
simonw wrote: post #11

"Cloudflare CEO is lying" is a bit of an aggressive take when he linked to the exact data so you can see it for yourself - and that's how this article was able to analyze it: https://radar.cloudflare.com/traffic#bot-vs-human Update: I see the problem. Here's the full tweet: https://x.com/eastdakota/status/2062212701414187452 "Thought it would be end of 2027, then early 2027, but agentic traffic growing so fast that b…

It's aggressive but true. Spam traffic exceeded real email back in 2003. He's not even in the right DECADE to declare a first for non-human traffic. It's pure marketing BS that is only remotely true if you accept the conditions he goes to lengths to hide.

What lengths did he go to in order to hide them?

Re: Cloudflare CEO is lying to you about the bot traffic jump

#66

Post by gruez »

ai_fry_ur_brain wrote: post #54
jimrandomh wrote: post #31

Earlier quoted context omitted.

Have you checked your IP address's reputation with a service such as ipqualityscore.com? If cloudflare thinks your traffic is bot traffic, it's likely that there is bot traffic you don't know about coming from your IP, either from a compromised device on your network or a sketchy VPN product.

Also, it doesnt apply to this person if they've had the same IP for years but if your ISP rotates IPs frequently (mine does everytime I reset modem) or you use 5G and CGNAT is being used, its almost a garuntee that your proxy has been labled as having been used by a residential proxy network. So many people have sketchy TV boxes or whatever other sketchy IOT decice that is a larp for using your network to sell bandwi…

>However, CF is unnecessarily making people on 5G connections from desktops do turnstiles as it looks like a scraper using a mobile proxy. This will become more and more of an issue as more laptops have 5G modems in them.

What makes you think this is the cause, rather than something more straightforward like: CGNAT means more users are sharing IPs, and there's a higher chance that the IP pool gets contaminated by bad behavior? Apparently cloudflare tries to detect CGNAT pools and give them more leeway, but at the same time they can't give them unlimited leeway.

[1] https://blog.cloudflare.com/detecting-cgn-to-reduce-collater...

Re: Cloudflare CEO is lying to you about the bot traffic jump

#67

Post by arjie »

jimrandomh wrote: post #29

I deal with scrapers that sometimes border on DDoSes for LessWrong. The amount of bot traffic varies greatly between sites; if you have more URLs you get more bot traffic (regardless of whether those URLs represent a deep content catalog, or useless URL parameter permutations). It's bad for LW because of the content-catalog depth. It's easy to drastically underestimate the amount of bot traffic, because bots make eff…

Does LW have a downloadable archive? I can only find references to GreaterWrong but no public answer. Would be useful.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#68

Post by reconnecting »

Groxx wrote: post #55
reconnecting wrote: post #5

Cloudflare bot detection has taught me a reflex to close the tab every time I see its logo.

Yeah, I've seriously considered finding or building a CF-protected-detector browser extension to flag domains. Having one company MITMing so much traffic is straightforwardly dangerous, and not just an annoyance. We need competition.

Why so? They're all in NS already: *.ns.cloudflare.com

Re: Cloudflare CEO is lying to you about the bot traffic jump

#69

Post by epistasis »

Groxx wrote: post #52
majke wrote: post #34

Earlier quoted context omitted.

(I'm ex CF) This is backwards. Nobody "allowed" anyting. CF serves a customers need. You can argue with the solution but you can't argue with the core problem. It's more healthy to start the conversation of _why_ CF services are valuable.

I think it counts as "allowed" regardless of utility. CF is so massively over-weight on the internet that it's impossible to trust them with anything because if they can be forced to do something by a hostile government (hint: they can be!) then they can get away with it invisibly and affect billions of people. That is something that should not be allowed to exist. It's one of the reasons monopolies (or even majority…

Then I think the real question is why haven't any serious competitors emerged that can handle the essential services that Cloudflare provides?

Are there network effects like what happens with Microsoft in the business computing space? With Microsoft, I'm also aware of a great amount of anti-competitive behavior, and though I haven't seen that from Cloudflare personally and haven't heard accusations of it, I also haven't paid attention.

When I learned econ 101 in high school there was a concept of a "natural monopoly" like an electricity utility, a concept that was probably mostly post-hoc rationalization of the regulatory structures that were chosen a century ago, but it at least was a coherent narrative. I can't see any coherent narrative about Cloudflare's services being a natural monopoly. So I'm left wondering if they are just way better at what they do than anybody else, and perhaps the space isn't big enough to drive a competitor to enter it?

I hope somebody on HN has a much better explanation of this than I do.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#70

Post by sophrosyne42 »

Groxx wrote: post #52
majke wrote: post #34

Earlier quoted context omitted.

(I'm ex CF) This is backwards. Nobody "allowed" anyting. CF serves a customers need. You can argue with the solution but you can't argue with the core problem. It's more healthy to start the conversation of _why_ CF services are valuable.

I think it counts as "allowed" regardless of utility. CF is so massively over-weight on the internet that it's impossible to trust them with anything because if they can be forced to do something by a hostile government (hint: they can be!) then they can get away with it invisibly and affect billions of people. That is something that should not be allowed to exist. It's one of the reasons monopolies (or even majority…

Seems more of a moral hazard of government intervention than of ventures whose economies of scale demand large market cap to most economically serve customer needs.
Post reply on HN