Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare CEO is lying to you about the bot traffic jump

flyingpenguin.com

Post reply on HN
Print view View original

121–130 of 149 posts

Re: Cloudflare CEO is lying to you about the bot traffic jump

#121

Post by layer8 »

stickfigure wrote: post #88
layer8 wrote: post #58

Earlier quoted context omitted.

Regulation could conceivably disallow a single company to control such a significant portion of internet traffic. The parent can be interpreted as lamenting the absence of such regulation.

The only thing more annoying than people chanting "regulation is bad" is people chanting "regulation is good". What regulation? Be specific. CloudFlare provides significant utility to me. I chose to use them. Explain why you think someone else needs to butt into this relationship.

I wasn't advocating regulation. I was arguing against the claim that "nobody allowed anything". The absence of regulation is allowing things.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#122

Post by autoexec »

Groxx wrote: post #110
autoexec wrote: post #109

Earlier quoted context omitted.

> Everyone using the free service likes that, of course, but honestly I wish we'd make it illegal to do. I can see the benefit, but if you made free services illegal companies like google would just offer gmail for a penny. You'd could go farther and say that no one should be allowed to provide a good or service for less than it costs them to provide it. That still has problems though. For example, Netflix sends serv…

There are plenty of ways to mitigate that "losing a lot" (like government subsidies: run X, get paid $Y per Z -> easier access for both users and providers. we already do this for a lot of research and tons of industry, it's not some utopian dream), and since a lot of the free internet stuff we have now is ad-supported and addiction-oriented... I'm not convinced that'd be bad to lose. The adtech business causes horri…

Maybe you could avoid the thornier issues entirely by outlawing the adtech industry instead of free services. With the backbone of surveillance capitalism out of the picture most of the free services would stop existing or move to pay models anyway. Worthwhile free services like the internet archive and wikipedia that aren't spying on users could continue to exist without worry. We'd still lose youtube I think, but maybe video hosting isn't best centralized either.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#123

Post by Groxx »

autoexec wrote: post #122
Groxx wrote: post #110

Earlier quoted context omitted.

There are plenty of ways to mitigate that "losing a lot" (like government subsidies: run X, get paid $Y per Z -> easier access for both users and providers. we already do this for a lot of research and tons of industry, it's not some utopian dream), and since a lot of the free internet stuff we have now is ad-supported and addiction-oriented... I'm not convinced that'd be bad to lose. The adtech business causes horri…

Maybe you could avoid the thornier issues entirely by outlawing the adtech industry instead of free services. With the backbone of surveillance capitalism out of the picture most of the free services would stop existing or move to pay models anyway. Worthwhile free services like the internet archive and wikipedia that aren't spying on users could continue to exist without worry. We'd still lose youtube I think, but m…

yes, I think that's a good idea too. and possibly easier to achieve.

I don't think advertising is inherently bad, and at some point you get ambiguous about "is this advertising or is it just describing something so I can find it in a search engine" and it's literally impossible to eliminate it all. and being able to tell people with problem X that solution Y exists can be a good thing, in moderation. it's more that the current maximally-invasive insanity and e.g. tons of highly distracting billboards are probably a net loss for humanity. São Paulo shows that it's quite nice without it, for example.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#124

Post by Groxx »

reconnecting wrote: post #115
Groxx wrote: post #112

Earlier quoted context omitted.

I want to visibly indicate it on links / visits, for myself, so I can learn what the impact is and what might be worth avoiding. And how tightly bound use of it is - often these kinds of services are used/avoided by pockets that are somewhat closed (spam networks, indie webrings, etc), and I'd like to get a feel for where those pockets might be. I assume there's some way to detect that mechanically, which is probably…

There are domain zone files, and they sometimes contain NS records. Cloudflare hosted domains are easy to identify via their NS records, which also makes Cloudflare customers an ideal target for phishing.

(replying for myself, and for anyone else interested)

Actually I think this might not be too hard. Cloudflare has a relatively small, published list of IP addresses: https://www.cloudflare.com/ips/ and Firefox has a DNS API for extensions: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/Web... (and kinda mostly not chrome, dev channel only: https://developer.chrome.com/docs/extensions/reference/api/d... )

That's not too hard to plug together, and Firefox even has stuff like `["offline"]` so you can avoid sending any requests at all.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#125

Post by ai_fry_ur_brain »

gruez wrote: post #66
ai_fry_ur_brain wrote: post #54

Earlier quoted context omitted.

Also, it doesnt apply to this person if they've had the same IP for years but if your ISP rotates IPs frequently (mine does everytime I reset modem) or you use 5G and CGNAT is being used, its almost a garuntee that your proxy has been labled as having been used by a residential proxy network. So many people have sketchy TV boxes or whatever other sketchy IOT decice that is a larp for using your network to sell bandwi…

>However, CF is unnecessarily making people on 5G connections from desktops do turnstiles as it looks like a scraper using a mobile proxy. This will become more and more of an issue as more laptops have 5G modems in them. What makes you think this is the cause, rather than something more straightforward like: CGNAT means more users are sharing IPs, and there's a higher chance that the IP pool gets contaminated by bad…

That could be the case, but Im just speaking from my experience I find I get way more turnstiles when using a 5G modem or proxying through my phone. Whatever they're doing, its not reducing collateral damage.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#126

Post by kordlessagain »

gonzalohm wrote: post #21
kordlessagain wrote: post #2

I concur and have been talking about this for a while. The fact is, Cloudflare is a man-in-the-middle. That's their focus, that's their purpose. They will limit your local crawler from accessing pages. They will demand you use their crawler. They will decrypt your traffic if they get a warrant. They always decrypt your traffic anyway, but they will give it to state actors if they demand it. That's not to say anyone s…

Careful I posted something similar in another Cloudfare thread and people threw at me like lions. They don't see anything wrong with one entity controlling most of the internet traffic

I'm done being careful.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#127

Post by kordlessagain »

gruez wrote: post #62
kordlessagain wrote: post #2

I concur and have been talking about this for a while. The fact is, Cloudflare is a man-in-the-middle. That's their focus, that's their purpose. They will limit your local crawler from accessing pages. They will demand you use their crawler. They will decrypt your traffic if they get a warrant. They always decrypt your traffic anyway, but they will give it to state actors if they demand it. That's not to say anyone s…

>They will limit your local crawler from accessing pages. They will demand you use their crawler. Source? According to cloudflare, their crawling service don't get any special treatment from their WAF/CDN.

Come on. I'm not here to Google things for you. Go investigate.

Re: Cloudflare CEO is lying to you about the bot traffic jump

#128

Post by gruez »

kordlessagain wrote: post #127
gruez wrote: post #62

Earlier quoted context omitted.

>They will limit your local crawler from accessing pages. They will demand you use their crawler. Source? According to cloudflare, their crawling service don't get any special treatment from their WAF/CDN.

Come on. I'm not here to Google things for you. Go investigate.

>Come on. I'm not here to Google things for you.

And what, I'm here to do research for your arguments?

Re: Cloudflare CEO is lying to you about the bot traffic jump

#129

Post by reconnecting »

Groxx wrote: post #119
reconnecting wrote: post #118

Earlier quoted context omitted.

This is really interesting. I indeed looked at this problem from the wrong perspective. I'm working on an open-source tool that could be useful for bot detection, but I'm still not confident that anyone would deploy it on-prem and make the setup/maintenance instead of just routing traffic through the cloud. Perhaps performance as a KPI could work. Thanks!

I think you'd definitely find some interest, e.g. anyone that intentionally avoids "the cloud" will want something local. Honestly I assume there are some of these already, monitoring apache/nginx/etc logs. Anubis is arguably similar and has been exploding lately, for example, though I'm not sure if it auto-updates its rules at all: https://github.com/TecharoHQ/anubis As to if it'd get enough interest: yea no idea at…

Our team develops a risk-based analytics system that we also use for bot detection. From our perspective, bots shouldn't be blindly blocked, but rather properly monitored and blocked only when necessary. Here is a live demo (1) to give you a general idea.

1. https://play.tirreno.com (admin/tirreno)

Re: Cloudflare CEO is lying to you about the bot traffic jump

#130

Post by [deleted] »

burnte wrote: post #44
simonw wrote: post #11

"Cloudflare CEO is lying" is a bit of an aggressive take when he linked to the exact data so you can see it for yourself - and that's how this article was able to analyze it: https://radar.cloudflare.com/traffic#bot-vs-human Update: I see the problem. Here's the full tweet: https://x.com/eastdakota/status/2062212701414187452 "Thought it would be end of 2027, then early 2027, but agentic traffic growing so fast that b…

It's aggressive but true. Spam traffic exceeded real email back in 2003. He's not even in the right DECADE to declare a first for non-human traffic. It's pure marketing BS that is only remotely true if you accept the conditions he goes to lengths to hide.

[deleted]
Post reply on HN