Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You don’t want to be on Cloudflare’s naughty list

ctrl.blog

Post reply on HN
Print view View original

111–120 of 354 posts

Re: You don’t want to be on Cloudflare’s naughty list

#111

Post by ufmace »

superkuh wrote: post #6

Daniel Aleksandersen of ctrl.blog has absolutely no foot to stand on here. He is a proponent of this kind of algorithmic blocking for weird browsers and even implemented it on his own site and argued for it. https://www.ctrl.blog/entry/detect-non-browser-form-submissi... It's only after it happened to him that now he's suddenly against it. Until he removes the same type of blocks from his own website I have absolutel…

I just read it, and I don't see any contradiction here. IMO, he's recommending simple and direct anti-bot methods to web admins specifically because it's better than relying solely on Cloudflare etc for all bot blocking. He never recommends making un-appealable access control decisions based on third-party lists, and specifically recommends caution on methods that might potentially impact innocent users. Seems perfectly consistent to me.

Re: You don’t want to be on Cloudflare’s naughty list

#112

Post by pessimizer »

Veen wrote: post #86
trasz wrote: post #79

Earlier quoted context omitted.

It’s naive to assume Cloudflare CTO would not be lying if beneficial to him or Cloudflare.

It's even more naive to assume Cloudflare's CTO would tell lies that can be trivially shown to be untrue.

No post body was provided.

Re: You don’t want to be on Cloudflare’s naughty list

#113

Post by jgrahamc »

pessimizer wrote: post #110
judge2020 wrote: post #31

Earlier quoted context omitted.

The point is that Cloudflare does not beam IP reputation data to Google. If Google and CF are blocking this IP separately, what's the chance there's some malicious device or hacked IoT device on the network, participating in DDOS attacks or unauthorized vulnerability scanning of random websites?

According to another comment, it's a wrong point: https://blog.cloudflare.com/cleaning-up-bad-bots/ > Once enabled, when we detect a bad bot, we will do three things: (1) we’re going to disincentivize the bot maker economically by tarpitting them, including requiring them to solve a computationally intensive challenge that will require more of their bot’s CPU; (2) for Bandwidth Alliance partners, we’re going to hand…

Yeah. I'm looking into that.

Re: You don’t want to be on Cloudflare’s naughty list

#114

Post by d2wa »

jgrahamc wrote: post #106
d2wa wrote: post #100

Earlier quoted context omitted.

> That's not what Bandwidth Alliance is at all. It's about reducing or eliminating egress fees between a cloud provider and Cloudflare. Not sure where the idea that it's about sharing IP reputation data comes from. It comes from the Cloudflare blog. https://blog.cloudflare.com/cleaning-up-bad-bots/ There’s a support page about it too. https://developers.cloudflare.com/bots/get-started/free/

I need to look into that. Thanks for pointing it out. I had totally forgotten about that post. Edit: team tells me this idea never got off the ground. Did talk with some potential partners (which did NOT include Google) but didn’t happen. So if Google was throwing CAPTCHAs it wasn’t because of our IP reputation.

I edited and added a second link to a support page that mentions it too.

Re: You don’t want to be on Cloudflare’s naughty list

#115

Post by johnklos »

andrewnyr wrote: post #101
johnklos wrote: post #84

Imagine all the people in countries deemed less desirable by Cloudflare that go through this all the time. Cloudflare, whether it's their stated goal or not, is re-stratifying and re-centralizing the Internet because of their desire to be a monopoly, and we'll all suffer as a result.

there are multiple other large CDNs out there... its a lot more like 5 market leaders tbh

But how many of them:

1) refuse to take responsibility for content they host by claiming they don't host

2) discriminate against huge parts of the Internet with no publicly known rules, nor methods to change that discrimination

3) make the abuse reporting process intentionally difficult and time-consuming

4) want to aggregate all the DNS data they can by making a deal with Firefox to turn on DNS-over-https by default without asking or even informing end users

5) want to re-centralize the Internet, in part so they can mix bad actors with good, in ways that make blocking next to impossible

How many of them do the discrimination we're all writing about here?

Re: You don’t want to be on Cloudflare’s naughty list

#116

Post by alexforster »

d2wa wrote: post #98
DethNinja wrote: post #8

There is a chance you might’ve been hacked. You would be surprised to see how easy it is to hack domestic routers. 1. Find and disinfect the devices, including the router. If you don’t have enough technical knowledge, then buy a new router. 2. Use 30 character long random password on the router. 3. Disable UPnP. 4. Anything with WI-FI and weak password can be hacked within minutes, so check your other devices as well…

(Author here.) My router isn’t a domestic router. It’s a MikroTik running RouterOS, completely unsupported by the ISP. Outgoing connections and DNS is logged. UPnP is only allowed for the Xbox, PS4, and off-most-of-the-time gaming PC. Nothing out of the ordinary in the logs.

> It’s a MikroTik running RouterOS

https://google.com/search?q=mikrotik+botnet

These things are the absolute scourge of the internet.

Re: You don’t want to be on Cloudflare’s naughty list

#117

Post by bornfreddy »

d2wa wrote: post #94
bastardoperator wrote: post #44

Has he tried unplugging the router for 15 minutes and plugging it back in? I jest but I know Comcast and Spectrum will both issue a new IP address in that timeframe.

(Author.) My ISP only rotates IPs when they reboot their central equipment. Not enough to do it on my end.

With some ISPs, they will issue a new IP if you change the router's (WAN) MAC address. Might be worth a try next time (crossing fingers you don't need it).

Re: You don’t want to be on Cloudflare’s naughty list

#118

Post by jgrahamc »

d2wa wrote: post #114
jgrahamc wrote: post #106

Earlier quoted context omitted.

I need to look into that. Thanks for pointing it out. I had totally forgotten about that post. Edit: team tells me this idea never got off the ground. Did talk with some potential partners (which did NOT include Google) but didn’t happen. So if Google was throwing CAPTCHAs it wasn’t because of our IP reputation.

I edited and added a second link to a support page that mentions it too.

Thanks. I'm talking with the team.

Edit: see comment above.

Re: You don’t want to be on Cloudflare’s naughty list

#119

Post by d2wa »

jgrahamc wrote: post #106
d2wa wrote: post #100

Earlier quoted context omitted.

> That's not what Bandwidth Alliance is at all. It's about reducing or eliminating egress fees between a cloud provider and Cloudflare. Not sure where the idea that it's about sharing IP reputation data comes from. It comes from the Cloudflare blog. https://blog.cloudflare.com/cleaning-up-bad-bots/ There’s a support page about it too. https://developers.cloudflare.com/bots/get-started/free/

I need to look into that. Thanks for pointing it out. I had totally forgotten about that post. Edit: team tells me this idea never got off the ground. Did talk with some potential partners (which did NOT include Google) but didn’t happen. So if Google was throwing CAPTCHAs it wasn’t because of our IP reputation.

Dear John. What am I — as a normal human being/end-user — supposed to do in this situation? People can’t do anything without any information about why they’re blocked. Who do you contact? Where do you go? What to do? The challenge page doesn’t help the end user understand why this is happening to them. It’s okay if you only see it for two seconds. But the page stays on screen for over a minute. When this happens for every website — what do you do? You’d be furious if this had happen to you. I’m just trying to read my online comics and lookup some stuff about some interests and hobbies. It reduced my quality of life/sanity for a week. The last two days, I started worrying that this was going to be the new normal. I even looked into swapping ISP to get a new IP address.

PS: I love all the innovation and engineering stuff you guys regularly share on the Cloudflare blog. It’s [almost] always an interesting read. Even though I’m no fan of the massive centralization your company has caused.

Re: You don’t want to be on Cloudflare’s naughty list

#120

Post by toast0 »

LinuxBender wrote: post #40
daenney wrote: post #26

Earlier quoted context omitted.

Burn the witch! Lets read through that page for a second though: Drop support for obsolete HTTP versions Doesn't seem like that's going to cause much issue for any legitimate client from the past 10-20 years. He only recommends blocking HTTP 0.9/1.0, which fair enough Append a #hash to the form’s action URL Hah. Clever man. I don't see how this is going to stop any legitimate user from loading your website or submitt…

He's quite tame compared to me I suppose. I block anything that is not HTTP/2.0 which currently knocks out all the bots and all crawlers except Bing. But I just have hobby sites these days. Nobody would notice or care if my sites went offline. Using NGinx as an example: if ($server_protocol != HTTP/2.0) { return 403 'Nope'; } Another thing I have found useful to drops some bots is to become invisible to them. Many of…

> Many of the poorly written scanning tools do not properly set MSS for reasons I still don't understand.

MSS issues attract me like a moth to flame [1], so let me ask some questions.

It looks like this is dropping syns with MSS over 16384??? That is indeed a pretty crazy high number. 9000ish seems reasonable for someone on a jumbo network without a mss clamping router, but above that is someone weird for sure.

Under 420 seems unlikely too, but technically acceptable, but sure, I'd drop it. In theory, a proper OS will send several SYNs with MSS, then assume your server doesn't support TCP options and send you a SYN with no options. Going to take a while, but if someone legitimately has a mss less than 536, their internet is probably pretty junky anyway, so ok, seems fine.

[1] I just built a browser based pmtud test site, http://pmtud.enslaves.us/

Post reply on HN