Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Cloudflare outage might be a good thing

gist.github.com

Post reply on HN
Print view View original

171–180 of 209 posts

Re: The Cloudflare outage might be a good thing

#171

Post by Fnoord »

Joel_Mckay wrote: post #64
stroebs wrote: post #36

Earlier quoted context omitted.

Yes, literally impossible. The barrier to entry for anyone on the internet to create a proxy or VPN to bypass your geofencing is significantly lower than your cost to prevent them.

Actually, the 140k Tor exit nodes, VPNs, and compromised proxy servers have been indexed. It takes 24 minutes to compile these firewall rules, but the black-list along with tripwires have proven effective at banning game cheats. Example, dropping connections from TX with a hop-count and latency significantly different from their peers. Preemptively banning all bad-reputation cloud IP ranges except whitelisted hosts h…

I don't have a filter list for compromised proxy servers and VPNs. Do you have a link? I'd be interested in logging such. For Tor, I use [1] (formats in json, txt, md) on OPNsense, but I've also been able to indeed simply parse ASNs (which I currently use for "Twitter, Inc.").

> Preemptively banning all bad-reputation cloud IP ranges except whitelisted hosts has zero impact on clients. =3

This. There's outbound and inbound, and it is very unlikely your print server requires connections from Russia or China (to name an example). You're probably better off making a whitelist, jumphost, or using a VPN with proper authentication to access your services.

Outbound, now that is more difficult to assess. On a desktop, I like a personal firewall for that purpose. Little Snitch on macOS and Open Snitch on Linux have helped me a lot here, but ultimately your hardware firewall is probably lenient on outgoing connections, when you should ask yourself does my network require this, or are they better off with only a HTTP(S) proxy by default?

[1] https://github.com/7c/torfilter

Re: The Cloudflare outage might be a good thing

#172

Post by Dylan16807 »

dmoy wrote: post #153
teiferer wrote: post #65

Earlier quoted context omitted.

> In this specific case I don't think it's about being anti-open? It's that a business with only physical presence in one country selling a service that is only accessible physically inside the country.... doesn't.... have any need for selling compressed air to someone who isn't like 15 minutes away from one of their gas stations? But that person might be physically further away at the time they want to order somethi…

I guess GP didn't provide enough info, but to me it looked like it was the underlying infra that is networked That is I'm assuming: 1. Customers are meatspace only, never use any computer interface 2. The network access is for administration only 3. That administration is exclusively in the US

That's the most obvious answer but if that's the case then restricting to "US" is way too wide in the general case and also too narrow if an employee takes a trip to another country and tries to check in. That simple task is fundamentally flawed to the point it's not worth worrying about.

Re: The Cloudflare outage might be a good thing

#173

Post by Dylan16807 »

WJW wrote: post #121
Xelbair wrote: post #75

Earlier quoted context omitted.

Genuine question - why are you spending time and effort on geofencing when you could spend it on improving your software/service? It takes time and effort for no gain in any sensible business goal. People outside of US won't need it, bad actors will spoof their location, and it might inconvenience your real customers. And if you want a secure communication just setup zero-trust network.

> bad actors will spoof their location Isn't that exactly the point? Why are North Korean hackers even allowed to connect to the service, and why is spoofing location still so easy and unverifiable? Nobody is expected to personally secure their physical location against hostile state actors. My office is not artillery proof, nor does it need to be: hostile actions against it would be an act of war and we have the mil…

North Korea in particular is weird because of sanctions, but pick any country in Europe instead: The user might be a past or future visitor to the gas station and need to access the system even if they're outside the US right now. Or maybe they're actually at the gas station but their phone's data is based in Europe.

Even accurate country tracking is flawed in most situations.

If the goal is specifically "is at the gas station right now" then maybe there's a gap in functionality here, but you could make them connect to the wifi.

Also country-sponsored hackers can easily get a real presence in the US. If country level geoblocking became perfect, they wouldn't be slowed down for more than a week.

Re: The Cloudflare outage might be a good thing

#174

Post by beanjuiceII »

peaseagee wrote: post #154
beanjuiceII wrote: post #150

Earlier quoted context omitted.

what do you mean no lesson learned? seems like you haven't been paying attention..there's always a lesson learned

I believe they mean that Crowdstrike learned that they could screw up on this level and keep their customers....

that IS the lesson! there are a million questions i can ask myself about those incidents. What dictates they can't ever screw up? sure it was a big screw up, but understanding the tolerances for screw ups is important to understanding how fast and loose you can play it. AWS has at least a big outage a year, whats the breaking point? risk and reward etc.

I've worked places where every little thing is yak shaved, and places where no one is even sure if the servers are up during working hours. Both jobs paid well.. both jobs had enough happy customers

Re: The Cloudflare outage might be a good thing

#175

Post by bigiain »

Fnoord wrote: post #22
stroebs wrote: post #21

The problem is far more nuanced than the internet simply becoming too centralised. I want to host my gas station network’s air machine infrastructure, and I only want people in the US to be able to access it. That simple task is literally impossible with what we have allowed the internet to become. FWIW I love Cloudflare’s products and make use of a large amount of them, but I can’t advocate for using them in my prof…

Literally impossible? On the contrary; Geofencing is easy. I block all kind of nefarious countries on my firewall, and I don't miss them (no loss not being able to connect to/from a mafia state like Russia). Now, if I were to block FAMAG... or Cloudflare...

It is definitely "literally impossible" if your acceptable false positive and false negative rates are zero.

Having said that, vanishingly few companies/projects require that. For probably 99+% of websites, just using publicly available GeoIP databases to block countries will work just fine, so long as you don't pretend to yourself that North Korean or Chinese or Russian (or wherever) web users (or attackers) cannot easily get around that. And you'll also need to accept that occasionally a "local/wanted" user will end up with an IP address that gets blocked due to errors in the database.

I worked on a project a decade or so back where we needed to identify which (Australian) state a website user was in, to correctly display total driveaway prices including all state taxes/charges (stamp duty, ctp insurance, and registration) for new cars. The MaxMind GeoIP database was not all that accurate at a state or city level, especially for mobile devices with CGNATed IP addresses. We ended up with "known errors and estimates of error rates", and a way for our Javascript to detect some of the known problems (like Vodafone's national CGNAT IP addresses) and popped up a "We detected you're in NSW, and are displaying NSW pricing. Click here to change state." message where we could, and got legal signoff that we could claim "best effort" at complying with the driveway price laws. 100% compliance with the laws as-written was "literally impossible" with zero error rates.

Re: The Cloudflare outage might be a good thing

#176

Post by bigiain »

Aurornis wrote: post #44
stroebs wrote: post #36

Earlier quoted context omitted.

Yes, literally impossible. The barrier to entry for anyone on the internet to create a proxy or VPN to bypass your geofencing is significantly lower than your cost to prevent them.

I don’t even understand where this line of reasoning is going. Did you want a separate network blocked off from the world? A ban on VPNs? What are we supposed to believe could have been disallowed to make this happen?

There are a lot of lists around for known VPN endpoints and datacenter IP address ranges, that people use to reduce error rates in ip address to location lookups. That cannot possibly itself be 100% effective, but it can probably drop the error rate of semi-technical users switching their VPN location to circumvent your geo blocking by an order of magnitude or two. It certainly won't stop a sufficiently motivated technical of malicious user.

Re: The Cloudflare outage might be a good thing

#177

Post by eddd-ddde »

stroebs wrote: post #21

The problem is far more nuanced than the internet simply becoming too centralised. I want to host my gas station network’s air machine infrastructure, and I only want people in the US to be able to access it. That simple task is literally impossible with what we have allowed the internet to become. FWIW I love Cloudflare’s products and make use of a large amount of them, but I can’t advocate for using them in my prof…

I absolutely hate companies thinking they are being smart by blocking foreign IPs from using their websites.

Every single time I want to order a burger from the local place, I have to use a VPN to fake being in the country (even though I actually am already physically here) so that it will let me give them my money.

My phone's plan is not from here, so my IP address is actually not geographically in the same place as me.

Re: The Cloudflare outage might be a good thing

#178

Post by tracker1 »

ectospheno wrote: post #149
krick wrote: post #10

It would be a good thing, if it would cause anything to change. It obviously won't. As if a single person reading this post wasn't aware that the Internet is centralized, and couldn't name specifically a few sources of centralization (Cloudflare, AWS, Gmail, Github). As if it's the first time this happens. As if after the last time AWS failed (or the one before that, or one before…) anybody stopped using AWS. As if a…

I’m pretty cloudflare centric. I didn’t start that way. I had services spread out for redundancy. It was a huge pain. Then bots got even more aggressive than usual. I asked why I kept doing this to myself and finally decided my time was worth recapturing. Did everything become inaccessible the last outage? Yep. Weighed against the time it saves me throughout the year I call it a wash. No plans to move.

I'm of a similar mindset... yeah, it's inconvenient when "everything" goes down... but realistically so many things go down now and then, it just happens.

Could just as easily be my home's internet connection, or a service I need from/at work, etc. It's always going to be something, it's just more noticeable when it affects so many other things.

Re: The Cloudflare outage might be a good thing

#179

Post by tracker1 »

captainkrtek wrote: post #16
krick wrote: post #10

It would be a good thing, if it would cause anything to change. It obviously won't. As if a single person reading this post wasn't aware that the Internet is centralized, and couldn't name specifically a few sources of centralization (Cloudflare, AWS, Gmail, Github). As if it's the first time this happens. As if after the last time AWS failed (or the one before that, or one before…) anybody stopped using AWS. As if a…

> It would be a good thing, if it would cause anything to change. It obviously won't. I agree wholeheartedly. The only change is internal to these organizations (eg: CloudFlare, AWS) Improvements will be made to the relevant systems, and some teams internally will also audit for similar behavior, add tests, and fix some bugs. However, nothing external will change. The cycle of pretending like you are going to impleme…

And even in multi-region, you experience a DNS failure and it all goes up in flames anyway. There's always going to be something.

Re: The Cloudflare outage might be a good thing

#180

Post by bcrl »

timenotwasted wrote: post #14

"Embrace outages, and build redundancy." — It feels like back in the day this was championed pretty hard especially by places like Netflix (Chaos Monkey) but as downtime has become more expected it seems we are sliding backwards. I have a tendency to rely too much on feelings so I'm sure someone could point me to some data that proves otherwise but for now that's my read on things. Personally, I've been going a lot m…

I have cell phone calls regularly drop during tower handoffs, and codec errors that result in a blast of static upon answering a call. I can't remember a single time I had a phone call fail on the old PSTN built out of DMS10 and DMS100s locally (well, until we lost all trunks due to a fibre issue a couple of weeks ago on November 10th -- the incumbent didn't notice the outage which started at ~3:20am until ~9:30am, and it wasn't fixed until 17:38). One time when I was a teenager in the '90s, a friend and I had a 14 hour call using landlines.

The modern tech stack is disappointing in its lack of reliability. Complexity is the root of all evil.

Post reply on HN