Live data from Hacker News

The Cloudflare outage might be a good thing

gist.github.com

61–70 of 209 posts

Re: The Cloudflare outage might be a good thing

#61
post #26

Earlier quoted context omitted.

> It would be a good thing, if it would cause anything to change. It obviously won't. I agree wholeheartedly. The only change is internal to these organizations (eg: CloudFlare, AWS) Improvements will be made to the relevant systems, and some teams internally will also audit for similar behavior, add tests, and fix some bugs. However, nothing external will change. The cycle of pretending like you are going to impleme…

the root cause is customers refusing to punish these downtime. Checkout how hard customers punish blackouts from the grid - both via wallet, but also via voting/gov't. It's why they are now more reliable. So unless the backbone infrastructure gets the same flak, nothing is going to change. After all, any change is expensive, and the cost of that change needs to be worth it.

Grid reliability depends on where you live. In some places, UPS or even a generator is a must have. So it's a bad example, I would say.

Re: The Cloudflare outage might be a good thing

#62
post #21

The problem is far more nuanced than the internet simply becoming too centralised. I want to host my gas station network’s air machine infrastructure, and I only want people in the US to be able to access it. That simple task is literally impossible with what we have allowed the internet to become. FWIW I love Cloudflare’s products and make use of a large amount of them, but I can’t advocate for using them in my prof…

not a sysadmin here. why wouldn't this be behind a VPN or some kind of whitelist where only confirmed IPs from the offices / gas stations have access to the infrastructure?

In practice, many gas stations have VPNs to various services, typically via multiple VPN links for redundancy. There’s no reason why this couldn’t be yet another service going over a VPN.

Gas stations didn’t stop selling gas during this outage. They have planned for a high degree of network availability for their core services. My guess is this particular station is an independent or the air pumping solution not on anyone’s high risk list.

Re: The Cloudflare outage might be a good thing

#63
post #10

It would be a good thing, if it would cause anything to change. It obviously won't. As if a single person reading this post wasn't aware that the Internet is centralized, and couldn't name specifically a few sources of centralization (Cloudflare, AWS, Gmail, Github). As if it's the first time this happens. As if after the last time AWS failed (or the one before that, or one before…) anybody stopped using AWS. As if a…

If anything, centralisation shields companies using a hyperscaler from criticism. You’ll see downtime no matter where you host. If you self host and go down for a few hours, customers blame you. If you host on AWS and “the internet goes down”, then customers treat it akin to an act of God, like a natural disaster that affects everyone. It’s not great being down for hours, but that will happen regardless. Most compani…

>If anything, centralisation shields companies using a hyperscaler from criticism. You’ll see downtime no matter where you host. If you self host and go down for a few hours, customers blame you.

What if you host on AWS and only you go down? How does hosting on AWS shield you from criticism?

Re: The Cloudflare outage might be a good thing

#64
post #36
post #22

Earlier quoted context omitted.

Literally impossible? On the contrary; Geofencing is easy. I block all kind of nefarious countries on my firewall, and I don't miss them (no loss not being able to connect to/from a mafia state like Russia). Now, if I were to block FAMAG... or Cloudflare...

Yes, literally impossible. The barrier to entry for anyone on the internet to create a proxy or VPN to bypass your geofencing is significantly lower than your cost to prevent them.

Actually, the 140k Tor exit nodes, VPNs, and compromised proxy servers have been indexed.

It takes 24 minutes to compile these firewall rules, but the black-list along with tripwires have proven effective at banning game cheats. Example, dropping connections from TX with a hop-count and latency significantly different from their peers.

Preemptively banning all bad-reputation cloud IP ranges except whitelisted hosts has zero impact on clients. =3

Re: The Cloudflare outage might be a good thing

#65
post #51

Earlier quoted context omitted.

> and I only want people in the US to be able to access it. That simple task is literally impossible with what we have allowed the internet to become. Is anyone else as confused as I am about how common anti-openness and anti-freedom comments are becoming on HN? I don’t even understand what this comment wants: Banning VPNs? Walling off the rest of the world from US internet? Strict government identity and citizenship…

> Is anyone else as confused as I am about how common anti-openness and anti-freedom comments are becoming on HN? In this specific case I don't think it's about being anti-open? It's that a business with only physical presence in one country selling a service that is only accessible physically inside the country.... doesn't.... have any need for selling compressed air to someone who isn't like 15 minutes away from on…

> In this specific case I don't think it's about being anti-open? It's that a business with only physical presence in one country selling a service that is only accessible physically inside the country.... doesn't.... have any need for selling compressed air to someone who isn't like 15 minutes away from one of their gas stations?

But that person might be physically further away at the time they want to order something or gather information etc. Maybe they are on holidays in Spain and want to access their account to pay a bill. Maybe they are in Mexico on a work trip and want to help their aunt back home to use some service for which they need to log in from abroad.

The other day I helped a neighbor (over here in Europe) prepare for a trip to Canada where he wanted to make adjustments to a car sharing account. The website always timed out. It was geofenced. I helped him set up a VPN. That illustrated how locked in this all has become, geofencing without thinking twice.

Re: The Cloudflare outage might be a good thing

#66
post #48
post #21

The problem is far more nuanced than the internet simply becoming too centralised. I want to host my gas station network’s air machine infrastructure, and I only want people in the US to be able to access it. That simple task is literally impossible with what we have allowed the internet to become. FWIW I love Cloudflare’s products and make use of a large amount of them, but I can’t advocate for using them in my prof…

> I want to host my gas station network’s air machine infrastructure, and I only want people in the US to be able to access it. That simple task is literally impossible with what we have allowed the internet to become. That task was never simple and is unrelated to Cloudflare or AWS. The internet at a fundamental level only knows where the next hop is, not where the source or destination is. And even if it did, it wo…

And that is a good thing and we should embrace it instead of giving in to some idiotic ideas from a non-technical C-suite demanding geofencing.

Re: The Cloudflare outage might be a good thing

#67

Earlier quoted context omitted.

If anything, centralisation shields companies using a hyperscaler from criticism. You’ll see downtime no matter where you host. If you self host and go down for a few hours, customers blame you. If you host on AWS and “the internet goes down”, then customers treat it akin to an act of God, like a natural disaster that affects everyone. It’s not great being down for hours, but that will happen regardless. Most compani…

>If anything, centralisation shields companies using a hyperscaler from criticism. You’ll see downtime no matter where you host. If you self host and go down for a few hours, customers blame you. What if you host on AWS and only you go down? How does hosting on AWS shield you from criticism?

This discussion is assuming that the outage is entirely out of your control because the underlying datacenter you relied on went down.

Outages because of bad code do happen and the criticism is fully on the company. They can be mitigated by better testing and quick rollbacks, which is good. But outages at the datacenter level - nothing you can do about that. You just wait until the datacenter is fixed.

This discussion started because companies are actually fine with this state of affairs. They are risking major outages but so are all their competitors so it’s fine actually. The juice isn’t worth the squeeze to them, unless an external entity like the banking regulator makes them care.

Re: The Cloudflare outage might be a good thing

#68
I'll die on the hill that centralization is more efficient than decentralization and that rare outages of hugely centralized systems that are otherwise highly reliable are much better than full decentralization with much worse reliability.

In other words, when AWS or Cloudflare go down it's catastrophic in the sense that everyone sees the issues at the same time, but smaller providers usually have much more ongoing issues, that just happen to be "chronic" vs "acute" pains.

Re: The Cloudflare outage might be a good thing

#69
post #19

Spot on article, but without a call to action. What can we do to combat the migration of society to a centralized corpro-government intertwined entity with no regard for unprofitable privacy or individualism?

Individuals are unlikely to be able to do something about the centralization problem except vote for politicians that want to implement countermeasures. I don’t know of any politicians (with a chance to win anything) that have that on their agenda.

There is a crucial step between having an opinion and voting. It's conversations within society. That's what makes democracy and facilitates change. If you only take your opiniom, isolated from everybody else, and vote from that, there isn't much democracy going on and your chance for change is slim. It's when there is broad conversations happening when movements have an impact.

And that step is here on HN. That's why it's very relevant to observe that that HN crowd is increasingly happy to support a non-free internet. Be it walled gardens, geofencing, etc.

Re: The Cloudflare outage might be a good thing

#70
So were going backwards to a world where there are basically 5 computers running everything and everyone is basically accessing the world through a dumb terminal.Even though the digital slab in our pockets has more compute than a roomful of the early gen devices. Hopefully critical infrashifts back to managed metal or private clouds - dont see it though with the last decades of cloud evangalism to move all legacy systems to the cloud doesnt look like reversing anytime soon.
Post reply on HN