Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Questions for Cloudflare

entropicthoughts.com

Post reply on HN
Print view View original

31–40 of 71 posts

Re: Questions for Cloudflare

#31

Post by colesantiago »

vlovich123 wrote: post #18
colesantiago wrote: post #10

Maybe instead of asking "questions" to a corporation which their only interest is profit, is now beholden Wall Street and wouldn't care what we think, we should look for answers and alternatives like BunnyCDN [0], Anubis [1], etc. [0] https://bunny.net/ [1] https://github.com/TecharoHQ/anubis

Bunny has raised money from VC which indicates it’s going the “Wall Street” path. Anubis is a bot firewall not a CDN.

> Bunny has raised money from VC which indicates it’s going the “Wall Street” path.

Yet it is an available alternative to Cloudflare that is not on Wall Street (a public company).

If you want to do this 100% yourself there is Apache Traffic Control.

https://github.com/apache/trafficcontrol

> Anubis is a bot firewall not a CDN.

For now. If we support alternatives they can grow into an open source CDN.

Re: Questions for Cloudflare

#32

Post by Nextgrid »

psim1 wrote: post #23
Nextgrid wrote: post #14

It is unfair to blame Cloudflare (or AWS, or Azure, or GitHub) for what’s happening, and I say that as one of the biggest “yellers at the cloud” on here. Ultimately end-users don’t have a relationship with any of those companies. They have relationships with businesses that chose to rely on them. Cloudflare, etc publish SLAs and compensation schedules in case those SLAs are missed. Businesses chose to accept those SL…

> It is unfair to blame Cloudflare (or AWS, or Azure, or GitHub) for what’s happening > Ultimately end-users don’t have a relationship with any of those companies. They have relationships with businesses that chose to rely on them Could you not say this about any supplier relationship? No, in this case, we all know the root of the outage is CloudFlare, so it absolutely makes sense to blame CloudFlare, and not their c…

Devil’s advocate: I operate the equivalent of an online lemonade stand, some shitty service at a cheap price offered with little guarantees (“if I fuck up I’ll refund you the price of your ‘lemonade’”) for hobbyists to use to host their blog and Visa decides to use it in their critical path. Then this “lemonade stand” goes down. Do you think it’s fair to blame me? I never chose to be part of Visa’s authorization loop, and after all is done I did indeed refund them the price of their “lemonade”. It’s Visa’s fault they introduced a single point of failure with inadequate compensation schedules in their critical path.

Re: Questions for Cloudflare

#33

Post by stronglikedan »

psim1 wrote: post #23
Nextgrid wrote: post #14

It is unfair to blame Cloudflare (or AWS, or Azure, or GitHub) for what’s happening, and I say that as one of the biggest “yellers at the cloud” on here. Ultimately end-users don’t have a relationship with any of those companies. They have relationships with businesses that chose to rely on them. Cloudflare, etc publish SLAs and compensation schedules in case those SLAs are missed. Businesses chose to accept those SL…

> It is unfair to blame Cloudflare (or AWS, or Azure, or GitHub) for what’s happening > Ultimately end-users don’t have a relationship with any of those companies. They have relationships with businesses that chose to rely on them Could you not say this about any supplier relationship? No, in this case, we all know the root of the outage is CloudFlare, so it absolutely makes sense to blame CloudFlare, and not their c…

If I'm paying a company that chose Cloudflare, and my SLA with that company entitles me to some sort of compensation for outages, then I expect that company to compensate me regardless of whose fault it is, and regardless of whether they were compensated by Cloudflare. I can know that the cause of the outage is Cloudflare, but also know that the company that I'm paying should have had a backup plan and not be solely reliable on one vendor. In other words, I care about who I pay, not who they decide to use.

Re: Questions for Cloudflare

#34

Post by stronglikedan »

Nextgrid wrote: post #32
psim1 wrote: post #23

Earlier quoted context omitted.

> It is unfair to blame Cloudflare (or AWS, or Azure, or GitHub) for what’s happening > Ultimately end-users don’t have a relationship with any of those companies. They have relationships with businesses that chose to rely on them Could you not say this about any supplier relationship? No, in this case, we all know the root of the outage is CloudFlare, so it absolutely makes sense to blame CloudFlare, and not their c…

Devil’s advocate: I operate the equivalent of an online lemonade stand, some shitty service at a cheap price offered with little guarantees (“if I fuck up I’ll refund you the price of your ‘lemonade’”) for hobbyists to use to host their blog and Visa decides to use it in their critical path. Then this “lemonade stand” goes down. Do you think it’s fair to blame me? I never chose to be part of Visa’s authorization loop…

> Do you think it’s fair to blame me?

Absolutely, yes. Where's your backup plan for when Visa doesn't behave as you expect? It's okay to not have one, but it's also your fault for not having one, and that is the sole reason that the lemonade stand went down.

Re: Questions for Cloudflare

#35

Post by cogman10 »

timenotwasted wrote: post #9

"I don’t know. I wish technical organisations would be more thorough in investigating accidents." - This is just armchair quarterbacking at this point given that they were forthcoming during the incident and had a detailed post-mortem shortly after. The issue is that by not being a fly on the wall in the war room the OP is making massive assumptions about the level of discussions that take place about these types of…

People outside of tech (and some inside) can be really bad at understanding how something like this could slip through the cracks.

Reading cloudflare's description of the problem, this is something that I could easily see my own company missing. It's the case that a file got too big which tanked performance enough to bring everything down. That's a VERY hard thing to test for. Especially since this appears to have been a configuration file and a regular update.

The reason it's so hard to test for is because all tests would show that there's no problem. This isn't a code update, it was a config update. Without really extensive performance tests (which, when done well, take a long time!) there really wasn't a way to know that a change that appeared safe wasn't.

I personally give Cloudflare a huge pass for this. I don't think this happened due to any sloppiness on their part.

Now, if you want to see a sloppy outage you look at the Crowdstrike outage from a few years back that bricked basically everything. That is what sheer incompetence looks like.

Re: Questions for Cloudflare

#36

Post by vlovich123 »

colesantiago wrote: post #31
vlovich123 wrote: post #18

Earlier quoted context omitted.

Bunny has raised money from VC which indicates it’s going the “Wall Street” path. Anubis is a bot firewall not a CDN.

> Bunny has raised money from VC which indicates it’s going the “Wall Street” path. Yet it is an available alternative to Cloudflare that is not on Wall Street (a public company). If you want to do this 100% yourself there is Apache Traffic Control. https://github.com/apache/trafficcontrol > Anubis is a bot firewall not a CDN. For now. If we support alternatives they can grow into an open source CDN.

Anubis is a piece of software not a CDN service.

You realize to run a CDN you have to buy massive amounts of bandwidth and computers? DIY here belies a misunderstanding of what it takes to be DOS resistant and also what it takes to actually have CDN deliver a performance benefit.

Re: Questions for Cloudflare

#37

Post by colesantiago »

koakuma-chan wrote: post #28
vlovich123 wrote: post #18

Earlier quoted context omitted.

Bunny has raised money from VC which indicates it’s going the “Wall Street” path. Anubis is a bot firewall not a CDN.

I wouldn't trust a provider that has "Excellent (underlined) star star star star star STAR TrustPilot 4.8 on G2" on their landing page. I bet they are also award winning, and 150 best place to work at. Really shows they have no taste.

?

I don't remember telling anyone to trust the reviews?

I think it is healthy to try alternatives to Cloudflare and then come to your own decision.

Re: Questions for Cloudflare

#38

Post by kqr »

timenotwasted wrote: post #9

"I don’t know. I wish technical organisations would be more thorough in investigating accidents." - This is just armchair quarterbacking at this point given that they were forthcoming during the incident and had a detailed post-mortem shortly after. The issue is that by not being a fly on the wall in the war room the OP is making massive assumptions about the level of discussions that take place about these types of…

The article makes no claim about the effort that has gone into the analysis. You can apply a lot of effort and still only produce a shallow analysis.

If the analysis has not uncovered the feedback problems (even with large effort, or without it), my argument is that a better method is needed.

Re: Questions for Cloudflare

#39

Post by wongarsu »

psim1 wrote: post #23
Nextgrid wrote: post #14

It is unfair to blame Cloudflare (or AWS, or Azure, or GitHub) for what’s happening, and I say that as one of the biggest “yellers at the cloud” on here. Ultimately end-users don’t have a relationship with any of those companies. They have relationships with businesses that chose to rely on them. Cloudflare, etc publish SLAs and compensation schedules in case those SLAs are missed. Businesses chose to accept those SL…

> It is unfair to blame Cloudflare (or AWS, or Azure, or GitHub) for what’s happening > Ultimately end-users don’t have a relationship with any of those companies. They have relationships with businesses that chose to rely on them Could you not say this about any supplier relationship? No, in this case, we all know the root of the outage is CloudFlare, so it absolutely makes sense to blame CloudFlare, and not their c…

Don't we say that about all supplier relationships? If my Samsung washing machine stops working I blame Samsung. Even when it turns out that it was a broken drive belt I don't blame the manufacturer of the drive belt, or whoever produced the rubber that went into the drive belt, or whoever made the machine involved in the production of this batch of rubber. Samsung choose to put the drive belt in my washing machine, that's where the buck stops. They are free to litigate the matter internally, but I only care about Samsung selling me a washing machine that's now broken

Same with cloudflare. If you run your site on cloudflare you are responsible for any downtime caused to your site by cloudflare

What we can blame cloudflare for is having so many customers that a cloudflare outage has outsized impact compared to the more uncorrelated outages we would have if sites were distributed among many smaller providers. But that's not quite the same as blaming any individual site being down on cloudflare

Re: Questions for Cloudflare

#40

Post by Nextgrid »

mschuster91 wrote: post #29
Nextgrid wrote: post #14

It is unfair to blame Cloudflare (or AWS, or Azure, or GitHub) for what’s happening, and I say that as one of the biggest “yellers at the cloud” on here. Ultimately end-users don’t have a relationship with any of those companies. They have relationships with businesses that chose to rely on them. Cloudflare, etc publish SLAs and compensation schedules in case those SLAs are missed. Businesses chose to accept those SL…

> look at the uptime of card networks, stock exchanges, or airplane avionics. In fact, I'd say... airplane avionics are not what you should be looking at. Boeing's 787? Reboot every 51 days or risk the pilots getting wrong airspeed indicators! No, I'm not joking [1], and it's not the first time either [2], and it's not just Boeing [3]. [1] https://www.theregister.com/2020/04/02/boeing_787_power_cycl... [2] https://ww…

> Reboot every 51 days or risk the pilots getting wrong airspeed indicators

If this is documented then fair enough - airlines don’t have to buy airplanes that need rebooting every 51 days, they can vote with their wallets and Boeing is welcome to fix it. If not documented, I hope regulators enforced penalties high enough to force Boeing to get their stuff together.

Either way, the uptime of avionics (and redundancies - including the unreliable airspeed checklists) are much higher than anything conventional software “engineering” has been putting out the past decade.

Post reply on HN