Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS says it can't restore some data from mideast facilities struck by Iran

wsj.com

Post reply on HN
Print view View original

201–210 of 463 posts

Re: AWS says it can't restore some data from mideast facilities struck by Iran

#201

Post by marcosdumay »

houssc wrote: post #119
krick wrote: post #109

Earlier quoted context omitted.

That is actually surprising to me. Claims like that are pretty common, they make sense and they should be true, so even though I don't really know AWS (/Backblaze/Azure/whatever) redundancy planning in enough detail, I used to trust them. It's really worrying when they outright say it will be ok, and then a week later it turns out to be not ok.

The caveat is always "if you're using the service correctly " which is not necessarily free. Meaning taking advantage of multiple geo zones, building in redundancy to your stack, etc. Like everything he said is possible if your technology stack living in AWS was designed to survive it. Everyone who has ever had the "we lost your data" email from AWS knows at the end of the day the cloud is just someone else's data ce…

No, that quotation on the GP clearly states that AWS has enough redundancy within the same region that they will continue all services running on it if a datacenter is destroyed.

It's very clearly not about you being able to set-up redundancy for yourself.

Re: AWS says it can't restore some data from mideast facilities struck by Iran

#202

Post by Avicebron »

jacquesm wrote: post #130
houssc wrote: post #119

Earlier quoted context omitted.

The caveat is always "if you're using the service correctly " which is not necessarily free. Meaning taking advantage of multiple geo zones, building in redundancy to your stack, etc. Like everything he said is possible if your technology stack living in AWS was designed to survive it. Everyone who has ever had the "we lost your data" email from AWS knows at the end of the day the cloud is just someone else's data ce…

The problem is that lots of people seem to be under the impression that they are doing it right because they are using AWS. They don't realize that AWS is a toolbox, not a 'ready made solution for redundancy against all catastrophes you are possibly exposed to'. They use that to their advantage by pricing such solutions at a level that people will either pay through the nose or will be left without recourse when AWS…

Not to be callous but even the 3-2-1 rule is pretty basic, the issue is that people don't apply it. But that's a hiring and strategy thing.

Re: AWS says it can't restore some data from mideast facilities struck by Iran

#203

Post by ployable7 »

the8472 wrote: post #127
mpyne wrote: post #107

Earlier quoted context omitted.

Are you saying that most data loss happens because your data center gets blown up in a shooting war? Like, AWS is the first digital service provider to lose data in decades?

I'm saying that if you have eliminated more mundane failures like dying harddrives, cosmic rays and so on from your systems and your calculation ends up with 11 nines then actually those "force majeure" events are probable enough that they dominate whatever other residuals are supposedly hiding in those last 0.0000000001%. The region has seen a bunch of wars in the last 100 years, so the annual war-rate is > 1%. Even…

You don’t want to blend probabilities like this, because the tactics you use as a consumer vary between the two. If you consider 11 9s like “object AFR”, you might build systems that are resilient to very occasional single object loss. And it’s useful to know at what rate that might occur.

Whereas with these force majeure events you’d want a complete DR setup, and it’s typically an async recovery. Here it is useful to understand the fault domain (single server or single building or multi-building) so you can plan.

Blending the two numbers doesn’t help you build better against the systems. And the force majeure events are rare enough that they won’t happen … until they do. I’m not sure that knowing the precise probability that Iran would attack a gulf nation would change the fact that if they do, you need to have a DR story.

Re: AWS says it can't restore some data from mideast facilities struck by Iran

#204

Post by bonestamp2 »

wand3r wrote: post #168
dannyobrien wrote: post #122

Earlier quoted context omitted.

Hi, I'm from the past. When countries in the 2010s -- especially Western countries -- started seeing data residency requirements as an acceptable aspect of national policies, as opposed to a weird authoritarian thing that only China and Russia imposed on their citizens, we[1] spent a bunch of time explaining to their lawmakers that having geographical redundancy was a good thing, actually, and that you should stop in…

This is a very engineer-centric view. I studied economics in school, so an analogy in that realm is ironically how all countries should specialize and raise the PPC curve. The reality of the situation was that in 2010 not many people understood how powerful big data actually was. Data sovereignty is actually quite logical when you consider the scale and power of not only the company, but the US as a whole. I can assu…

> You can also create reasonably diversified data silos within a country.

I generally agree, although if a small country only had half a dozen or so redundant data centers then it would be relatively easy for a powerful adversary to wipe out all of the data centers and potentially have a significant economic impact on that country.

Having a backup data center in an ally country might make sense. Kind of like how I keep an encrypted backup hard drive at my parents house. Whenever I go to visit I pull it out and backup my laptop there too.

Re: AWS says it can't restore some data from mideast facilities struck by Iran

#205

Post by kjs3 »

dannyobrien wrote: post #195
kjs3 wrote: post #189

Earlier quoted context omitted.

That's fair, and it sounds like we aren't that far apart. It is, in fact, about control. So I'll restate my central theme as "until the idea of enforceable data sovereignty requirements were enshrined in law, the cloud providers did not and would not delegate control of any body of data to 'controllers' that weren't in jurisdictions where they could be influenced/coerced to compromise that data". Was this a slippery…

Yep, exactly. There's a peculiarly unsatisfying kind of vindication that comes from making "slippery slope" arguments, and then watch them play, and now you are now both a) technically correct, and b) fucked. You'll excuse me if I have a brief "I told you so" moment about a scenario about Amazon's UAE datacenters being bombed without bakcups because of a US-instigated Iranian conflict, where -- if I'd ever dared to d…

I guess we aren't really close.

You're saying (correct me if I'm wrong) that data sovereignty laws are unconditionally bad because they inevitably lead to totalitarian followon laws and there's nothing to stop them. I'm saying if we didn't have enforceable data sovereignty laws we would be in worse shape for data privacy and we should have prevented the followon laws from coming to be (and, true enough, we didn't).

Further, the UAE datacenter issue is a red herring. It's an engineering issue not a political one (data sovereignty without physical redundancy is...stupid?), but schadenfreude is a helluva drug.

Re: AWS says it can't restore some data from mideast facilities struck by Iran

#207

Post by testbjjl »

houssc wrote: post #119
krick wrote: post #109

Earlier quoted context omitted.

That is actually surprising to me. Claims like that are pretty common, they make sense and they should be true, so even though I don't really know AWS (/Backblaze/Azure/whatever) redundancy planning in enough detail, I used to trust them. It's really worrying when they outright say it will be ok, and then a week later it turns out to be not ok.

The caveat is always "if you're using the service correctly " which is not necessarily free. Meaning taking advantage of multiple geo zones, building in redundancy to your stack, etc. Like everything he said is possible if your technology stack living in AWS was designed to survive it. Everyone who has ever had the "we lost your data" email from AWS knows at the end of the day the cloud is just someone else's data ce…

Pretty much this, it’s your responsibility to use their tools to make sure your data is managed in such a way that any data destroyed is already elsewhere before the event.

Re: AWS says it can't restore some data from mideast facilities struck by Iran

#208

Post by gilbetron »

11.3 Force Majeure. Except for payment obligations, neither party nor any of their affiliates will be liable for any delay or failure to perform any obligation under this Agreement where the delay or failure results from any cause beyond its reasonable control, including acts of God, labor disputes or other industrial disturbances, electrical or power outages, utilities or other telecommunications failures, earthquake, storms or other elements of nature, blockages, embargoes, riots, acts or orders of government, acts of terrorism, or war.

Re: AWS says it can't restore some data from mideast facilities struck by Iran

#209

Post by dannyobrien »

kjs3 wrote: post #205
dannyobrien wrote: post #195

Earlier quoted context omitted.

Yep, exactly. There's a peculiarly unsatisfying kind of vindication that comes from making "slippery slope" arguments, and then watch them play, and now you are now both a) technically correct, and b) fucked. You'll excuse me if I have a brief "I told you so" moment about a scenario about Amazon's UAE datacenters being bombed without bakcups because of a US-instigated Iranian conflict, where -- if I'd ever dared to d…

I guess we aren't really close. You're saying (correct me if I'm wrong) that data sovereignty laws are unconditionally bad because they inevitably lead to totalitarian followon laws and there's nothing to stop them. I'm saying if we didn't have enforceable data sovereignty laws we would be in worse shape for data privacy and we should have prevented the followon laws from coming to be (and, true enough, we didn't). F…

Oh man, I was so close.

No, I'm making no "unconditionality" claim here: there are just risks and benefits. Sometimes you're the person in the room highlighting the potential problems. The risk with doing that is that when those problems don't happen, you look like a fool. But someone should raise the problems anyway, because that's part of the risk assessment!

Of course, if the problems do happen, then you get to indulge in "I told you so". But only if you failed to convince anyone at the time.

Post reply on HN