Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS says it can't restore some data from mideast facilities struck by Iran

wsj.com

Post reply on HN
Print view View original

191–200 of 457 posts

Re: AWS says it can't restore some data from mideast facilities struck by Iran

#191

Post by dannyobrien »

wand3r wrote: post #168
dannyobrien wrote: post #122

Earlier quoted context omitted.

Hi, I'm from the past. When countries in the 2010s -- especially Western countries -- started seeing data residency requirements as an acceptable aspect of national policies, as opposed to a weird authoritarian thing that only China and Russia imposed on their citizens, we[1] spent a bunch of time explaining to their lawmakers that having geographical redundancy was a good thing, actually, and that you should stop in…

This is a very engineer-centric view. I studied economics in school, so an analogy in that realm is ironically how all countries should specialize and raise the PPC curve. The reality of the situation was that in 2010 not many people understood how powerful big data actually was. Data sovereignty is actually quite logical when you consider the scale and power of not only the company, but the US as a whole. I can assu…

See my other answers, but briefly: no, they were not thinking about this, which is why we were raising it. I guess the counterintuitive point we were trying to get across is that with most things, the best way to keep it safe from being lost is to put it in a known place, and lock it away. But for data, the strategy -- for that scenario -- is to keep it in a lot of places, with heterogenous defense strategies. This is for data loss, of course, not access or surveillance or unlawful processing. But there is a cost as well as a benefit to deliberately limiting your options.

(I can feel someone saying "but surely having redundancy in one country is good enough, so I'll just say that I know relatively sane people who try to have hemispheric redundancy in their data, and also you never know when two different-in-every-quality-but one locations will suffer from the same disaster. Floods; heat-waves; national protests and strikes. It's surprising how often rare things happen!)

On your second point, it really is crazy. And also amazing that this is a capability that is -- or should be -- available to anyone in the world, not just in the US, and not just devs. Hopefully without also having to think about their data suddenly finding itself in a warzone.

Re: AWS says it can't restore some data from mideast facilities struck by Iran

#193

Post by firesteelrain »

niij wrote: post #175
tgsovlerkhgsel wrote: post #120

Earlier quoted context omitted.

They're likely going to target two datacenters, not the datacenters + your medium sized company's office NAS and the safe in the office manager's home. (Encryption handles confidentiality concerns.)

> (Encryption handles confidentiality concerns.) Which is why data residency is such a stupid concept.

Yes and no. For example if you are doing Azure, technically Azure can see tenant traffic I believe and you need to use both a Platform Key and CMK for data rest. VMs need encryption at host turned on too.

There is nothing to say that a determined adversary may still get at your data so it needs to stay in country.

Re: AWS says it can't restore some data from mideast facilities struck by Iran

#195

Post by dannyobrien »

kjs3 wrote: post #189
dannyobrien wrote: post #182

Earlier quoted context omitted.

(I like the accidental pun of "Clout Provider" btw, which sadly conveys some of what they try to imply). We may not be disagreeing that much. My argument was, and is, it's not about where the data is , it's about who has control over it. The counter-argument was "well if it's in another country, then we don't have jurisdiction, so it's going to be much harder". But what you need jurisdiction over is the people . Othe…

That's fair, and it sounds like we aren't that far apart. It is, in fact, about control. So I'll restate my central theme as "until the idea of enforceable data sovereignty requirements were enshrined in law, the cloud providers did not and would not delegate control of any body of data to 'controllers' that weren't in jurisdictions where they could be influenced/coerced to compromise that data". Was this a slippery…

Yep, exactly. There's a peculiarly unsatisfying kind of vindication that comes from making "slippery slope" arguments, and then watch them play, and now you are now both a) technically correct, and b) fucked. You'll excuse me if I have a brief "I told you so" moment about a scenario about Amazon's UAE datacenters being bombed without bakcups because of a US-instigated Iranian conflict, where -- if I'd ever dared to describe it -- would definitely have got me laughed out of those rooms in 2010.

Re: AWS says it can't restore some data from mideast facilities struck by Iran

#196

Post by donavanm »

Twirrim wrote: post #173
dzonga wrote: post #169

Earlier quoted context omitted.

black swan events. this is one of one of those things - were in the current era either a cloud provider should provide automatic backups in another geographic zone. if you're in us-east, then your back-ups should ideally be in eu-west + africa for redundancy.

That's absolutely something you can configure your S3 bucket to do if you want (I have one of mine replicating elsewhere). The amount of data S3 stores "automatically replicating" to other geographical locations would make things prohibitively expensive, especially when you consider the daily delta, and how much of that is ephemeral or frequently mutated data that is stored. The bandwidth costs alone would be eye-wat…

> The amount of data S3 stores "automatically replicating" to other geographical locations would make things prohibitively expensive

It did work like this! And it was! My recollection is that the first S3 was out of SEA and had no user concept of region. Then “VDC” was added in virginia. That provided API endpoints in what became us-east-1. A bucket could be accessed from either location, and the original intent was for object store to replicate between them. By the time dub/eu-west-1 came along that was obviously not tenable; itd take 10s of gbs to replicate.

So S3 became regional. But the original sea/vdc deployments still had shared APIs and data in both regions. Your object would be stored in the region of the API you geolocated to via DNS, but read from either. _eventually_ all the data migrated to IAD, but those API endpoints were transparently proxying across the continent until 2013 or so.

And of course glacier had much more interesting takes on this with cross dc/az/region erasure encoding. But i dont think any of the wacky multi dimensional cross region stuff ever materialised in practice.

PS: hi!

Re: AWS says it can't restore some data from mideast facilities struck by Iran

#197

Post by jaggederest »

jacquesm wrote: post #130
houssc wrote: post #119

Earlier quoted context omitted.

The caveat is always "if you're using the service correctly " which is not necessarily free. Meaning taking advantage of multiple geo zones, building in redundancy to your stack, etc. Like everything he said is possible if your technology stack living in AWS was designed to survive it. Everyone who has ever had the "we lost your data" email from AWS knows at the end of the day the cloud is just someone else's data ce…

The problem is that lots of people seem to be under the impression that they are doing it right because they are using AWS. They don't realize that AWS is a toolbox, not a 'ready made solution for redundancy against all catastrophes you are possibly exposed to'. They use that to their advantage by pricing such solutions at a level that people will either pay through the nose or will be left without recourse when AWS…

If I were a bit more bloody-minded I would launch a service for vibe-coded apps that, under the hood, did everything "the right way" and just charged a flat fee + percent on the underlying.

I feel like if this was done correctly it would eat a bunch of the market, but I question how many people are actually willing to pay for "the right way". The last time I had that experience it was with Heroku which was quite a leaky abstraction.

Re: AWS says it can't restore some data from mideast facilities struck by Iran

#199

Post by jaggederest »

nobodyandproud wrote: post #129
echoangle wrote: post #111

Earlier quoted context omitted.

Isn’t the problem that multiple datacenters in one zone were blown up?

They had nine years and more money than god to build redundancy in an unstable region.

If they're region-locking data appropriately (i.e. for people to comply with domestic storage / gdpr-style requirements) they really can't. Bahrain is only about 300 square miles / 80k hectares.

They could, of course, open other regional data centers in other countries, or say "data in this geo zone may be in any of X, Y or Z" countries, but for the latter that pretty starkly limits some of the major customers they'd have, I would guess, and for the former, well, they have other geo zones already, so if people weren't replicating to them, I'm not sure why adding me-east-1 me-west-1 me-central-1 would fix that issue, they just wouldn't replicate there either.

Post reply on HN