Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Operational issue – Multiple services (UAE)

health.aws.amazon.com

Post reply on HN
Print view View original

81–90 of 125 posts

Re: Operational issue – Multiple services (UAE)

#81

Post by protocolture »

hi_hi wrote: post #79
protocolture wrote: post #68

Earlier quoted context omitted.

Equinix in Sydney plonked 2 datacenters right on top of each other, and still insists that they are useful as redundant sites. There was a locally very funny situation for a while when a tech influencer was insisting both equinix sites could be shut down by a single building collapse. He was wrong, but he wasn't so wrong that people shouldn't make better infrastructure decisions.

I'm intrigued...how was he wrong?

The building in question wasnt really tall enough. And would have to be precision demolished to collapse in the way he was afraid of.

It would still cause chaos and possible power issues.

Needs to be taken in context with some Sydney buildings having maintenance defects a few years after they open. Largely due to inferior materials imported from china. The building in question developed some cracks in supporting beams and was briefly evacuated. There was never a chance it was going to topple on its own in a way that impacted more than 1/2 datacenters, so he pivoted to possible terrorism, but even thats largely nonsensical.

I just went hunting for the case and couldnt find it. The gentleman in question had published the claim to his business that was it happens trying to build contacts with defense and intelligence agencies for third party threat assessment. As far as I can tell the business no longer exists and he has deleted their footprint.

But he also posted the claim on public mailing lists so I can probably trawl it up if necessary.

Re: Operational issue – Multiple services (UAE)

#82

Post by itemize123 »

DANmode wrote: post #78
seydor wrote: post #70

Iran may be heavily overpowered but they play their cards well. They will lose the war but the economic damage to the gulf is irreparable

> they play their cards well. They will lose the war so…

so they self-destruct to force better terms of negotiation.

Re: Operational issue – Multiple services (UAE)

#83

Post by DANmode »

itemize123 wrote: post #82
DANmode wrote: post #78

Earlier quoted context omitted.

> they play their cards well. They will lose the war so…

so they self-destruct to force better terms of negotiation.

I’ve done some light game theory reading - but this isn’t covered - if they’re gone, who are they getting better terms for?

Re: Operational issue – Multiple services (UAE)

#84

Post by sebastiennight »

dehrmann wrote: post #76
bgentry wrote: post #25

The important quote from the timeline: Mar 01 9:41 AM PST We want to provide some additional information on the power issue in a single Availability Zone in the ME-CENTRAL-1 Region. At around 4:30 AM PST, one of our Availability Zones (mec1-az2) was impacted by objects that struck the data center, creating sparks and fire. The fire department shut off power to the facility and generators as they worked to put out the…

> impacted by objects Well that's a way of phrasing it. I assume these were missiles.

Most likely, those were debris from the interception of missiles flying overhead and being destroyed on their way to a military target.

AFAIK, there have been no confirmed signs of civilian sites being targeted directly, and it would also be unlikely that actual missiles would cause so little damage that you could patch your datacenter up and get it ready to go within hours.

Re: Operational issue – Multiple services (UAE)

#85

Post by [deleted] »

bgentry wrote: post #25

The important quote from the timeline: Mar 01 9:41 AM PST We want to provide some additional information on the power issue in a single Availability Zone in the ME-CENTRAL-1 Region. At around 4:30 AM PST, one of our Availability Zones (mec1-az2) was impacted by objects that struck the data center, creating sparks and fire. The fire department shut off power to the facility and generators as they worked to put out the…

[deleted]

Re: Operational issue – Multiple services (UAE)

#86

Post by pabs3 »

alexfoo wrote: post #24
Cyph0n wrote: post #14

Earlier quoted context omitted.

No, they are more aware of the customer demand for compute in the region.

And demand for data sovereignty.

Public cloud or cloud run by external organisations can never provide data sovereignty, by definition.

Re: Operational issue – Multiple services (UAE)

#87

Post by Imustaskforhelp »

pwarner wrote: post #58
Imustaskforhelp wrote: post #34

Earlier quoted context omitted.

> we will ensure we restore power and connectivity safely this would require human intervention and I am a bit worried what if the strike can happen again and human lives might be lost. IIRC there have been cases in history where sometimes a same location is targeted across multiple days. Obviously, AWS might have local employees working in the region but would there be an evaluation of this threat itself within the…

But I mean,are the employees safe at home? I guess if the really targeted the data center then home is safer, but in the fog of war maybe the data center wasn't the target?

> But I mean,are the employees safe at home? I guess if the really targeted the data center then home is safer, but in the fog of war maybe the data center wasn't the target?

My gut feeling says that they would be safer at homes than at datacenters. The only large info I have heard is attack on hotels, this datacenter etc. (atleast till right now).

> but in the fog of war maybe the data center wasn't the target?

We can't say this for sure but even if that was the case, I do think that they would see some damage was caused and then, try to double tap it for even more damage. So chances are, even if it wasn't the target previously, it might be the target now?

Re: Operational issue – Multiple services (UAE)

#88

Post by bigfatkitten »

protocolture wrote: post #68
jiggawatts wrote: post #48

Earlier quoted context omitted.

This reminds me of a visit to an Equinix data centre where the sales person was droning on and on about how incredibly reliable their power supplies were, how uninterruptible everything was, etc, etc… Essentially, he was trying to assure us that no-no-no, we don’t need multiple zones like the public clouds, they can instead guarantee 100% uninterrupted power under all circumstances. A bit bored and annoyed, I pointed…

Equinix in Sydney plonked 2 datacenters right on top of each other, and still insists that they are useful as redundant sites. There was a locally very funny situation for a while when a tech influencer was insisting both equinix sites could be shut down by a single building collapse. He was wrong, but he wasn't so wrong that people shouldn't make better infrastructure decisions.

The two in Alexandria are of course just down the road from the airport. A widebody jet cratering in that location will put them both out of business.

Re: Operational issue – Multiple services (UAE)

#89

Post by Imustaskforhelp »

ok_dad wrote: post #51
flymasterv wrote: post #50

Earlier quoted context omitted.

Man, don’t be a “hero” trying to restore a lower ping to someone trying to buy a kindle in Jeddah.

What about local hospitals which may have service from that data center? There are heroes needed everywhere, all the time.

> What about local hospitals which may have service from that data center? There are heroes needed everywhere, all the time.

Off-site backups/Multi Cloud Strategy while encrypting data (and keeping the key safe, key point) might be a better strategy for such mission critical infrastructure.

Re: Operational issue – Multiple services (UAE)

#90

Post by Imustaskforhelp »

stonogo wrote: post #57
ok_dad wrote: post #51

Earlier quoted context omitted.

What about local hospitals which may have service from that data center? There are heroes needed everywhere, all the time.

In that case, the hero was the person who avoided relying on a single AZ when they deployed to cloud.

100% absolutely but its a bit worrying if in the future multiple AZ/datacenters could be start to get targeted?

attacking datacenters within a particular region so that service would have a hard time.

I guess someone can use some other regions DC to have more than (regional?) AZ but for mission critical infra, I can see that having sometimes issue too and you genuinely can't predict any of all of this.

That being said, There should be more than one AZ reliance but IMO also off-site or multi-cloud backups should also be preferred/used as well.

Post reply on HN