Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is Amazon's cloud service too big to fail?

fnlondon.com

Post reply on HN
Print view View original

61–70 of 164 posts

Re: Is Amazon's cloud service too big to fail?

#61

Post by jrimbault »

Rjevski wrote: post #55
smegel wrote: post #10

Is it possible for AWS to have a multi-region outage - as in is there anything connecting them that could bring them all (or several) down at once? (Apart from the result of a botched patching or update to the core software stack that was done worldwide at the same time and hopefully never happens).

Well given that you can manage services across all regions from a single web interface, I assume someone compromising this web interface would be able to control and bring stuff down across all regions.

Do you think there's a global world admin web console ?

Re: Is Amazon's cloud service too big to fail?

#62

Post by 65827 »

tyfon wrote: post #18
dalbasal wrote: post #2

This is (I was surprised) a pretty good article. Financial services are regulated and based on recent experience, they're concerned with systemic risk. Most industries do not have anyone responsible for worrying about this kind of thing. It seems reasonable to start worrying about the fragility potentially introduced by these massive internet infrastructure companies.

I really hate the "too big to fail" meme and I strongly agree with Bernie in that if you are too big to fail you are too big to exist. That should be the priority.

Why is every arrangement of characters now a "meme". That word has moved beyond devoid of meaning, at this point it's like a black hole of nothingness of a word.

Re: Is Amazon's cloud service too big to fail?

#63

Post by TrickyRick »

tyfon wrote: post #18
dalbasal wrote: post #2

This is (I was surprised) a pretty good article. Financial services are regulated and based on recent experience, they're concerned with systemic risk. Most industries do not have anyone responsible for worrying about this kind of thing. It seems reasonable to start worrying about the fragility potentially introduced by these massive internet infrastructure companies.

I really hate the "too big to fail" meme and I strongly agree with Bernie in that if you are too big to fail you are too big to exist. That should be the priority.

Agreed, it seems like the software approach as well. You wouldn't want a class or a piece of code to be "too big" to fail, you'd refactor it into smaller pieces which can be overviewed more easily.

Re: Is Amazon's cloud service too big to fail?

#64

Post by TrickyRick »

65827 wrote: post #62
tyfon wrote: post #18

Earlier quoted context omitted.

I really hate the "too big to fail" meme and I strongly agree with Bernie in that if you are too big to fail you are too big to exist. That should be the priority.

Why is every arrangement of characters now a "meme". That word has moved beyond devoid of meaning, at this point it's like a black hole of nothingness of a word.

> A meme (/ˈmiːm/ MEEM) is an idea, behavior, or style that spreads from person to person within a culture.

Seems like it's fitting here, does it not? Certain banks being "too big to fail" is an idea passed among persons within our culture.

Re: Is Amazon's cloud service too big to fail?

#65

Post by dalbasal »

tyfon wrote: post #18
dalbasal wrote: post #2

This is (I was surprised) a pretty good article. Financial services are regulated and based on recent experience, they're concerned with systemic risk. Most industries do not have anyone responsible for worrying about this kind of thing. It seems reasonable to start worrying about the fragility potentially introduced by these massive internet infrastructure companies.

I really hate the "too big to fail" meme and I strongly agree with Bernie in that if you are too big to fail you are too big to exist. That should be the priority.

I don't know what Bernie's specific plans were/are but it's not unusual. At least, at first blush, this is a lot of people's reactions. Too-big-to-fail = danger = make it smaller.

Realisitically, this has not been the solution implmented (in the EU & US, at least). In the EU, it is even more crucial as the "solutions" to this problem are applied to state finances as well as financial institutions.

In terms of policies, there are two competing approaches: (1) Reduce the size of "too-big-to-fail" institutions. (2) Regulate them more heavily (or some other strategy) so that they will not fail. In the EU, this is being applied to states, not just financial institutions. Rules that (supposedly) reduce catastrophic risk.

Almost all seripous policy proposals are in the no. 2 category. Tighten regulation, reduce the risk of failure. Tighter regulation lends to stronger incumbents and larger average company size so by doing 2, you are probably doing the opposite of 1.

As I said, I don't know what Bernie's proposal is or how mature it is as a policy (as opposed to a politician statement). It would be notable if a left wing politician propsed loosening bank regulations, though definitely not impossible or unreasonable.

Re: Is Amazon's cloud service too big to fail?

#66

Post by TrickyRick »

peteretep wrote: post #38
gaius wrote: post #36

Earlier quoted context omitted.

legitimate national security concern, and the government steps in to provide protection There are a LOT of far softer targets that go unprotected. A terrorist attack on a sewage plant for a major city would be far more devastating than knocking out a few websites.

I feel like the point of the article we're commenting on is that AWS is far from "a few websites"

Granted it was only for a few hours but they had a huge outage in February this year (https://aws.amazon.com/message/41926/). This affected pretty much all us-east customers (Which are most AWS customers) so it should be possible to extrapolate from there. Although I think a lot of customers learned the importance of replication that day so the next outage might not be that bad.

Re: Is Amazon's cloud service too big to fail?

#67

Post by cm2187 »

mediascreen wrote: post #45
cm2187 wrote: post #30

Earlier quoted context omitted.

All you need to blow is a few cables.

Well, you need to disable redundant ISP or power cables to all datacenters in a region. and that would probably be pretty easy to recover from in a day or two. I imagine Amazon has some on site security measures as well. It would be interesting to see how important services would cope with their main region going down for more than a few days.

Not that I think it is wise to discuss optimising a terrorist activity on a public forum, though it is interesting from a threat analysis point of view.

You don't need to severe both power and data cables. For power the datacentre should be able to cope for a while, particularly if it has access to fuel deliveries. Data cable should be both easier to severe and more difficult to recover from (not the least to identify which cable has been cut where).

Most of the infrastructure of a country in term of cables run along railway tracks, sewers, etc. This means thousands of kilometers of cable even for a small country, and it is impossible to secure everything. It is impossible to severe everything either but you don't need to severe every single cable. As long as you severe enough of the backbone, the other cables will be overloaded.

So I don't think it would take that much effort for a network of terrorists to create havoc in the communications of a country for at least a few days to a week. And the consequences for the economy can be pretty dire. We have seen with BA what happens when their datacentre goes offline. Their all fleet is grounded. I imagine the consequences of a country-wide outage could be pretty dramatic. Unlikely anyone would die but you could really dent the GDP.

Re: Is Amazon's cloud service too big to fail?

#68

Post by Rjevski »

jrimbault wrote: post #61
Rjevski wrote: post #55

Earlier quoted context omitted.

Well given that you can manage services across all regions from a single web interface, I assume someone compromising this web interface would be able to control and bring stuff down across all regions.

Do you think there's a global world admin web console ?

Well as a customer how do you interact with it? Do you have one admin interface per region with its own URL or do you just go on an AWS homepage, login and control everything from there?

Re: Is Amazon's cloud service too big to fail?

#69

Post by arethuza »

michaelt wrote: post #57
arethuza wrote: post #33

Earlier quoted context omitted.

I'd be very surprised if big users of AWS or Azure pay the rates advertised on the public web sites.

Right, nobody pays /more/ than the published prices. And prospective users can look at the published prices, and see that historically they've gone down more than they've gone up (although obviously that trend could reverse). So people think they're safe from the risk of Amazon quadrupling their bill over night. Of course, vendor lock-in can have other negative effects, but apparently people aren't worried about them…

Well, if you aren't in the US there is forex risk - Azure certainly increased their prices in the aftermath of the Brexit vote decrease of GDP against the USD.

Re: Is Amazon's cloud service too big to fail?

#70

Post by dsandip »

unkown-unknowns wrote: post #15
PaulKeeble wrote: post #5

Earlier quoted context omitted.

It is one of the issues with choosing the cloud providers and taking their stack. They are hoping the cost of swapping once bought into their way is too costly to a competitor who can offer similar service cheaper. Lockin used to be considered bad but something changed with cloud providers and ops/developers don't seem to care as much anymore.

There are some open source implementations of parts of the APIs of cloud providers that might help someone a bit when trying to migrate. For example, Minio [1] [2] implements the AWS S3 v4 API. [1]: https://news.ycombinator.com/item?id=12392081 [2]: https://minio.io/

We[1] have seen a fair number of requests for managed services as devs claim that "we don't have the time or skills to maintain these open source components" (quoting verbatim from a request on Intercom). I don't think this is about having open source substitutes in all the cases. Personally not a fan of how/where the build vs. buy debate is playing out here.

[1]: https://hasura.io

Post reply on HN