Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: AWS appears to be down again

news.ycombinator.com

Post reply on HN
Print view View original

281–290 of 646 posts

Re: Tell HN: AWS appears to be down again

#281

Post by merlyn »

ItsBob wrote: post #249
notyourday wrote: post #241

Earlier quoted context omitted.

> I've only had a DC fail once when the engineer was performing work on the power circuitry for the DC and thought he was taking down one, but was in fact the wrong one and took both power circuits down at the same time. This is all local scale. Your setup would not survive a data center scale power outage. At scale power outages are datacenter scale. Data centers lose supply lines. They lose transformers. Sometimes…

Yes but if you have reliable power from two different sources then the biggest risk (I'd imagine) is the failover circuitry! Something that should be tested tbh. Also, there are banks of batteries and generators in between the power company cables and the kit: did they not kick-in? Again, this is all pure speculation: I have absolutely no idea of the exact failure, nor how their infrastructure is held together - this…

Frying hardware can affect much wider scope.

I've had bad power supplies fry out taking the whole power circuit with it, and thus half (or whatever fraction) of the rack's power. I've also had bad power supplies bring down the whole machine as they shunted everything internal too.

When things go bad, anything can happen. You can provide the best effort, and it'll usually work as expected, but there will always be something that can overcome your best efforts.

Re: Tell HN: AWS appears to be down again

#282

Post by jtc331 »

lmilcin wrote: post #275
quantumfissure wrote: post #261

Me: Hesitation at last job moving absolutely everything (including backups) to AWS because if it goes down it's a problem I'm a firm believer in some kind of physical/easily accessible backup. Coworkers: "You're an f'n idiot. Amazon and Facebook don't go down, you're holding us back!" Me: leaves cause that treatment was the final straw Amazon and Facebook both go down within a month of each other, and supposedly they…

Think about it this way: 1) Can you make your on prem infrastructure go down less than Amazon's? 2) Is it worth it? In my experience most people grossly underestimate how expensive it is to create reliable infrastructure and at the same time overestimate how important it is for their services to run uninterrupted. -- EDIT: I am not arguing you shouldn't build your more reliable infrastructure. AWS is just a point on…

You’re missing a huge factor: agency.

Re: Tell HN: AWS appears to be down again

#283

Post by junon »

bsenftner wrote: post #268
junon wrote: post #90

Earlier quoted context omitted.

No, fair use does NOT allow you to retain the original copyright. That would be passing off a derived work as the original copyright holder's work, which could be very damaging. This is a violation of fair use, if it could even be considered that to begin with.

Fair use in the case of satire is not retaining the original copyright, it is referencing the copyright. It is a legal split hair, but it stands in court.

I think you misunderstand. The website in question has "Copyright (c) Amazon, Inc" in the bottom, when in fact the derived work (the site) is not created by Amazon, but by a third party. IANAL but my understanding is that this copyright notice being retained without any clarification of the owner of the derived work can be seen as endorsement, which is an infringement of copyright unless Amazon has expressly permitted such use (which is usually indicated as such, anyway).

It is also clearly not satire. That would not hold up in court, and there are many instances where they have tried that angle and failed.

Re: Tell HN: AWS appears to be down again

#284

Post by autosharp »

lmilcin wrote: post #275
quantumfissure wrote: post #261

Me: Hesitation at last job moving absolutely everything (including backups) to AWS because if it goes down it's a problem I'm a firm believer in some kind of physical/easily accessible backup. Coworkers: "You're an f'n idiot. Amazon and Facebook don't go down, you're holding us back!" Me: leaves cause that treatment was the final straw Amazon and Facebook both go down within a month of each other, and supposedly they…

Think about it this way: 1) Can you make your on prem infrastructure go down less than Amazon's? 2) Is it worth it? In my experience most people grossly underestimate how expensive it is to create reliable infrastructure and at the same time overestimate how important it is for their services to run uninterrupted. -- EDIT: I am not arguing you shouldn't build your more reliable infrastructure. AWS is just a point on…

Also, you can just take two different amazon regions and hope they don't both go down at the same time.

For extra safety, and extra work, you could even take Azure as a backup if you're not locked in with AWS.

Re: Tell HN: AWS appears to be down again

#285

Post by Nextgrid »

lmilcin wrote: post #275
quantumfissure wrote: post #261

Me: Hesitation at last job moving absolutely everything (including backups) to AWS because if it goes down it's a problem I'm a firm believer in some kind of physical/easily accessible backup. Coworkers: "You're an f'n idiot. Amazon and Facebook don't go down, you're holding us back!" Me: leaves cause that treatment was the final straw Amazon and Facebook both go down within a month of each other, and supposedly they…

Think about it this way: 1) Can you make your on prem infrastructure go down less than Amazon's? 2) Is it worth it? In my experience most people grossly underestimate how expensive it is to create reliable infrastructure and at the same time overestimate how important it is for their services to run uninterrupted. -- EDIT: I am not arguing you shouldn't build your more reliable infrastructure. AWS is just a point on…

> Can you make your on prem infrastructure go down less than Amazon's?

Obviously depends on what you need, but for a small to medium web app that needs a load-balancer, a few app servers, a database and a cache, yes absolutely - all of these have been solved problems for over a decade and aren't rocket science to install & maintain.

> Is it worth it?

I'd argue that the "worth" would be less about immunity to occasional outages but the continuous savings when it comes to price per performance & not having to pay for bandwidth.

> overestimate how important it is for their services to run uninterrupted.

Agreed. However when running on-prem, should your service go down and you need it back up, you can do something about it. With the cloud, you have no choice but to wait.

Re: Tell HN: AWS appears to be down again

#286

Post by junon »

gilrain wrote: post #245
junon wrote: post #223

Earlier quoted context omitted.

I don't really see your point. My original comment was more pointing out that the operators, if not Amazon, could be seen as infringing their rights. They should update their legalese if they want to be truly protected. How is that defending Amazon? Regardless of your pointed comment, I'm operating in the land of legal objectivity. The law doesn't care about your feelings much.

Your argument is that IP law should be equally observed by everyone because it protects individuals and corporations alike: > Nope. Because these laws also protect people who make such websites from the corporations they're commenting on, too. My response is that your assumption is very obviously wrong: the law does not protect individuals and corporations alike. That’s all.

> Your argument is that IP law should be equally observed by everyone because it protects individuals and corporations alike

That is a weird understanding of what I said, and I don't really think you're arguing in good faith here. There's a lot of bias so I am choosing to not further this conversation.

Re: Tell HN: AWS appears to be down again

#287

Post by exogenousdata »

Looks like the SEC's Edgar website is affected. This is the site the SEC uses to post the filings of public companies. Normally there are a hundred or more company filings in the morning starting at 6am ET. This morning there are two.

https://www.sec.gov/cgi-bin/browse-edgar?action=getcurrent

Re: Tell HN: AWS appears to be down again

#288

Post by growse »

ItsBob wrote: post #224

I've built out many 42U racks in DC's in my time and there were a couple of rules that we never skipped: 1. Dual power in each server/device - One PSU was powered by one outlet, the other PSU by a different one with a different source meaning that we can lose a single power supply/circuit and nothing happens 2. Dual network (at minimum) - For the same reasons as above since the switches didn't always have dual power…

> 1. Dual power in each server/device - One PSU was powered by one outlet, the other PSU by a different one with a different source meaning that we can lose a single power supply/circuit and nothing happens

Nothing happens if you remember that your new capacity limit per DC supply is 50% of the actual limit, and you're 100% confident that either of your supplies can seamlessly handle their load suddenly increasing by 100%.

I've seen more than one failure in a DC where they wired it up as you described, had a whole power side fail, followed by the other side promptly also failing because it couldn't handle the sudden new load placed on it.

Re: Tell HN: AWS appears to be down again

#289

Post by jerf »

lmilcin wrote: post #275
quantumfissure wrote: post #261

Me: Hesitation at last job moving absolutely everything (including backups) to AWS because if it goes down it's a problem I'm a firm believer in some kind of physical/easily accessible backup. Coworkers: "You're an f'n idiot. Amazon and Facebook don't go down, you're holding us back!" Me: leaves cause that treatment was the final straw Amazon and Facebook both go down within a month of each other, and supposedly they…

Think about it this way: 1) Can you make your on prem infrastructure go down less than Amazon's? 2) Is it worth it? In my experience most people grossly underestimate how expensive it is to create reliable infrastructure and at the same time overestimate how important it is for their services to run uninterrupted. -- EDIT: I am not arguing you shouldn't build your more reliable infrastructure. AWS is just a point on…

I think if you put a bit of effort into classifying importance, you can likely justify backing up certain critical systems in more than one way. Let "the cloud" handle everyone's desktop backups and all the ancillary systems you don't really need immediately to do business, but certain important systems should perhaps be backed up both to the cloud and locally, like Windows Domain Controllers and other things you can't do anything without.

Backup is cheap when you're focused about what you're backing up.

In this case, the game isn't "going down less than Amazon", it's about going down uncorrelated to Amazon. Though that's getting harder!

"In more than one way" doesn't have to be local, but it may be across multiple cloud services. Still, "local" is nice in that it doesn't require the Internet. ("The Internet" doesn't tend to go down, but the portion you are on certainly can.) Of course, as workers disperse, "local" means less and less nowadays.

Re: Tell HN: AWS appears to be down again

#290

Post by rafale »

quantumfissure wrote: post #261

Me: Hesitation at last job moving absolutely everything (including backups) to AWS because if it goes down it's a problem I'm a firm believer in some kind of physical/easily accessible backup. Coworkers: "You're an f'n idiot. Amazon and Facebook don't go down, you're holding us back!" Me: leaves cause that treatment was the final straw Amazon and Facebook both go down within a month of each other, and supposedly they…

Did u file a complaint on the use of swear words?
Post reply on HN