Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is S3 down?

news.ycombinator.com

Post reply on HN
Print view View original

761–770 of 1001 posts

Re: Ask HN: Is S3 down?

#761

Post by vacri »

DenisM wrote: post #459

Now might be a good time to ponder a lasting solution. Clearly, we cannot trust AWS, or any other single provider, to stay up. What is the shortest, quickest to implement, path to actual high availability? You would have to host your own software which can also fail, but then at least you could do something about it. For example, you could avoid changing things during critical times of your own business (e.g. a trade…

Quickest path? Use AWS (or similar). Outages happen. The more robust you want to be against outage, the more prep work you have to do. In order to have 'no outage', you have to do an incredible amount of work. There is no quick path.

Re: Ask HN: Is S3 down?

#762

Post by ptrptr »

boulos wrote: post #735

Disclosure: I work on Google Cloud. Apologies if you find this to be in poor taste, but GCS directly supports the S3 XML API (including v4): https://cloud.google.com/storage/docs/interoperability and has easy to use multi-regional support at a fraction of the cost of what it would take on AWS. I directly point my NAS box at home to GCS instead of S3 (sadly having to modify the little PHP client code to point it to st…

Now that's what I call a shameless plug!

Re: Ask HN: Is S3 down?

#763

Post by kseifried »

dcosson wrote: post #721
devy wrote: post #706

So S3's been down for at least 3 hours. Does AWS break this year's S3 durability & reliability promise of eleven 9s by now? [1][2] [1]: https://aws.amazon.com/s3/details/ [2]: https://en.wikipedia.org/wiki/High_availability#Percentage_c...

That's durability (data loss) not availability. Here's [1] their official SLA. This outage so far brings them to less than 3 nines of uptime this month (43.8 minutes) but still more than 2 nines (7.2 hours) so it sounds like everyone gets 10% off their S3 bill. Very curious if Amazon will apply this automatically or only if you complain. Edit: from further down the same page, it looks like only if you write in to sup…

I have gotten several credits from AWS for under a dollar due to incorrect billing calculations on their end that they caught, notified me about and sent a credit for. So I will assume yes, they do this automatically (if they don't I'd be quite surprised).

Re: Ask HN: Is S3 down?

#764

Post by movedx »

gamache wrote: post #739
movedx wrote: post #734

Earlier quoted context omitted.

My advice is: don't keep your eggs in one basket. AZs a localised redundancy, but as Cloud is cheap and plentiful, you should be using two or more regions, at least, to house your solution (if it's important to you.) EDIT: less arrogant. I need a coffee.

But now you're talking about added effort. Multi-AZ on AWS is easy and fairly automatic, multi-region (and multi-provider) not so much. It's easy to say things like this, but people who can do ops are not cheap and plentiful.

The only difficult aspect of multi-region use is data replication, which I can confirm is a (somewhat) difficult problem. This issue was with S3 which has an option to automatically replicate data from the bucket's region to another one. It's a check box. A simple bit of logic in the application and you can move between regions with ease.

Even data replication has options for this, too.

And I work in Ops.

Re: Ask HN: Is S3 down?

#765

Post by ___start »

ta_wh wrote: post #715
benevol wrote: post #711

Here we go again: Technology leads to technology (and wealth) monopolies, in other words: more centralization. Which has always been bad. Just like with Cloudflare leaking highly sensitive data all over the Internet, a couple of days ago.

Wouldn't go that far. It's always been the case in the cloud that if you're not region/provider replicated, you're susceptible to localized outages.

but the sentiment is valid imo (albeit arguably benign)...

Re: Ask HN: Is S3 down?

#766

Post by amaks »

emrekzd wrote: post #312
alexleclair wrote: post #12

Yup, same here. It has been a few minutes already. Wanna bet the green checkmark[1] will stay green until the incident is resolved? [1] https://status.aws.amazon.com/

In December 2015 I received an e-mail with the following subject line from AWS, around 4 am in the morning: "Amazon EC2 Instance scheduled for retirement" When I checked the logs it was clear the hardware failed 30 mins before they scheduled it for retirement. EC2 and root device data was gone. The e-mail also said "you may have already lost data". So I know that Amazon schedules servers for retirement after they alr…

That what happens when cloud provider doesn't support live migration for VMs.

Re: Ask HN: Is S3 down?

#767

Post by lambdasquirrel »

AtheistOfFail wrote: post #694
cyberferret wrote: post #680

Well, at least our decision to split services has paid off. All of our web app infrastructure is on AWS, which is currently down, but our status page [0] is on Digital Ocean, so at least our customers can go see that we are down! A pyrrhic victory... ;) [0] - http://status.hrpartner.io EDIT UPDATE: Well, I spoke too soon - even our status page is down now, but not sure if that is linked to the AWS issues, or simply t…

Could be worse, your entire infrastructure could be hosted on Heroku. You don't use S3 but because they do, your entire infrastructure crumbles.

I don't see why this is being downvoted. It's a pretty legitimate concern.

Re: Ask HN: Is S3 down?

#768

Post by movedx »

jacquesm wrote: post #738
movedx wrote: post #734

Earlier quoted context omitted.

My advice is: don't keep your eggs in one basket. AZs a localised redundancy, but as Cloud is cheap and plentiful, you should be using two or more regions, at least, to house your solution (if it's important to you.) EDIT: less arrogant. I need a coffee.

Two different vendors if you can afford it. It's a bit of a hassle though.

I like to stick to one, but I have seen some success stories with an AWS/GCE mix :-)

HashiCorp's Terraform makes it a lot easier to go multi Cloud, and abstracting away configuration of the OS and applications/state with Ansible makes the whole process a lot easier too.

Re: Ask HN: Is S3 down?

#769

Post by koolba »

ocdtrekkie wrote: post #742

I like how you know this comment is in poor taste, and posted it anyways.

I'm happy he posted it. TIL that they're API compatible. That's a big deal for anybody thinking of insulating themselves from something like this in the future without maintaining two code bases.

Re: Ask HN: Is S3 down?

#770

Post by qaq »

boulos wrote: post #724
danohu wrote: post #160

Earlier quoted context omitted.

Backblaze's B2 is an S3 near-clone: https://www.backblaze.com/b2/cloud-storage.html I don't know about reliability, but it's a fraction of the price of S3.

As I've said previously: I love the Backblaze folks, but B2 is apparently hosted in a single building. So that's not a question of if, but when. Disclosure: I work on Google Cloud.

In theory you are totally right but looking at the AWS it seams that the likelihood of issues due to ops complexity could be higher than the risk of a simple but single site service going down.
Post reply on HN