Ask HN: Is S3 down? • Hacker News Board

Re: Ask HN: Is S3 down?

#701

Post by vacri » Tue, Feb 28, 2017, 9:11 PM UTC

problems wrote: post #623

emrekzd wrote: post #312

Earlier quoted context omitted.

In December 2015 I received an e-mail with the following subject line from AWS, around 4 am in the morning: "Amazon EC2 Instance scheduled for retirement" When I checked the logs it was clear the hardware failed 30 mins before they scheduled it for retirement. EC2 and root device data was gone. The e-mail also said "you may have already lost data". So I know that Amazon schedules servers for retirement after they alr…

That's completely ridiculous, get some fucking RAID Amazon. I order drives off newegg directly to my DC and I'm yet to lose data with the cheapest drives available in RAID10.

Yes, the only way a server can die is from non-raided disks.

Re: Ask HN: Is S3 down?

#702

Post by problems » Tue, Feb 28, 2017, 9:13 PM UTC

vacri wrote: post #701

problems wrote: post #623

Earlier quoted context omitted.

That's completely ridiculous, get some fucking RAID Amazon. I order drives off newegg directly to my DC and I'm yet to lose data with the cheapest drives available in RAID10.

Yes, the only way a server can die is from non-raided disks.

Otherwise they should at least be providing customers their data back.

Re: Ask HN: Is S3 down?

#703

Post by sk2code » Tue, Feb 28, 2017, 9:15 PM UTC

Now what kind of business choose to remain down for 2 hours plus during the peak business hours?

Seems cloud computing still has a lot to learn.

Re: Ask HN: Is S3 down?

#704

Post by nolite » Tue, Feb 28, 2017, 9:16 PM UTC

thraway2016 wrote: post #492

zedpm wrote: post #476

Can anyone comment on mitigating issues like this with S3 Cross-region replication? I'm reading up on it now while one of my services is dead in the water.

The only appropriate comment is that this issue is affecting all of our buckets, both in us-west and us-east. Replicating to another region would yield no useful benefits in this specific failure scenario.

Can't agree with this. Buckets in eu-west-1 are fine

Re: Ask HN: Is S3 down?

#705

Post by shiven » Tue, Feb 28, 2017, 9:16 PM UTC

Status page is lit up like a Christmas tree! Looks like AWS finally found the myriad non-green icons.

Re: Ask HN: Is S3 down?

#706

Post by devy » Tue, Feb 28, 2017, 9:16 PM UTC

So S3's been down for at least 3 hours. Does AWS break this year's S3 durability & reliability promise of eleven 9s by now? [1][2]

[1]: https://aws.amazon.com/s3/details/

[2]: https://en.wikipedia.org/wiki/High_availability#Percentage_c...

Re: Ask HN: Is S3 down?

#707

Post by melor » Tue, Feb 28, 2017, 9:16 PM UTC

Only limited impact to Aiven services due to service migration capability http://help.aiven.io/announcements/aiven-customer-notice-aws...

Re: Ask HN: Is S3 down?

#708

Post by cagataygurturk » Tue, Feb 28, 2017, 9:17 PM UTC

mwfj wrote: post #196

tlogan wrote: post #164

Earlier quoted context omitted.

These service health boards are more like advertisement page then actual status of the service.

I guess their bizarre thinking is something along the lines of: "unless we have proof that noone can access the service, we won't change the indicator from green to yellow. Seriously: I don't understand why you guys stay with AWS.

Because you perceive public clouds only as virtual machine providers, that you can replace with other provider in two days. A detailed cloud migration consists of replacing some parts of your software to use managed services provided by a specific cloud provider, and AWS is still has the best service offerings IMHO. When you use these services carefully also you will see that AWS is very cheap and reliable enough. Outages like today's are happening in every platform and it is possible to mitigate them.

You can use Adwords as a self-service user. Without knowing so much of details you can run your ads but also you can bery easily ruin your budget. But many enterprise customers use it very differently than those users and they are extremely optimizing the cost. Cloud is the same. If you don't know how big customers use AWS, it is normal that you are surprised because AWS is still leading the market.

You say GCP is better than AWS. Which part is better? GCP does not have many services of AWS we benefit from. How can you compare totally different providers? You can only say AWS EC2 is worse than GCP. But you cannot compare whole platforms in one sentence.

Re: Ask HN: Is S3 down?

#709

Post by noir_lord » Tue, Feb 28, 2017, 9:18 PM UTC

devy wrote: post #706

So S3's been down for at least 3 hours. Does AWS break this year's S3 durability & reliability promise of eleven 9s by now? [1][2] [1]: https://aws.amazon.com/s3/details/ [2]: https://en.wikipedia.org/wiki/High_availability#Percentage_c...

11 9's is durability not availability.

Re: Ask HN: Is S3 down?

#710

Post by vacri » Tue, Feb 28, 2017, 9:18 PM UTC

So this is particularly weird - one of my instances was showing 0% CPU in CloudWatch (dropped from 60% at the start of the event), but the logs were saying 'load 500'. I ssh'd in... and the problem resolved itself. The only thing I did was run htop to look at the load, and it dropped from 500 (reported in htop) to it's normal level. Just ssh'ing in fixed that issue.