"Amazon's EBSs are a barrel of laughs in terms of performance and reliability" • Hacker News Board

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#101

Post by bmurphy » Fri, Mar 18, 2011, 7:49 PM UTC

riledhel wrote: post #97

cachemoney wrote: post #84

Earlier quoted context omitted.

EBS-RAID0 is much faster for reads than local. Local is faster for writes.

this seems to contradict several comments here. "citation needed".

My experience has been that EBS handles concurrent loads better, ephemeral drives handle non-concurrent loads better.

Transferring 100gb+ of data on EBS (even with an 8x RAID) is a nightmare. Ephemeral drives, however, it's fairly fast.

Throw 100+ database connections at a few ephemeral drives (even in a RAID) and watch your web site slow to a crawl.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#102

Post by bmurphy » Fri, Mar 18, 2011, 7:51 PM UTC

danielrhodes wrote: post #92

What's the failure rate of EBS versus having direct access to physical disks? My guess is that at scale, it's probably similar. Although you would hope that the storage components of AWS's cloud were highly reliable, I think the main benefit is not single instance reliability but being able to recover faster because of quickly available hardware.

I don't have solid numbers, just some experience using this. Ephemeral drives outright fail more often than EBS volumes, however, EBS volumes suffer performance degradation significantly more often than ephemeral drives. EBS volume performance is HIGHLY variable, at all times of day, no matter what load you throw at it. Ephemeral drives are very consistent most of the time.

Both types of drives CAN and DO fail, so RAID-10, fail over, and replication are a must have.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#103

Post by cachemoney » Fri, Mar 18, 2011, 7:51 PM UTC

riledhel wrote: post #97

cachemoney wrote: post #84

Earlier quoted context omitted.

EBS-RAID0 is much faster for reads than local. Local is faster for writes.

this seems to contradict several comments here. "citation needed".

I run a database cluster with dozens of nodes on EC2. Small entries, lots of small IOPS.

From http://orion.heroku.com/past/2009/7/29/io_performance_on_ebs..., "On a good day, an EBS can give you 7,000 seeks per second and on a not so good day will give you only 200."

The ephemeral store will never give you more than a hundred seeks per second. If you're seek-bound, then EBS, every time.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#104

Post by krakensden » Fri, Mar 18, 2011, 7:59 PM UTC

gruseom wrote: post #79

krakensden wrote: post #76

Earlier quoted context omitted.

All of your EC2 instances can disappear without warning and everything on the local storage is now gone forever.

That's the "backup to S3" part.

That's a fair point, but I don't think it holds up real well. What are the semantics? Do you block until everything is fully backed up on S3? Are you continuously taking database snapshots and forwarding them to S3? What happens if the backups start to fall further and further behind production?

What do you tell the hordes of angry redditors when the last thirty minutes of carefully (or angrily) composed comments vanish?

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#105

Post by metageek » Fri, Mar 18, 2011, 8:12 PM UTC

icey wrote: post #83

tzs wrote: post #73

We've been looking at moving some or all of our stuff to either Amazon EC2/EBS/S3 or Rackspace cloud hosting, and it has been interesting. Amazon seems more flexible, since you buy block storage (EBS) independent of instances. If you have an application that needs a massive amount of data, but only a little RAM and CPU, you can do it. Rackspace, on the other hand, ties storage to instances. If you only need the RAM a…

I use the Rackspace cloud for a few Windows servers. The experience has been mostly positive, but they disappear for a few minutes each week, which is kind of troubling.

Disappear as in they crash and reboot, or disappear as in they're unpingable?

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#106

Post by yuhong » Fri, Mar 18, 2011, 8:17 PM UTC

On the comment itself, I have this: http://news.ycombinator.com/item?id=2339715

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#107

Post by rbranson » Fri, Mar 18, 2011, 8:27 PM UTC

danielrhodes wrote: post #88

absconditus wrote: post #41

How is it that Amazon.com is so reliable if there are so many problems with their "cloud" products? Do they not use the same software to run their site?

Amazon.com is not hosted on EC2. It's entirely separate.

This isn't entirely true. Amazon.com uses EC2 in addition to dedicated servers.

http://searchcloudcomputing.techtarget.com/news/1516269/Amaz...

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#108

Post by bretpiatt » Fri, Mar 18, 2011, 8:41 PM UTC

delano wrote: post #90

tzs wrote: post #73

We've been looking at moving some or all of our stuff to either Amazon EC2/EBS/S3 or Rackspace cloud hosting, and it has been interesting. Amazon seems more flexible, since you buy block storage (EBS) independent of instances. If you have an application that needs a massive amount of data, but only a little RAM and CPU, you can do it. Rackspace, on the other hand, ties storage to instances. If you only need the RAM a…

If you're interested to see how sites perform on EC2 and Rackspace over time: https://www.blamestella.com/vendor/ec2 https://www.blamestella.com/vendor/rackspace

You're monitoring from AWS US-East it looks like, you'll want to mention that to give people some context around the latency numbers.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#109

Post by obfuscate » Fri, Mar 18, 2011, 8:59 PM UTC

For a data set in the mere tens to hundreds of GB (in MongoDB, if anyone's curious), is there any reason I shouldn't conclude from this that I should use instance storage only (with multi-AZ replication and backups to S3, both of which I would be doing in any case)? Moderately slower recovery in the rare event of an instance failure seems better than the constant possibility of incurable killing performance degradation.

(Edit: I hadn't considered the possibility of somehow killing all my instances through human error. Ouch. That probably warrants one slave on EBS per AZ.)

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#110

Post by PaulHoule » Fri, Mar 18, 2011, 9:03 PM UTC

smhinsey wrote: post #31

asb wrote: post #28

Earlier quoted context omitted.

Even if true, I don't think that comment is fair to the EBS team. It seems a likely reason for that behaviour is they don't have enough resource to work on and test (presumably) large changes to fix the underlying issue while also fixing the tickets which crop up. The tone of the comment seems to imply they're foolishly overlooking the obvious solution.

This is totally true, but at the same time, given the success and scale of AWS, it's insane that they would not have the resources they need.

Word in the industry is that AWS is insanely profitable, so they've got not problem finding the money to hire the help.

My gripe with EBS is that hiccups in EBS cause my Linux instances to "lock up", consume 100% CPU and become unresponsive. AMZN is providing their own Linux distribution and drivers for the EBS devices so they can also attack this problem by patching the Linux kernel.