Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

reddit.com

Post reply on HN
Print view View original

61–70 of 157 posts

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#61

Post by jedsmith »

romaniv wrote: post #60
jedsmith wrote: post #30

Earlier quoted context omitted.

The cloud significantly lowers capital expenditure to get into an Internet-enabled business, which cultivates the very startup ecology that Y Combinator exists to leverage and support. Those teenagers who started the Facebook Pokemon game would have never had the resources to build a scalable solution with hardware that they own. (That is, unless Y Combinator paid a lot more money as part of participating. They might…

You can do a lot of the things you describe here with VPS (Virtual Private Servers). You get root access, you don't manage hardware, you often receive some virtualization benefits (images, snapshots). Does that count as "Cloud Computing"?

Definitely. Another term for mostly the same thing, in my book. I talked about this here: http://news.ycombinator.com/item?id=2340734

VPS providers give you the tools, cloud providers give you the tools and a few finished products with less flexibility. You can use VPS as part of a cloud implementation, just like you can use dedicated servers as part of a cloud implementation, too.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#63

Post by ceejayoz »

watchandwait wrote: post #59
snorkel wrote: post #43

Having been at a startup that used hundreds of EC2 instances and EBS volumes I can assure you all that Amazon EBS performance is downright terrible and Amazon didn't inspire any confidence that they could solve it. Even worse than the EBS performance is Amazon does not offer any shared storage solutions between EC2 instances. You have to cobble together your own shared storage using NFS and EBS volumes making it suck…

My experience with the AWS RDS database product has been excellent.

The biggest issue I have with RDS is that I can't do a multi-master deployment to scale up writes. I've got a very write-heavy workload in my systems (roughly one write for every two reads).

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#64

Post by joevandyk »

grourk wrote: post #53
joevandyk wrote: post #46

Earlier quoted context omitted.

Did you use Raid10? I would love to see a post on using postgresql with ec2/ebs -- how to setup raid, etc.

Orion Henry at Heroku wrote about this and described different software RAID configurations and the performance characteristics of each a while back: http://orion.heroku.com/past/2009/7/29/io_performance_on_ebs...

Yes, but as a lowly developer, I have no idea how to set read-ahead buffers or change io schedulers.

Plus, that's a year old, would love to see some updated advice. You'd think Amazon would write more guides like this.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#65

Post by jerf »

prakash wrote: post #22

We (Cedexis) presented our findings on - How do EC2's East, West, EU & APAC zones compare: (pdf) http://www.cloudconnectevent.com/2011/presentations/free/76-... If you would like to know more please send me an email: prakash [at] cedexis.com

You should post that to HN, if you haven't already. Possibly wrap a blog post around it.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#66

Post by andrewvc »

saurik wrote: post #52
andrewvc wrote: post #27

Earlier quoted context omitted.

One other huge downside of raiding EBS volumes is you can't use EBS's snapshotting features as you cannot guarantee a perfect sync (you could use LVM yourself however). Honestly, since EBS vols are supposedly not tied to a single disk, the raiding should be done on Amazon's end. That it isn't is telling.

You have to snapshot at the system level anyway if you want a consistent snapshot: otherwise the filesystem (or your database) could have been reordering and delaying writes that end up not being part of the "consistent snapshot". This is simply not a RAID-specific issue, nor is it a problem with EBS (as it is generally easy to use LVM, xfs, and/or PostgreSQL to handle that part of the job).

True. However, for some cases where you don't mind losing some data due to a recovery process EBS snapshots are 'good enough'. Additionally, with a database like CouchDB with a 'crash only' design, it should work for some cases as well.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#67

Post by tomkarlo »

ck2 wrote: post #24

I firmly believe "the cloud" is a fad, unless for some reason you own and operate all the hardware yourself (ie. Google). Like other technical fads, everyone will probably come back to servers they can reach out and touch when needed, sooner or later.

If you've ever had to deal with the expense and overhead associated with running a business that has extensive production systems, you wouldn't say that. The cloud represents a huge decrease in the initial cost necessary to set up production systems, and it relieves businesses of all kinds of issues regarding long-term leases on equipment or depreciation / amortization of equipment. You don't have to worry about swapping out racks just because they've reached an arbitrary end-of-lease date. You don't have to worry about provisioning hardware months in advance to make sure it's available "if" you need it.

There are definitely hiccups, but I can't imagine many guys running an internet-heavy business going forward are seriously going to say "let's build out our own datacenter rather than solve the issues with the cloud" unless they're doing something really, really, specialized.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#68

Post by WALoeIII »

andrewvc wrote: post #27
rlpb wrote: post #26

RAIDing together multiple EBS volumes feels like a massive hack to me. I can't help but wonder if this compounds the problem at Amazon's end. If EBS performance is a problem, Amazon need to fix it. For example, if some way of tying together multiple EBS volumes is a reasonable way of working around the problem, then why aren't Amazon providing "high performance" EBS volumes which do that under the hood? If I were fac…

One other huge downside of raiding EBS volumes is you can't use EBS's snapshotting features as you cannot guarantee a perfect sync (you could use LVM yourself however). Honestly, since EBS vols are supposedly not tied to a single disk, the raiding should be done on Amazon's end. That it isn't is telling.

xfs_freeze

In fact there is a handy package called ec2-consistent-snapshot (https://launchpad.net/ec2-consistent-snapshot) that will manage this for you!

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#69

Post by WALoeIII »

ceejayoz wrote: post #63
watchandwait wrote: post #59

Earlier quoted context omitted.

My experience with the AWS RDS database product has been excellent.

The biggest issue I have with RDS is that I can't do a multi-master deployment to scale up writes. I've got a very write-heavy workload in my systems (roughly one write for every two reads).

You can't do multi-master with MySQL anyways, which until very recently has been the only "engine" RDS supports. Even if you could do multi-master, replication is still single threaded. You have to come up with your own sharding scheme. This is a limitation of MySQL not RDS.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#70

Post by khafra »

parasubvert wrote: post #19

Generally speaking this is the sort of thing that people warn about when they say "if you want to run on a cloud, you need to design your application for a cloud". Meaning, you can't presume your infrastructure is dedicated and carries similar MTBFs of (say) an enterprise hard drive, which upwards of 1 million hours. Amazon provides plenty of opportunities to mitigate for this, such as providing multiple availability…

> modulo hiccups that can bring it down by a factor of 5.

The comments on reddit indicated hiccups more on a factor of 10x and, sometimes, 100x.

Either way, the issue is that the more drives you add to your RAID0, the more often one of those drives experiences a "hiccup," and kills the performance of the entire volume.

Post reply on HN