Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

reddit.com

Post reply on HN
Print view View original

31–40 of 157 posts

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#31

Post by smhinsey »

asb wrote: post #28
jameskilton wrote: post #6

This comment further down, supposedly from an Amazon employee, paints a grim picture for EBS: http://www.reddit.com/r/blog/comments/g66f0/why_reddit_was_d...

Even if true, I don't think that comment is fair to the EBS team. It seems a likely reason for that behaviour is they don't have enough resource to work on and test (presumably) large changes to fix the underlying issue while also fixing the tickets which crop up. The tone of the comment seems to imply they're foolishly overlooking the obvious solution.

This is totally true, but at the same time, given the success and scale of AWS, it's insane that they would not have the resources they need.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#32

Post by gpapilion »

mithaler wrote: post #10

We were bitten by EBS' slowness at my company recently, when moving an existing project to AWS. You effectively can't get decent performance off of a single EBS volume with PostgreSQL; you need to set up 10 or so of them and make a software RAID to remove the bottleneck. It's a fairly large time commitment to build and maintain, but it's pretty fast and reliable once it's up and running (cases like the recent downtim…

MySQL does not fare any better on a single EBS volume. The issues with EBS are systematic. Similarly you have to raid several volumes together to see decent performance, and this is the recommended AWS solution.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#33

Post by api »

jedsmith wrote: post #30
ck2 wrote: post #24

I firmly believe "the cloud" is a fad, unless for some reason you own and operate all the hardware yourself (ie. Google). Like other technical fads, everyone will probably come back to servers they can reach out and touch when needed, sooner or later.

The cloud significantly lowers capital expenditure to get into an Internet-enabled business, which cultivates the very startup ecology that Y Combinator exists to leverage and support. Those teenagers who started the Facebook Pokemon game would have never had the resources to build a scalable solution with hardware that they own. (That is, unless Y Combinator paid a lot more money as part of participating. They might…

The cloud's real advantage is the ability to build out fast, but it is not cost. It is cheaper to build it yourself and run it yourself if you know exactly what you need, and have time to do so. If you don't, the cloud is cheaper.

So you're right that the cloud is great for startups. It is not so great for established stuff.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#34

Post by jedsmith »

api wrote: post #33
jedsmith wrote: post #30

Earlier quoted context omitted.

The cloud significantly lowers capital expenditure to get into an Internet-enabled business, which cultivates the very startup ecology that Y Combinator exists to leverage and support. Those teenagers who started the Facebook Pokemon game would have never had the resources to build a scalable solution with hardware that they own. (That is, unless Y Combinator paid a lot more money as part of participating. They might…

The cloud's real advantage is the ability to build out fast, but it is not cost. It is cheaper to build it yourself and run it yourself if you know exactly what you need, and have time to do so. If you don't, the cloud is cheaper. So you're right that the cloud is great for startups. It is not so great for established stuff.

Oh, it's certainly about cost. When you talk about paying for your own transit, your own power, cage space, and remote hands, cloud providers can be significantly cheaper than owning the hardware. You also lose the administrative overhead of having to perform drive swaps when your units degrade -- it's just computing capacity that exists with a minimum of hassle to you. I think if you add up all of the variables, cloud can (and does) come out more cost-effective.

I think private clouds are fantastic for established stuff, and many companies use public clouds to their benefit as well.

I added this in an edit after you replied, but cloud is a term that is difficult to nail to the wall: my explanation to people that I like to run with is that the cloud is a way to think about your architecture.

Rather than have a DNS box, two Web servers, a DB box, and so on, then another server for every development environment, virtualizing the hardware makes a lot of sense. You get a lot more traction out of each U, and with a large number of of-the-shelf utilities, you can automate the hell out of that. Need a clean test environment to try an installation of your software? There are ways to accomplish that in minutes, and dispose of it and reuse the space. That to me is a cloud. Virtualization and automation on top of it. That's what Linode has been doing for nearly eight years now, so it's arguable that Linode pioneered the cloud space. In 2003, it was just called VPS hosting.

Integrating a public cloud and a private cloud makes a lot of sense, and a lot of established big-iron is taking this approach. Big players are realizing that the cloud makes a lot of sense, which we see with HP's announcement that they intend to enter the cloud market.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#35

Post by bluegene »

rlpb wrote: post #26

RAIDing together multiple EBS volumes feels like a massive hack to me. I can't help but wonder if this compounds the problem at Amazon's end. If EBS performance is a problem, Amazon need to fix it. For example, if some way of tying together multiple EBS volumes is a reasonable way of working around the problem, then why aren't Amazon providing "high performance" EBS volumes which do that under the hood? If I were fac…

May be I'm missing something here; Why there's even a discussion about RAID at the EBS level? When Amamzon says, "Amazon EBS volumes are designed to be highly available and reliable" and if we have to talk about RAID then the issue is on Amazon's end

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#36

Post by ck2 »

jedsmith wrote: post #30
ck2 wrote: post #24

I firmly believe "the cloud" is a fad, unless for some reason you own and operate all the hardware yourself (ie. Google). Like other technical fads, everyone will probably come back to servers they can reach out and touch when needed, sooner or later.

The cloud significantly lowers capital expenditure to get into an Internet-enabled business, which cultivates the very startup ecology that Y Combinator exists to leverage and support. Those teenagers who started the Facebook Pokemon game would have never had the resources to build a scalable solution with hardware that they own. (That is, unless Y Combinator paid a lot more money as part of participating. They might…

If you start with the cloud, you best formulate an "exit plan".

Reddit for example doesn't seem to have one and seems quite stuck.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#38

Post by hemancuso »

I've never understood how people can use EBS in production. The durability numbers they quote are bad and they wave their hands around about increased durability with snapshots, but never quantify what that means.

Hard drives are unreliable and they certainly don't fail independently of one another - but the independence of their failure is much more independent than EBS.

With physical dives and n-parity RAID you drastically reduce the rate of data loss. This is because although failures are often correlated, it's quite unlikely to have permenant failure of 3 drives out of a pool of 7 within 24 hours. It happens, but it is very rare.

With EBS, your 7 volumes might very well be on the same underlying RAID array. So you have no greater durability by building software RAID on top of that. If anything, it potentially decreases durability.

You could utilize snapshots to S3, but is that really a good solution? It seems that deploying onto EBS at any meaningful scale is a recipe for garunteed data-loss. Raid on physical disks isn't a great solution either, and there is no substitute for backups - but at least you can build a 9 disk RaidZ3 array that will experience pool failure so rarely that you can more safely worry about things like memory and data bus corruption.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#39

Post by Zak »

I recently had an EBS volume lose data for no apparent reason. I'm not a heavy EC2 user at all - I was just doing some memory/cpu-heavy stuff that wouldn't fit in to RAM on my laptop and using EBS as a temporary store so I could transfer data using a cheap micro instance and only spin up the big expensive instances when everything was in place. I ended up downloading files on an m2.4xlarge because the files I had just downloaded to the EBS volume vanished.

Re: "Amazon's EBSs are a barrel of laughs in terms of performance and reliability"

#40

Post by woodrow »

api wrote: post #33
jedsmith wrote: post #30

Earlier quoted context omitted.

The cloud significantly lowers capital expenditure to get into an Internet-enabled business, which cultivates the very startup ecology that Y Combinator exists to leverage and support. Those teenagers who started the Facebook Pokemon game would have never had the resources to build a scalable solution with hardware that they own. (That is, unless Y Combinator paid a lot more money as part of participating. They might…

The cloud's real advantage is the ability to build out fast, but it is not cost. It is cheaper to build it yourself and run it yourself if you know exactly what you need, and have time to do so. If you don't, the cloud is cheaper. So you're right that the cloud is great for startups. It is not so great for established stuff.

As the parent post mentions, it's also about moving your costs from a mix of capital and operating to 100% operating. This is one of the arguments that has motivated Netflix' move to AWS (and it makes sense for other SaaS): their costs scale more or less directly with their customer base, and thus revenue, with no up-front capital required.

See http://qconsf.com/dl/qcon-sanfran-2010/slides/AdrianCockcrof... for more.

Post reply on HN