Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

rapgenius.com

Post reply on HN
Print view View original

241–250 of 437 posts

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#241

Post by codewright »

jholman wrote: post #106
46Bit wrote: post #85

Earlier quoted context omitted.

Several Rails apps I develop have been suffering from similar issues. Perhaps 2-3% of requests take 0.4-2s in just processing. If the allocation is a little intelligent, it'll not perform too badly and is less work than much harder optimization. Yet if it's random, it'll queue up horribly . I'm pissed. Spent way too much time unable to explain it to coworkers, thinking I just didn't understand Heroku's platform and t…

> Easiest fix: moving to EC2 next week. I've wanted to ever since these issues became evident but it's hard to make a good argument from handwaving about 'problems'. Of course, then you need to solve all these problems yourself. That sounds pretty easy, you'll have it done next week no problem! That was sarcastic, but this isn't: good luck, let us know how it goes.

I've done dev-ops in the ads industry before, it's really not that hard if you're a competent programmer. You just have to take a more studious approach than most non-dev-ops programmers and read up on things before deploying them.

But if you want to let the various PAAS providers put the fear into you, that's your cowardice.

Let the others learn as they may.

Edit:

To clarify, Heroku is making the problem harder on themselves than it would be for an individual to serve their own needs because of the complexity managing so many customers and apps.

You don't have to be Heroku to do for yourself what they offer.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#242

Post by Dylan16807 »

chc wrote: post #115
FireBeyond wrote: post #5

This should be more prominent. I want to love Heroku, and am sure that I could. But really, throwing in the towel at intelligent routing and replacing it with "random routing" is horrific, if true. It's arguable that the routing mesh and scaling dynamics of Heroku are a large part, if not -the- defining reason for someone to choose Heroku over AWS directly. Is it a "hard" problem? I'm absolutely sure it is. That's on…

> But really, throwing in the towel at intelligent routing and replacing it with "random routing" is horrific, if true. The thing is, their old "intelligent routing" was really just "we will only route one request at a time to a dyno." In other words, what changed is that they now allow dynos to serve multiple requests at a time. When you put it that way, it doesn't sound as horrific, does it?

Yes it does sound as horrific. It should never route more than N requests to a dyno, where N is the number of requests it can handle simultaneously. It doesn't matter if N is 1, 2, or 10. And it should always send new requests to the emptiest dyno.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#243

Post by tibbon »

scottshea wrote: post #237
nthj wrote: post #71

I'm inclined to wait until Heroku weighs in to render judgement. Specifically, because their argument depends on this premise: > But elsewhere in their current docs, they make the same old statement loud and clear: > The heroku.com stack only supports single threaded requests. Even if your applicaExplaintion were to fork and support handling multiple requests at once, the routing mesh will never serve more than a sin…

To amplify, I do a ton of queue adjustment work with Unicorn at the backlog level. It is so frequent that we set up Unicorn startup to read from an ENV variable on Heroku that we set as needed. With two Unicorn workers we found that 25 was the best backlog threshold to accept (it refuses additional requests). When we were able to go to 5 Unicorn workers on Heroku we had to start to adjust that.

You don't happen to have any documentation for how to do that do you? Very curious. Never seen anything about setting up Unicorn like this prior (I'm just using 3 or 4 Unicorns/dyno currently)

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#244

Post by joseph_cooney »

teraflop wrote: post #197
damian2000 wrote: post #171

Earlier quoted context omitted.

I believe the article is saying that this wait will only occur with Rails due to it being single threaded? Reactor pattern frameworks like node don't suffer the same issue.

They will if the request is CPU-bound. In that case, throwing more concurrent requests at a server than it has cores just slows all of them down.

How many web requests are typically CPU bound though? Not many.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#245

Post by Dylan16807 »

ajsharp wrote: post #235

My hunch is that Heroku isn't doing this to bleed customers dry. I know more than a few really, really great people who work there, and I don't think they'd stand for that type of corporate bullshittery. If this were the case, I think we'd have heard about it by now. My best guess is that they hit a scaling problem with doing smart load balancing. Smart load balancing, conceptually, requires persistent TCP connection…

The thing that baffles me is that you could do high-level random balancing onto smaller clusters that do smart balancing. This would solve most of the problem of overloaded servers. An entire cluster would have to clog up with slow requests before there was any performance impact. So why don't they do this?

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#246

Post by sigre »

squidsoup wrote: post #15

Given that ElasticBeanstalk has support for rails now, does Heroku still have any advantage over AWS for a new startup?

I spent a lot of time trying to come up with a stable installation of EB for a Rails 3.2 app using RDS and just couldn't get it to a state where I'd ever deploy it as production. Here's where I decided to pull the plug:

https://forums.aws.amazon.com/thread.jspa?messageID=410445&#...

Quite a few people still report a myriad of issues with Rails applications. I really want EB to work well with Rails, but just didn't have confidence in it.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#247

Post by ylansegal »

spoiler wrote: post #155
nthj wrote: post #71

I'm inclined to wait until Heroku weighs in to render judgement. Specifically, because their argument depends on this premise: > But elsewhere in their current docs, they make the same old statement loud and clear: > The heroku.com stack only supports single threaded requests. Even if your applicaExplaintion were to fork and support handling multiple requests at once, the routing mesh will never serve more than a sin…

Slightly off topic, but what are everyone's. experience and thoughts about Puma[1]? I am using it on a small production environment with Heroku and I like it, but when we officially launch the app, should we switch to Unicorn? [1] http://puma.io/

I did some performance analysis on puma vs unicorn vs thin a while ago:

http://ylan.segal-family.com/blog/2012/08/20/better-performa...

Although as noted in the comments, I neglected to run threadsafe! and should have probably tried rubinius or jruby. I have been meaning to redo. Take with a grain of salt

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#248

Post by encoderer »

trotsky wrote: post #175

Those charts of "simulated" load balancing strategies don't look at all reasonable at first glance. You certainly don't see such spiky patterns with normal web loads. I think you'd have to have some crazy amount of std. dev in completion time cranked way, way up in your simulation before you saw a bunch of servers stacked at 30 with others at 1. It's not that there is no benefit to better balancing, it's just that I'…

You don't think that page was chosen as an example because it should be a very inexpensive page to render? I read that as saying that the performance problem does affect all pages "even on THIS mostly static page."

As for your skepticism of the graphs, take a look at the annotated R source they provided. I didn't do a deep dive on it or anything but it looked reasonable to me.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#250

Post by fizx »

nostrademons wrote: post #136
harshreality wrote: post #80

Earlier quoted context omitted.

I seem to recall Google mentioning on some blog several years ago that high variance in response latency degrades user experience much more than slightly higher average request times. I can't find the link though; if anyone has it, I'd be grateful.

Jeff Dean wrote a paper on it for CACM: http://cacm.acm.org/magazines/2013/2/160173-the-tail-at-scal... There's a relatively easy fix for Heroku. They should do random routing with a backup second request sent if the first request times fails to respond after a relatively short period of time (say, 95th percentile latency), killing any outstanding requests when the first response comes back in. The amount of bookkeep…

Right now, heroku has one inbound load-balancer that's out of their control (probably ELB(s)). This load balancer hits another layer of mesh routers that heroku does control, and that perform all of herokus magic. In order for "intelligent routing," which is more commonly known as "least-conn" routing to work amongst the mesh layer, all of the mesh routers would have to share state with each other in real-time, which makes this a hard problem.

Alternately, heroku can introduce a third layer between the mesh routers and the inbound random load balancer. This layer consistently hashes (http://en.wikipedia.org/wiki/Consistent_hashing) the api-key/primary key of your app, and sends you to a single mesh router for all of your requests. Mesh routers are/should be blazing fast relative to rails dynos, so that this isn't really a bottleneck for your app. Since the one mesh router can maintain connection state for your app, heroku can implement a least-conn strategy. If the mesh router dies, another router can be automatically chosen.

Post reply on HN