Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

rapgenius.com

Post reply on HN
Print view View original

101–110 of 437 posts

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#101

Post by michaelrkn »

habosa wrote: post #32

Wow. Normally when I read "X is screwing Y!!!" posts on Hacker News I generally consider them to be an overreaction or I can't relate. In this case, I think this was a reasonable reaction and I am immediately convinced never to rely on Heroku again. Does anyone have a reasonably easy to follow guide on moving from Heroku to AWS? Let's keep it simple and say I'm just looking to move an app with 2 web Dynos and 1 worke…

My company switched off of Heroku for our high-load app because of these same problems, but I still really like Heroku for apps with smaller loads, or ones in which I'm will to let a very small percentage of requests take a long time.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#102

Post by ibdknox »

kennystone wrote: post #95
nthj wrote: post #71

I'm inclined to wait until Heroku weighs in to render judgement. Specifically, because their argument depends on this premise: > But elsewhere in their current docs, they make the same old statement loud and clear: > The heroku.com stack only supports single threaded requests. Even if your applicaExplaintion were to fork and support handling multiple requests at once, the routing mesh will never serve more than a sin…

If you have 2 unicorn servers and you happen to get 3 slow requests routed to it, you are still screwed, right? Seems to me like it will still queue on that dyno.

Yeah, the only real question is whether or not it's true that they no longer do intelligent routing. If that is the case, then regardless of anything else the problem exists once you pass a certain scale/request cost. It won't matter if that one dyno can handle hundreds of requests at once, it will still queue stupidly.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#103

Post by dbpatterson »

aristus wrote: post #24
mattj wrote: post #12

So the issue here is two-fold: - It's very hard to do 'intelligent routing' at scale. - Random routing plays poorly with request times with a really bad tail (median is 50ms, 99th is 3 seconds) The solution here is to figure out why your 99th is 3 seconds. Once you solve that, randomized routing won't hurt you anymore. You hit this exact same problem in a non-preemptive multi-tasking system (like gevent or golang).

I do perf work at Facebook, and over time I've become more and more convinced that the most crucial metric is the width of the latency histogram. Narrowing your latency band --even if it makes the average case worse -- makes so many systems problems better (top of the list: load balancing) it's not even funny.

That's probably true, but the value that Heroku is selling (and they charge a lot for it!) is that you _don't_ need to deal with this - that they will balance that out for you.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#104

Post by dedsm »

nthj wrote: post #71

I'm inclined to wait until Heroku weighs in to render judgement. Specifically, because their argument depends on this premise: > But elsewhere in their current docs, they make the same old statement loud and clear: > The heroku.com stack only supports single threaded requests. Even if your applicaExplaintion were to fork and support handling multiple requests at once, the routing mesh will never serve more than a sin…

Nevertheless, random routing is a bad idea even if a dyno can handle multiple requests simultaneously

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#105

Post by wwarnerandrew »

nthj wrote: post #71

I'm inclined to wait until Heroku weighs in to render judgement. Specifically, because their argument depends on this premise: > But elsewhere in their current docs, they make the same old statement loud and clear: > The heroku.com stack only supports single threaded requests. Even if your applicaExplaintion were to fork and support handling multiple requests at once, the routing mesh will never serve more than a sin…

Yes, it's true that the Cedar stack supports forking web servers like unicorn, and that an individual dyno can run multiple workers and therefore serve multiple requests at the same time.

However, dumb routing is still very problematic – even if your dyno can work on two requests simultaneously it's still bad for it to get sent a third request when there are other open dynos.

Also, for apps with a large-ish memory footprint, you can't run very many workers. A heroku dyno has 512mb memory, so if your app has a 250mb footprint, then you can basically only have two workers.

Another essential point to note is that the routing between cedar and bamboo is essentially unchanged. They simply changed the type of apps you can run.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#106

Post by jholman »

46Bit wrote: post #85
toast76 wrote: post #31

Wow. This is explains a lot. We've always been of the opinion that queues were happening on the router, not on the dyno. We consistently see performance problems that, whilst we could tie down to a particular user request (file uploads for example, now moved to S3 direct), we could never figure out why this would result in queuing requests given Heroku's advertised "intelligent routing". We mistakenly thought the occ…

Several Rails apps I develop have been suffering from similar issues. Perhaps 2-3% of requests take 0.4-2s in just processing. If the allocation is a little intelligent, it'll not perform too badly and is less work than much harder optimization. Yet if it's random, it'll queue up horribly . I'm pissed. Spent way too much time unable to explain it to coworkers, thinking I just didn't understand Heroku's platform and t…

> Easiest fix: moving to EC2 next week. I've wanted to ever since these issues became evident but it's hard to make a good argument from handwaving about 'problems'.

Of course, then you need to solve all these problems yourself. That sounds pretty easy, you'll have it done next week no problem!

That was sarcastic, but this isn't: good luck, let us know how it goes.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#107

Post by pencilcode »

zenazn wrote: post #13

Randomized routing isn't all bad. In fact, if Heroku were to switch from purely random routing to minimum-of-two random routing, they'd perform asymptotically better [1]. [1]: http://www.eecs.harvard.edu/~michaelm/postscripts/mythesis.p...

Thanks! Very useful.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#108

Post by pvh »

gleb wrote: post #78
gleb wrote: post #30

Earlier quoted context omitted.

Postgres has a limitation on number of open connections. This is because 1 connection = 1 process. MySQL uses threads, which scales better but has other downsides. The thread-based approach is also possible with Postgres using 3rd party connection pooling apps, e.g. pgbouncer.

BTW, here's a counter-intuitive solution if using pgbouncer is not possible. Simply drop and reestablish connections on every request. In theory this is horrible, since PG connections are so expensive. In practice the cost of establishing a connection is negligible for a Rails app. I do suspect this will make performance "fall-off-the cliff" as you get close to capacity.

Eh, I wouldn't recommend going this route. I'd go down the pgbouncer road instead.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#109

Post by anon640 »

kawsper wrote: post #67
anon640 wrote: post #52

"For a Rails app, each dyno is capable of serving one request at a time." Is this a deliberate design choice on Heroku's part, or is this just how Ruby and Rails work? It sounds bizarre that you would need multiple virtual OS instances just to serve multiple requests at the same time. What are the advantages of this over standard server fork()/threaded accept designs?

It is how Rails server behaves in itself, but that is also how Heroku tells you to do it. Rails can be served with Unicorn ( http://unicorn.bogomips.org/ ) which is a forking app-server. I do believe there was a trick a while back where you could get Heroku to run a Unicorn process on a dyno to get more requests out of it. The process is described here: http://blog.codeship.io/2012/05/06/Unicorn-on-Heroku.html

Isn't this kind of a step backwards? Is Rails really that great that people are willing endure these kinds of limitations just to use it?

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#110

Post by VeejayRampay »

timmaah wrote: post #92
VeejayRampay wrote: post #83

Earlier quoted context omitted.

But then again, those two links don't address the core of the problem: Heroku is used by tons of people around the world. Some of them are paying good money for the service. Given the amount of scrutiny under which they operate, what is the incentive for them to turn an algorithm into a less effective one and still charge the same amount of money in a growing "cloud economy" where companies providing the same kind of…

I didn't jump to any conclusion that they are doing this simply for the money. The only conclusion I jumped to was that they ditched the routing they originally said they had (without telling anyone) and that their routing is worse than what you get as a default from passenger.

My bad if that wasn't clear in my comment: the sensationalist bit wasn't directed at you but at the general tone of the thread. In fact, I actually upvoted your comment for providing two links highly relevant to the article.
Post reply on HN