Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

rapgenius.com

Post reply on HN
Print view View original

111–120 of 437 posts

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#111

Post by icebraining »

lil_tee wrote: post #33
zenazn wrote: post #13

Randomized routing isn't all bad. In fact, if Heroku were to switch from purely random routing to minimum-of-two random routing, they'd perform asymptotically better [1]. [1]: http://www.eecs.harvard.edu/~michaelm/postscripts/mythesis.p...

I'm not familiar with minimum-of-two-random routing, but it does seem like assigning request to dynos in sequence would perform much better than assigning randomly (ie in a scenario with n dyno capacity, request 1 => dyno 1, request 2 => dyno 2, ... request n => dyno n, request n+1 => dyno 1, ..., repeat) That'd be probably significantly better than the case of (request i => dyno picked out of hat) for all i

That's essentially round-robin, but it still requires syncing the data of which dyno is next, which is probably what they're trying to avoid.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#112

Post by beambot »

googletron wrote: post #66

Does anyone know how if python applications are affected by this? I know they can handle multiple requests per dyno, I would be interested to know if random routing affects python apps too.

For python / Django you can use a Procfile specifying gunicorn (rather than stock manage.py) with multiple worker threads, eg.

web: gunicorn myapp.wsgi -b 0.0.0.0:$PORT -w 5

Then you will have 5 parallel "single-threaded" instances on each dyno rather than just 1. This will partially ameliorate the problem, but probably not 100%. (NOTE: This is speculation since Heroku hasn't weighed in yet)

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#114

Post by barmstrong »

kennystone wrote: post #95
nthj wrote: post #71

I'm inclined to wait until Heroku weighs in to render judgement. Specifically, because their argument depends on this premise: > But elsewhere in their current docs, they make the same old statement loud and clear: > The heroku.com stack only supports single threaded requests. Even if your applicaExplaintion were to fork and support handling multiple requests at once, the routing mesh will never serve more than a sin…

If you have 2 unicorn servers and you happen to get 3 slow requests routed to it, you are still screwed, right? Seems to me like it will still queue on that dyno.

This is true - unicorn masks the symptoms for a period of time but does not solve the underlying problem in the way a global request queue would.

Also, if the unicorn process is doing something cpu intensive (vs waiting on a 3rd party service or io etc) then it won't serve 3 requests simultaneously as fast as single processes would.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#115

Post by chc »

FireBeyond wrote: post #5

This should be more prominent. I want to love Heroku, and am sure that I could. But really, throwing in the towel at intelligent routing and replacing it with "random routing" is horrific, if true. It's arguable that the routing mesh and scaling dynamics of Heroku are a large part, if not -the- defining reason for someone to choose Heroku over AWS directly. Is it a "hard" problem? I'm absolutely sure it is. That's on…

> But really, throwing in the towel at intelligent routing and replacing it with "random routing" is horrific, if true.

The thing is, their old "intelligent routing" was really just "we will only route one request at a time to a dyno." In other words, what changed is that they now allow dynos to serve multiple requests at a time. When you put it that way, it doesn't sound as horrific, does it?

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#116

Post by gojomo »

They want to force the issue with a public spat. Fair enough.

But, they also might also be able to self-help quite a bit. RG makes no mention of using more than 1 unicorn worker per dyno. That could help, making a smaller number of dynos behave more like a larger number. I think it was around when Heroku switched to random routing that they also became more officially supportive of dynos handling multiple requests at once.

There's still the risk of random pileups behind long-running requests, and as others have noted, it's that long-tail of long-running requests that messes things up. Besides diving into the worst offender requests, perhaps simply segregating those requests to a different Heroku-app would lead to a giant speedup for most users, who rarely do long-running requests.

Then, the 90% of requests that never take more than a second would stay in one bank of dynos, never having pathological pile-ups, while the 10% that take 1-6 seconds would go to another bank (by different entry URL hostname). There'd still be awful pile-ups there, but for less-frequent requests, perhaps only used by a subset of users/crawler-bots, who don't mind waiting.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#117

Post by 46Bit »

jholman wrote: post #106
46Bit wrote: post #85

Earlier quoted context omitted.

Several Rails apps I develop have been suffering from similar issues. Perhaps 2-3% of requests take 0.4-2s in just processing. If the allocation is a little intelligent, it'll not perform too badly and is less work than much harder optimization. Yet if it's random, it'll queue up horribly . I'm pissed. Spent way too much time unable to explain it to coworkers, thinking I just didn't understand Heroku's platform and t…

> Easiest fix: moving to EC2 next week. I've wanted to ever since these issues became evident but it's hard to make a good argument from handwaving about 'problems'. Of course, then you need to solve all these problems yourself. That sounds pretty easy, you'll have it done next week no problem! That was sarcastic, but this isn't: good luck, let us know how it goes.

> Of course, then you need to solve all these problems yourself. That sounds pretty easy, you'll have it done next week no problem!

I agree with this, actually. I know it's not simple to do your own servers when you're growing. Yet I'd rather improve my existing ops skills a bit than have to setup everything as async APIs (on EC2 anyway). That's the only way I can see that I can solve this.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#118

Post by douglasfshearer »

c3 wrote: post #37
htsh wrote: post #26

Why not hire a devops guy & rack your own hardware? Or get some massive computing units at amazon (just as good but more expensive)? This reminds me of the excellent 5 stages of hosting story shared on here from a while back: http://blog.pinboard.in/2012/01/the_five_stages_of_hosting/

we just switched 1/3rd of our infrastructure off our existing host (engineyard, which uses AWS) onto raw AWS and saved about $2500/month. You can do it too!

Interested in how you achieved this. Did you change server setup significantly from the default EY stack?

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#120

Post by kmfrk »

squidsoup wrote: post #15

Given that ElasticBeanstalk has support for rails now, does Heroku still have any advantage over AWS for a new startup?

SSL pain can be a major pain to set up. Is the process of setting it up remotely easy compared to Heroku?
Post reply on HN