What's the advantage of randomized routing over intelligent routing? Why would this change be made?
Statelessness. You don't have to remember where you've sent recent requests, which ones are still in process, or how long they've taken.
Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails
91–100 of 437 posts
Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails
#92Earlier quoted context omitted.
This is not a new revelation. I got them to admit to it 2 years ago. http://tiwatson.com/blog/2011-2-17-heroku-no-longer-using-a-... and specifically: https://groups.google.com/forum/?fromgroups#!msg/heroku/8eOo...
But then again, those two links don't address the core of the problem: Heroku is used by tons of people around the world. Some of them are paying good money for the service. Given the amount of scrutiny under which they operate, what is the incentive for them to turn an algorithm into a less effective one and still charge the same amount of money in a growing "cloud economy" where companies providing the same kind of…
I didn't jump to any conclusion that they are doing this simply for the money.
The only conclusion I jumped to was that they ditched the routing they originally said they had (without telling anyone) and that their routing is worse than what you get as a default from passenger.
Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails
#93How does this compare to EngineYard/AppFog/any other Heroku competitors?
Engine Yard is more like opinionated configuration management. It allocates and configures EC2 instances that you can log into like normal. The software stack is HAProxy, nginx, unicorn, etc, and customizable through the web interface and/or chef.
Is there a reason for using both HAproxy and Nginx?
Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails
#94I really like Rap Genius, but I wish they would tone down the blackness of the background. Reading #CCC text on #000 background makes my eyes bug out.
Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails
#95I'm inclined to wait until Heroku weighs in to render judgement. Specifically, because their argument depends on this premise: > But elsewhere in their current docs, they make the same old statement loud and clear: > The heroku.com stack only supports single threaded requests. Even if your applicaExplaintion were to fork and support handling multiple requests at once, the routing mesh will never serve more than a sin…
If you have 2 unicorn servers and you happen to get 3 slow requests routed to it, you are still screwed, right? Seems to me like it will still queue on that dyno.
Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails
#96OK, maybe I'm missing something here, but it seems to me that the OP's real problem is that he's artificially limiting himself to one request per dyno. They now allow a dyno to serve more than one request at a time, and he's presenting that as a bad thing ! It seems to me that the answer to Rap Genius' problems is not "rage at Heroku," but rather "gem 'unicorn'".
We did this, but all it did was buy us a bit of extra time before we ran into the same problem again - a very small percentage (<0.1%) of requests creating a queue that destroyed performance for the rest of them. Also, FWIW, Heroku does not officially support Unicorn, and you have to make sure that you don't run out of memory on your dynos (we tanked our app the first time we tried Unicorn with 4 processes).
Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails
#97What's the advantage of randomized routing over intelligent routing? Why would this change be made?
[deleted]
Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails
#98I'm inclined to wait until Heroku weighs in to render judgement. Specifically, because their argument depends on this premise: > But elsewhere in their current docs, they make the same old statement loud and clear: > The heroku.com stack only supports single threaded requests. Even if your applicaExplaintion were to fork and support handling multiple requests at once, the routing mesh will never serve more than a sin…
If you have 2 unicorn servers and you happen to get 3 slow requests routed to it, you are still screwed, right? Seems to me like it will still queue on that dyno.
That's exactly what happened to us - switching to unicorn bought us a little time and a bit of performance, but we hit the exact same problems again after a couple more weeks of growth.
Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails
#99Does anyone know how if python applications are affected by this? I know they can handle multiple requests per dyno, I would be interested to know if random routing affects python apps too.
As long as they've got a limit on the maximum amount of concurrent requests, they'll be affected. It might well not be as obvious.
Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails
#100So the issue here is two-fold: - It's very hard to do 'intelligent routing' at scale. - Random routing plays poorly with request times with a really bad tail (median is 50ms, 99th is 3 seconds) The solution here is to figure out why your 99th is 3 seconds. Once you solve that, randomized routing won't hurt you anymore. You hit this exact same problem in a non-preemptive multi-tasking system (like gevent or golang).
Re the distribution, absolutely. That "FIFTY TIMES" is totally due to the width of the distribution. Although, you know, even if their app was written such that every single request took exactly 100ms of dyno time, this random routing would create the problem all over again, to some degree.
As for the intelligent routing, could you explain the problem? The goal isn't to predict which request will take a long time, the goal is to not give more work to dynos that already have work. Remember that in the "intelligent" model it's okay to have requests spend a little time in the global queue, a few ms mean across all requests, even when there are free dynos.
Isn't it as simple as just having the dynos pull jobs from the queue? The dynos waste a little time idle-spinning until the central queue hands them their next job, but that tax would be pretty small, right? Factor of two, tops? (Supposing that the time for the dyno-initiated give-me-work request is equal to the mean handling time of a request.) And if your central queue can only handle distributing to say 100 dynos, I can think of relatively simple workarounds that add another 10ms of lag every factor-of-100 growth, which would be a hell of a lot better than this naive routing.
What am I missing?