81–90 of 437 posts
How does this compare to EngineYard/AppFog/any other Heroku competitors?
Maybe this is a dumb question, but wouldn't straightforward Round Robin routing by Heroku restore their "one dyno = one more concurrent request" promise without incurring the scaling liabilities of tracking load across an arbitrarily large number of dynos?
The only way that round robin would be arguably better than random routing is if your random selection is not evenly distributed.
Someone from Heroku really needs to weigh in on this.
This is not a new revelation. I got them to admit to it 2 years ago. http://tiwatson.com/blog/2011-2-17-heroku-no-longer-using-a-... and specifically: https://groups.google.com/forum/?fromgroups#!msg/heroku/8eOo...
Heroku is used by tons of people around the world. Some of them are paying good money for the service. Given the amount of scrutiny under which they operate, what is the incentive for them to turn an algorithm into a less effective one and still charge the same amount of money in a growing "cloud economy" where companies providing the same kind of service are a dime a dozen (AWS, Linode, Engine Yard, etc)?
How does that benefit their business if "calling their BS" is as easy as firing Apache Benchmark, collecting results, drawing a few charts and flat out "prove" that they're lying about the service they provide??
I mean, I doubt Heroku is that stupid, they know how their audience doesn't give them much room for mistakes. So as nice as the story sounds on paper, I'd really like another take on all this, either from other users of Heroku, independent dev ops, researchers, routing algorithms specialists or even Heroku themselves before we all too hastily jump to sensationalist conclusions.
Wow. This is explains a lot. We've always been of the opinion that queues were happening on the router, not on the dyno. We consistently see performance problems that, whilst we could tie down to a particular user request (file uploads for example, now moved to S3 direct), we could never figure out why this would result in queuing requests given Heroku's advertised "intelligent routing". We mistakenly thought the occ…
Is it so that a dyno can only handle a single user request at a time? Why dos it not use some kind of scheduling system to handle other task while one task is waiting on i/o?
Wow. This is explains a lot. We've always been of the opinion that queues were happening on the router, not on the dyno. We consistently see performance problems that, whilst we could tie down to a particular user request (file uploads for example, now moved to S3 direct), we could never figure out why this would result in queuing requests given Heroku's advertised "intelligent routing". We mistakenly thought the occ…
I'm pissed. Spent way too much time unable to explain it to coworkers, thinking I just didn't understand Heroku's platform and that it was my fault.
Turns out, I didn't understand it, because Heroku never thought to clearly mention something that's pretty important.
Easiest fix: moving to EC2 next week. I've wanted to ever since our issues became evident but it's hard to make a good argument from handwaving about 'problems'.
So the issue here is two-fold: - It's very hard to do 'intelligent routing' at scale. - Random routing plays poorly with request times with a really bad tail (median is 50ms, 99th is 3 seconds) The solution here is to figure out why your 99th is 3 seconds. Once you solve that, randomized routing won't hurt you anymore. You hit this exact same problem in a non-preemptive multi-tasking system (like gevent or golang).
I do perf work at Facebook, and over time I've become more and more convinced that the most crucial metric is the width of the latency histogram. Narrowing your latency band --even if it makes the average case worse -- makes so many systems problems better (top of the list: load balancing) it's not even funny.
What's the advantage of randomized routing over intelligent routing? Why would this change be made?
On the other hand, you can do randomised routing without knowing any state at all. You can do it with more than one routing node as well, which makes scaling almost trivial.
I presume there are Hard Problems associated with partitioning a Heroku-style cluster for intelligent routing, or that's what they would have done.
I'm inclined to wait until Heroku weighs in to render judgement. Specifically, because their argument depends on this premise: > But elsewhere in their current docs, they make the same old statement loud and clear: > The heroku.com stack only supports single threaded requests. Even if your applicaExplaintion were to fork and support handling multiple requests at once, the routing mesh will never serve more than a sin…
Puma define 4:8 threads or Unicorn 3 workers.
This should be more prominent. I want to love Heroku, and am sure that I could. But really, throwing in the towel at intelligent routing and replacing it with "random routing" is horrific, if true. It's arguable that the routing mesh and scaling dynamics of Heroku are a large part, if not -the- defining reason for someone to choose Heroku over AWS directly. Is it a "hard" problem? I'm absolutely sure it is. That's on…
Even their own docs were wrong on this for a long time. It bit me in the ass back in 2011 and I got them to clarify and update the documentation just a little. http://tiwatson.com/blog/2011-2-17-heroku-no-longer-using-a-...