Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

rapgenius.com

Post reply on HN
Print view View original

171–180 of 437 posts

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#171

Post by damian2000 »

arcatek wrote: post #153
chc wrote: post #115

Earlier quoted context omitted.

> But really, throwing in the towel at intelligent routing and replacing it with "random routing" is horrific, if true. The thing is, their old "intelligent routing" was really just "we will only route one request at a time to a dyno." In other words, what changed is that they now allow dynos to serve multiple requests at a time. When you put it that way, it doesn't sound as horrific, does it?

The requests will not be served at the same time , that's the whole point. If a request is routed to a busy dyno, you will have to wait that the previous job finish before being able to start yours.

I believe the article is saying that this wait will only occur with Rails due to it being single threaded? Reactor pattern frameworks like node don't suffer the same issue.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#172

Post by lil_tee »

mattj wrote: post #12

So the issue here is two-fold: - It's very hard to do 'intelligent routing' at scale. - Random routing plays poorly with request times with a really bad tail (median is 50ms, 99th is 3 seconds) The solution here is to figure out why your 99th is 3 seconds. Once you solve that, randomized routing won't hurt you anymore. You hit this exact same problem in a non-preemptive multi-tasking system (like gevent or golang).

Simulation author here with some additional analysis using a faster distribution of request times. If you use a distribution with median 50 ms, 90th percentile 225 ms, and 99.9th percentile 898 ms, then you need 30 intelligent dynos to handle 9000 requests/minute without queueing. In the same scenario with 30 naive dynos, 44% of requests get queued.

Animations and results are in the explanation at http://rapgenius.com/1502046

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#173

Post by mleach »

The balance of a subjective, sensationalist headline with objective statistical simulation was impressive.

I'm a huge Heroku fan using Cedar/Java, but can't help but wonder how many optimization options remain for Rails Developers, assuming nothing else changes on Heroku:

* Serving static HTML from CDN * Unicorn * Redis caching with multiget requests

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#174

Post by [deleted] »

gojomo wrote: post #138
gojomo wrote: post #116

They want to force the issue with a public spat. Fair enough. But, they also might also be able to self-help quite a bit. RG makes no mention of using more than 1 unicorn worker per dyno. That could help, making a smaller number of dynos behave more like a larger number. I think it was around when Heroku switched to random routing that they also became more officially supportive of dynos handling multiple requests at…

On further thought, Heroku users could probably even approximate the benefits from the Mitzenmacher power-of-two-choices insight (mentioned elsewhere in thread), without Heroku's systemic help, by having dynos shed their own excess load. Assume each unicorn can tell how many of its workers are engaged. The 1st thing any worker does – before any other IO/DB/net-intensive work – would be to check if the dyno is 'loaded…

[deleted]

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#175

Post by trotsky »

Those charts of "simulated" load balancing strategies don't look at all reasonable at first glance. You certainly don't see such spiky patterns with normal web loads. I think you'd have to have some crazy amount of std. dev in completion time cranked way, way up in your simulation before you saw a bunch of servers stacked at 30 with others at 1.

It's not that there is no benefit to better balancing, it's just that I've never seen it have anything close to that impact. It seems like it's only being perceived as a problem here because somebody drank too much of the (old) kool-aid.

Some of the other numbers are hard to take at face value as well. 6000ms avg on a specific page? If requests are getting distributed randomly shouldn't all your pages show a similar average time in queue? Sounds more like they're using a hash balancing alg and the static page was hashing on to a hot spot.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#176

Post by trotsky »

eli wrote: post #160
lkrubner wrote: post #143

Good lord!!!!! Percentage of the requests served within a certain time (ms) 50% 844 66% 2977 75% 5032 80% 7575 90% 16052 95% 20069 98% 29282 99% 30029 100% 30029 (longest request) Those numbers are amazingly awful. If I ever run ab and see 4 digits I assume I need to optimize my software or server. But 5 digits? Why in the world would a company spend $20,000 a month for service this awful?

Well, at X level of concurrency, wouldn't most set ups with load balancers start to spit numbers like that?

no, at x level of concurrency most set ups wont spend 16 seconds or more on 10% of their requests.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#177

Post by jpatokal »

benihana wrote: post #94

I really like Rap Genius, but I wish they would tone down the blackness of the background. Reading #CCC text on #000 background makes my eyes bug out.

I'm glad you clarified that your first sentence is about their CSS choices. ;)

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#179

Post by benjamincburns »

This kind of validates an idea I've been flirting with: a Heroku-like service which routes requests via AMQP or similar message broker and actually exposes the routing dynamics to the client apps.

From a naive, inexperienced view the idea of having web nodes "pull" requests from a central queue rather than the queue taking uneducated guesses seems to be a no-brainer. I can see this making long-running requests (keep-alive, streaming, etc) a bit more difficult, but not impossible.

What am I missing? This seems so glaringly obvious that it must have been done before...

Post reply on HN