181–190 of 437 posts
Good lord!!!!! Percentage of the requests served within a certain time (ms) 50% 844 66% 2977 75% 5032 80% 7575 90% 16052 95% 20069 98% 29282 99% 30029 100% 30029 (longest request) Those numbers are amazingly awful. If I ever run ab and see 4 digits I assume I need to optimize my software or server. But 5 digits? Why in the world would a company spend $20,000 a month for service this awful?
They want to force the issue with a public spat. Fair enough. But, they also might also be able to self-help quite a bit. RG makes no mention of using more than 1 unicorn worker per dyno. That could help, making a smaller number of dynos behave more like a larger number. I think it was around when Heroku switched to random routing that they also became more officially supportive of dynos handling multiple requests at…
On further thought, Heroku users could probably even approximate the benefits from the Mitzenmacher power-of-two-choices insight (mentioned elsewhere in thread), without Heroku's systemic help, by having dynos shed their own excess load. Assume each unicorn can tell how many of its workers are engaged. The 1st thing any worker does – before any other IO/DB/net-intensive work – would be to check if the dyno is 'loaded…
I'm unaware of how Heroku does things. I'd guess they dropped the global queue because it's unpractical (failure prone, not scalable as it's a single point of contention).
I'm mostly surprised to see people happy being able to handle 1 or 2 requests in parallel per instance in general. That sounds absolutely insane to me.
Why not get yourself ONE beefy server? (or two) That should be able to handle your 150 requests per second, simplify your architecture a lot, and buying it would be cheaper than 1 month on Heroku (at $20,000/month).
I'm inclined to wait until Heroku weighs in to render judgement. Specifically, because their argument depends on this premise: > But elsewhere in their current docs, they make the same old statement loud and clear: > The heroku.com stack only supports single threaded requests. Even if your applicaExplaintion were to fork and support handling multiple requests at once, the routing mesh will never serve more than a sin…
Slightly off topic, but what are everyone's. experience and thoughts about Puma[1]? I am using it on a small production environment with Heroku and I like it, but when we officially launch the app, should we switch to Unicorn? [1] http://puma.io/
Earlier quoted context omitted.
If you have 2 unicorn servers and you happen to get 3 slow requests routed to it, you are still screwed, right? Seems to me like it will still queue on that dyno.
how does it decide to queue at the dyno level anyway? Does it check for connection refusal at the TCP level?
Earlier quoted context omitted.
Well, at X level of concurrency, wouldn't most set ups with load balancers start to spit numbers like that?
no, at x level of concurrency most set ups wont spend 16 seconds or more on 10% of their requests.
Rap Genius is employing a classic rap-mogul strategy: start a beef
Maybe this is the start of Off the Rails Rap.
Earlier quoted context omitted.
Nope, requests could still get queued behind a dyno that's busy with a long request.
Sure, but the real issue the article identifies is that, under random routing, they need to keep doubling the number of dynos to halve the odds of bad queueing, which leads to absurd factor of 50 requirements to get back to what they had before. With round robin, the increase should be much more linear.
The real issue is being able to figure out instances to avoid when some requests end up being slow. To put it another way, ideal balancing in this case isn't about evenly splitting all requests, but evenly splitting all processing (or waiting) time.
If you can guarantee that requests tend to take pretty stable and uniform time, then random or round-robin distribution should give good results. If you can't, some requests will be stuck waiting behind others and their waiting time will accumulate. You'll see worse behaviour when two or more of the bad slow requests get queued one after the other.