371–380 of 437 posts
Earlier quoted context omitted.
The thing that baffles me is that you could do high-level random balancing onto smaller clusters that do smart balancing. This would solve most of the problem of overloaded servers. An entire cluster would have to clog up with slow requests before there was any performance impact. So why don't they do this?
My off the cuff answer to your question is, because it's probably not quite that simple ;)
Earlier quoted context omitted.
Why do you 'want to love' Heroku? Because their marketing speak is so great?
Because the premise is a good one, abstracting away some of the lower level details so people can just worry about building their apps. It has always been a tough abstraction to pull off though. It is possible the answer is simply that rap genius has outgrown heroku, in the same way as twitter outgrowing rails it doesn't make it a bad tool for everyone.
This is Oren Teich, I run Heroku. I've read through the OP, and all of the comments here. Our job at Heroku is to make you successful and we want every single customer to feel that Heroku is transparent and responsive. Getting to the bottom of this situation and giving you a clear understanding of what we’re going to do to make it right is our top priority. I am committing to the community to provide more information…
What's the point of posting a link to the front page of your blog, where the most recent article is 15 days old (4 hours after the comment above)? What we want to know: - is the OP right or wrong? That is, did you switch from smart to naive routing, for all platforms, and without telling your existing or future customers? - if you did switch from smart to naive routing, what was the rationale behind it? (The OP is li…
Well, he promised a detailed blog post, at which point that link will be extremely helpful.
I do not think it is fair to expect an immediate detailed response to those questions. If I were CEO of Heroku, I wouldn't say anything definite until after talking to the engineers and product managers involved--even if I was already pretty sure what happened. The worst thing you could do at this point is say something that's just wrong.
Earlier quoted context omitted.
How was 5 ms added? Multiple sleep states per request? I imagine the long tail disappears in a similar way that a traffic jam is prevented by lowering the speed limit.
I think you misunderstood: they optimized the long running requests and the optimization incurred 5ms performance loss for short requests. It is not that the additional 5ms solved the problem.
Earlier quoted context omitted.
We're discussing Rails on Heroku specifically which, non-unicorn, should be a "next available checkout counter" situation. Ideally it should be possible to make this an optional behavior that you can choose to turn on for Rails apps.
I agree there should be a better way - it's just important to understand than Rails doesn't get any special treatment on a PaaS done correctly, so it's important to come up with a generic solution. I think part of the solution would be customizable option(i.e.. how many requests can each dyno handle simultaneously), probably combined with intelligently monitoring/balancing proxy load so new requests always go to the…
Its kind of weird to describe not optimizing the entire platform provided to apps as "PaaS done correctly". Making a PaaS more generic has a certain kind of value in terms of broadening the audience and enabling heterogenous systems to be implemented on it, but if you are doing that by sacrificing the optimization of the individual application platforms available, you are losing some of what makes a PaaS valuable as opposed to roll-your-own platform support on top of a generic IaaS.
Its especially problematic to say that worsening support for the main existing app framework in use on an establish PaaS and giving existing customers orders of magnitude less value for their money is doing something right.
> I think part of the solution would be customizable option
That's probably a good idea, though the default for existing apps should not have changed, especially without clear up-front notice.
> But apps are increasingly unlike a checkout line.
Existing apps are, for the most part, exactly as much like a checkout line as they were before the unannounced change.
Earlier quoted context omitted.
If you have 2 unicorn servers and you happen to get 3 slow requests routed to it, you are still screwed, right? Seems to me like it will still queue on that dyno.
This is true - unicorn masks the symptoms for a period of time but does not solve the underlying problem in the way a global request queue would. Also, if the unicorn process is doing something cpu intensive (vs waiting on a 3rd party service or io etc) then it won't serve 3 requests simultaneously as fast as single processes would.
Why not get yourself ONE beefy server? (or two) That should be able to handle your 150 requests per second, simplify your architecture a lot, and buying it would be cheaper than 1 month on Heroku (at $20,000/month).
Because when that one beefy server goes tits-up, you're out of business. Same with two.
Earlier quoted context omitted.
To amplify, I do a ton of queue adjustment work with Unicorn at the backlog level. It is so frequent that we set up Unicorn startup to read from an ENV variable on Heroku that we set as needed. With two Unicorn workers we found that 25 was the best backlog threshold to accept (it refuses additional requests). When we were able to go to 5 Unicorn workers on Heroku we had to start to adjust that.
You don't happen to have any documentation for how to do that do you? Very curious. Never seen anything about setting up Unicorn like this prior (I'm just using 3 or 4 Unicorns/dyno currently)
Earlier quoted context omitted.
Rails doesn't need to be treated specially. All that is needed is a "maximum number of simultaneous connections to pass to this backend" setting coupled with load balancing by available slots rather than purely randomly. The issue here isn't that Rails needs to be treated specially - this problem applies to various extent in any type of backend where some types of requests might turn out to be computationally heavy o…
I'm sure OP could prefer the extra 10ms, but then everyone else who can deal with random dispatching right now has to pay a 10ms penalty because OP built his stuff on a technology that can deal with only one request at a time on a server, which boggles the mind to begin with.
The "everyone else who can deal with random dispatching right now" is a much smaller group than you think. Anyone who has long running requests that grind the CPU or disk when running, will be at high risk of seeing horribly nasty effects from random dispatching, no matter whether their stack in ideal conditions have no problem handling concurrent requests.
It's just less immediately apparent, as any dynos that start aggregating multiple long running requests will "just" get slower and slower instead of blocking normally low-latency requests totally.
To whom it may concern,
We are long time users of Heroku and are big fans of the service. Heroku allows us to focus on application development. We recently read an article on HN entitled 'Heroku's Ugly Secret' http://s831.us/11IIoMF
We have noticed similar behavior, namely increasing dynos does not provide performance increases we would expect. We continue to see wildly different performance responses across different requests that New Relic metrics and internal instrumentation can not explain.
We would like the following:
1. A response from Heroku regarding the analysis done in the article, and 2. Heroku-supplied persistant logs that include information how long requests are queued for processing by the dynos
Thanks in advance for any insight you can provide into this situation and keep up the good work.