Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Answering your questions about Heroku routing and web performance

blog.heroku.com

Post reply on HN
Print view View original

31–40 of 168 posts

Re: Answering your questions about Heroku routing and web performance

#31

Post by tptacek »

ebbv wrote: post #18
tptacek wrote: post #7

Earlier quoted context omitted.

Is anybody operating at Heroku's scale offering centralized request routing queues? At what price?

Not that I know of but that's why I'm saying it would be a premium product. Likely pricing would have to scale with the number of dynos running behind the router. But that's the service people thought they were getting and what they wanted. If Heroku prices out the intelligent routing and says; "Ok you can have intelligent routing with your current backend stack, but it's going to cost you $25/mo for evert 10 dynos,…

If it's truly impossible to get centralized request routing queues at Heroku's scale in any other product offering, that is evidence that a demand that Heroku provide it might be unreasonable.

Aside from that, I am extremely sympathetic to Heroku's engineering point here --- it's obviously hard for HN to extract the engineering from the drama in this case! Randomized dispatch seems like an eminently sound engineering solution to the request routing problem, and the problems actually implementing it in production seem traceable almost entirely to††† the ways Rails managed to set back scalable web request dispatch by roughly a decade††††.

††† IT IS ALL LOVE WITH ME AND THIS POINT COMING UP HERE...

†††† ...it was probably worth it!

Re: Answering your questions about Heroku routing and web performance

#32

Post by adamwiggins »

cynicalkane wrote: post #22
adamwiggins wrote: post #19

Earlier quoted context omitted.

I hear you. Heroku's value proposition is that we abstract away infrastructure tasks and let you focus on your app. Keeping you from needing to deal with load balancers is one part of that. If you're worried about how routing works then you're in some way dealing with load balancing. However, if someone chose us purely because of a routing algorithm, we probably weren't a great fit for them to begin with. We're not s…

You sound like a politician talking to someone of the opposite party, in that you say "I hear you", but then completely fail to address anyone's concerns. Selling a "magic black box" that guarantees certain properties, changes them, then lies about having changed them presents a liability for users who want to do serious work. A major selling point of Heroku is that scaling dynos wouldn't be a risk. This guarantee is…

Harsh. I'm going to ignore the more inflammatory parts of this (feel free to restate if you want me to engage in discussion), but one bit did grab my attention:

"A major selling point of Heroku is that scaling wouldn't be a risk"

This is interesting, especially the word "risk." Can you expand on this?

Re: Answering your questions about Heroku routing and web performance

#33

Post by bcgraham »

cynicalkane wrote: post #22
adamwiggins wrote: post #19

Earlier quoted context omitted.

I hear you. Heroku's value proposition is that we abstract away infrastructure tasks and let you focus on your app. Keeping you from needing to deal with load balancers is one part of that. If you're worried about how routing works then you're in some way dealing with load balancing. However, if someone chose us purely because of a routing algorithm, we probably weren't a great fit for them to begin with. We're not s…

You sound like a politician talking to someone of the opposite party, in that you say "I hear you", but then completely fail to address anyone's concerns. Selling a "magic black box" that guarantees certain properties, changes them, then lies about having changed them presents a liability for users who want to do serious work. A major selling point of Heroku is that scaling dynos wouldn't be a risk. This guarantee is…

This is really unfair. This comment:

>A lot of Heroku's apparent value came from the intelligent routing feature. Everybody knew that it was harder to implement than random routing, that's why they were willing to pay Heroku for it.

is being addressed by Adam in this comment:

>Heroku's value proposition is that we abstract away infrastructure tasks and let you focus on your app. Keeping you from needing to deal with load balancers is one part of that. If you're worried about how routing works then you're in some way dealing with load balancing.

I think Adam is getting to a really fair point here, which is that nobody really minds whether the particular algorithm is used. If A-Company is using "intelligent routing" and B-Company uses "random routing," but B-Company has better performance and slower queue times, who are you going to choose? You're going to choose B-Company.

At the end of the day, "intelligent routing" is really nothing more than a feather in your cap. People care about performance. That's what started this whole thing - lousy performance. Better performance is what makes it go away, not "intelligent routing."

Re: Answering your questions about Heroku routing and web performance

#35

Post by chc »

tvladeck wrote: post #23

All of the stuff that Heroku are doing now to mitigate the routing issues for their Bamboo customers are things they should have done when it first became an issue. They are not going above and beyond in any way to make up for the lost time and money their customers. Again, this is not about -how advanced Heroku's technology is on an absolute level -how challenging routing is for their scale -what competitors offer i…

Your complaints are all about the past. What are you looking for here? For Heroku to invent a time machine?

Re: Answering your questions about Heroku routing and web performance

#36

Post by meritt »

badgar wrote: post #26

> 1k req/min Also known as trickle of traffic. Hooray for using bigger numbers and a nonstandard unit to hide inadequacy! Does Heroku use req/min throughout their service? I can't understand why they would, unless they also can't build the infrastructure to measure on a per-second basis. > After extensive research and experimentation, we have yet to find either a theoretical model or a practical implementation that b…

I too was shocked at that. Also that 6 dynos is apparently the average size to handle that load.

It takes $179/mo (6 dynos) to handle 17 requests/second? That's insane.

Re: Answering your questions about Heroku routing and web performance

#37

Post by cmelbye »

They still fail to understand that using Unicorn doesn't magically fix this issue. Like, at all. It simply means that the dunk gets tied up when n+1 requests (n is number of unicorn workers) get randomly routed to it instead of just 1. It's in no way comparable to a node.js server that handles thousands of concurrent requests asynchronously. They're simply two different designs, and Ruby's traditional design is fundamentally incompatible with Heroku's router.

Re: Answering your questions about Heroku routing and web performance

#39

Post by adamwiggins »

thinkbohemian wrote: post #25
boundlessdreamz wrote: post #16

Earlier quoted context omitted.

Unless the dyno signals to the router that it is busy, isn't this just postponing the problem? Per dyno, unicorn can handle more requests but requests will still get queued at dyno level if one of the requests is a slow one (say 2-3 seconds)

If only 1 request is slow and you have 7-8 unicorn workers, only one of them will stay busy. Unicorn knows which of it's workers are busy and does not queue jobs behind individual worker but rather behind the master who delegates the request to the first available worker.

Precisely. As mentioned in the FAQ, putting the queueing logic closer to the process which is ultimately going to serve the request is a more horizontal-scale friendly way of tackling the queueing problem.

It works fantastically well for backends that can support 20+ concurrent connections, e.g. Node.js, Twisted, JVM threading, etc. It works less well as you can put fewer connections in each backend, which is part of why we're working on larger dynos.

Re: Answering your questions about Heroku routing and web performance

#40

Post by ebbv »

chc wrote: post #29
ebbv wrote: post #18

Earlier quoted context omitted.

Not that I know of but that's why I'm saying it would be a premium product. Likely pricing would have to scale with the number of dynos running behind the router. But that's the service people thought they were getting and what they wanted. If Heroku prices out the intelligent routing and says; "Ok you can have intelligent routing with your current backend stack, but it's going to cost you $25/mo for evert 10 dynos,…

$25 a month? Did you forget a few zeroes?

It was just a placeholder price. :)
Post reply on HN