Answering your questions about Heroku routing and web performance • Hacker News Board

Re: Answering your questions about Heroku routing and web performance

#71

Post by jcampbell1 » Wed, Apr 03, 2013, 7:33 PM UTC

ebbv wrote: post #4

It seems to me that Heroku is still failing to understand (or at least cop to) the fact that the switch from intelligent to randomized routing was a loss of a major reason people chose Heroku in the first place. A lot of Heroku's apparent value came from the intelligent routing feature. Everybody knew that it was harder to implement than random routing, that's why they were willing to pay Heroku for it. Nobody's argu…

Random routing will work fine so long as the operating system has the ability to intelligently schedule the work. The problem is that the dynos were setup to handle request sequentially. Unicorn will help mitigate the problem, but the best solution is to not try and serve webpages from hardware that is about as powerful as an old mobile phone. The 2x dynos are a step in the right direction, but I have no idea why they don't offer a real app server like a 16x dyno.

Re: Answering your questions about Heroku routing and web performance

#72

Post by tptacek » Wed, Apr 03, 2013, 7:38 PM UTC

jules wrote: post #66

tptacek wrote: post #31

Earlier quoted context omitted.

If it's truly impossible to get centralized request routing queues at Heroku's scale in any other product offering, that is evidence that a demand that Heroku provide it might be unreasonable. Aside from that, I am extremely sympathetic to Heroku's engineering point here --- it's obviously hard for HN to extract the engineering from the drama in this case! Randomized dispatch seems like an eminently sound engineering…

Random routing vs fully centralized request routing is a false dichtonomy. Suppose you have 100 nodes, and you have a router that routes randomly to one of those 100 nodes. This works very poorly. Now suppose you have 100 nodes, and you have a router that routes intelligently too one of those 100 nodes, e.g. to the one with the smallest request queue. From a theoretical perspective this works really well but it may b…

I understand and don't disagree with everything you are saying, but the focus of my attention is on what you're talking about in your 3rd graf. When you talk about your example problems (1) and (2) with routing to concurrent systems on large number of dynos, what you're really discussing is an engineering flaw in the typical Rails stack.

There's a tradeoff between:

* a well-engineered request handler (a solved problem more than a decade ago) and

* an efficient development environment (arguably a nearly-unsolved problem before the Rails era)

And I feel like mostly the Heroku drama is a result of Rails developers not grokking which end of that tradeoff they've selected.

Re: Answering your questions about Heroku routing and web performance

#73

Post by adamwiggins » Wed, Apr 03, 2013, 7:39 PM UTC

[deleted] wrote: post #64

adamwiggins wrote: post #32

Earlier quoted context omitted.

Harsh. I'm going to ignore the more inflammatory parts of this (feel free to restate if you want me to engage in discussion), but one bit did grab my attention: "A major selling point of Heroku is that scaling wouldn't be a risk" This is interesting, especially the word "risk." Can you expand on this?

[deleted]

Very clarifying, thanks.

Fire-fighting during the scaling phase is a problem that every fast-growing software-as-a-service business will probably have to face. I think Heroku makes it easier, perhaps way easier; but I hope our marketing materials etc have not implied a scaling panacea. Such a thing doesn't exist and is most likely impossible, in my view.

Re: Answering your questions about Heroku routing and web performance

#74

Post by bgentry » Wed, Apr 03, 2013, 7:41 PM UTC

bifrost wrote: post #61

I'd say there's a major flaw here - "the new world of concurrent web backends" - If now was 1995 I might agree with you, but having concurrency in web-app servers is not new. The lack of concurrency in test/demo app situations is totally understandable, in a production environment you'd have to be completely bonkers to think thats ever useful. I also agree that "having to deal with loadbalancing" is something that mo…

I'd say there's a major flaw here - "the new world of concurrent web backends" - If now was 1995 I might agree with you, but having concurrency in web-app servers is not new.

Sadly, concurrency is relatively new and unfamiliar territory to many in the Ruby on Rails community.

Re: Answering your questions about Heroku routing and web performance

#75

Post by nantes » Wed, Apr 03, 2013, 7:45 PM UTC

adamwiggins wrote: post #67

natmaster wrote: post #58

Earlier quoted context omitted.

What about potential customers? I've been evaluating your platform and just be completely frustrated with 3 days wasted trying to solve these performance issues. I'm using gunicorn with python, and if I use the sync worker the request queue easily hits 10 seconds and nothing works; if I switch to gevent or eventlet, new-relic tells me that redis is taking the same time stuck getting a value. This is using the same co…

That sucks. It may or may not be related to how the router works, but it's definitely about performance and visibility, which is what this is all about. Can you email me at adam at heroku dot com and I'll connect you with one of our Python experts? I can't promise we'll solve it, but I'd like to take a look.

I'd be really interested to see some public information resulting from debugging Python apps. We're holding pretty steady, but see a fairly constant stream of timeouts due, apparently, to variance in response times. To be sure, we're working on that. But, in the meantime, our experiments with gevent and Heroku have been less than inspiring.

Re: Answering your questions about Heroku routing and web performance

#76

Post by deedubaya » Wed, Apr 03, 2013, 7:45 PM UTC

Haters gonna hate.

I wonder how many people bitching here are actual customers who are having problems that haven't been address with a solution. I'm guessing that number is low.

Oh, you're a potential customer? That's why you're bitching? About a problem you may or may not have if you actually choose the product? Think about that argument for a second.

I've never seen such a transparent response and follow up as I have from Heroku on this issue. Most other companies would have gone into immediate damage mitigation mode and let the wound heal instead of re-opening it and giving feedback on how to fix the problem as Heroku has.

I applaud the Heroku team for their effort on their platform and being a kickass company.

Re: Answering your questions about Heroku routing and web performance

#77

Post by eaurouge » Wed, Apr 03, 2013, 7:52 PM UTC

adamwiggins wrote: post #30

ebbv wrote: post #24

Earlier quoted context omitted.

I doubt anyone chose Heroku solely because of the routing algorithm, but the sales point of intelligent routing was certainly an appealing one. Developers don't want to pay for abstraction just for abstraction's sake, they want to pay for abstraction of difficult things. Intelligent routing is one of those difficult things. Random routing is easy, which is of course why it's also more reliable, but also why you're se…

Well said. It's interesting — very few customers are actually bent out of shape about this. (A few are, for sure.) It's more non-customers who are watching from the sidelines that seem to be upset. I do want to try to explain ourselves to the community in general, and that's what this post was for. But my first loyalty is to serving our current customers well.

"It's interesting — very few customers are actually bent out of shape about this"

Seems to me you don't get it. Sure there are some very vocal non-customers but you also have a lot of potential customers and users (spinning up free instances) evaluating your product and hoping for a better product. I agree that your true value is the abstraction you provide. Some of these potential customers want to ensure Heroku is as good an abstraction as promised to justify the cost and commitment.

Re: Answering your questions about Heroku routing and web performance

#78

Post by andrewem » Wed, Apr 03, 2013, 7:52 PM UTC

streptomycin wrote: post #53

res0nat0r wrote: post #34

Earlier quoted context omitted.

Because they weren't victims of fraud.

Even if that's true, you're just distracting from the main point. If I buy X from a company and they don't provide it, I expect a refund. Like if I order a laptop from Dell and, oops, it gets lost in the mail, it's not okay to just say "oh, we changed our shipping company, so that should happen less in the future."

Your example is extremely clear-cut, but doesn't match the situation with Heroku very closely. If Heroku promised an intelligent router, but then it was always down and never served you any requests, that would make your Dell example a good analogue.

A closer comparison might be: What if Dell shipped you a laptop that they advertised as having an SSD with really good performance (and they thought that was true, or at least they did when they wrote the ad for the laptop), and it turns out that for your workload the performance isn't so great? Would you expect a refund then?

Re: Answering your questions about Heroku routing and web performance

#79

Post by adamwiggins » Wed, Apr 03, 2013, 7:54 PM UTC

aphyr wrote: post #60

ebbv wrote: post #4

It seems to me that Heroku is still failing to understand (or at least cop to) the fact that the switch from intelligent to randomized routing was a loss of a major reason people chose Heroku in the first place. A lot of Heroku's apparent value came from the intelligent routing feature. Everybody knew that it was harder to implement than random routing, that's why they were willing to pay Heroku for it. Nobody's argu…

"the switch from intelligent to randomized routing" As I understand it, Heroku (on the Bamboo stack) didn't up and decide "Hey, we're gonna switch from intelligent to random routing." The routers are still (individually) intelligent. It's just that there are more of them now, and they were never designed to distribute their internal queue state across the cluster. The system as a whole behaves more and more like a ra…

True, although it's more confusing than this. We didn't make an explicit decision for Bamboo, and thus the big problem — docs fell out of date, at a minimum.

For Cedar, we did make an explicit decision. People on the leading edge of web development were running concurrent backends and long-running requests. Our experimental support for Node.js in 2010 was a big driver here, but also people who wanted to use concurrency in Ruby, like Ilya Grigorick's Goliath and Rails becoming threadsafe. These folks complained that the single-request-per-backend algorithm was actually in their way.

This plus horizontal scaling / reliability demands caused us to make an explicit product decision for Cedar.

Re: Answering your questions about Heroku routing and web performance

#80

Post by davidkatz » Wed, Apr 03, 2013, 7:55 PM UTC

adamwiggins wrote: post #73

[deleted] wrote: post #64

Earlier quoted context omitted.

[deleted]

Very clarifying, thanks. Fire-fighting during the scaling phase is a problem that every fast-growing software-as-a-service business will probably have to face. I think Heroku makes it easier, perhaps way easier; but I hope our marketing materials etc have not implied a scaling panacea. Such a thing doesn't exist and is most likely impossible, in my view.

My company has been building apps for startups for years, and I can confirm that Heroku is consistently perceived as a "I never have to worry about scaling" solution.