Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Answering your questions about Heroku routing and web performance

blog.heroku.com

Post reply on HN
Print view View original

141–150 of 168 posts

Re: Answering your questions about Heroku routing and web performance

#141

Post by pytrin »

latchkey wrote: post #93

I am working for a fairly large heroku app running Node on ~50-100 web dyno's with another 20-50 backends. Here are the problems as I see it: We get H12's all the time. Randomly. The only suggestion we get from Heroku is to make the requests process as fast as possible. Thus, we've spent considerable amount of time going through everything we can possibly do to make all requests respond as fast as possible. I've give…

With that kind of money invested in hosting and scaling, why not get a dedicated professional to handle devops on your team, and go with more traditional hosting solutions? I'm interested in hearing why people still use Heroku at the scale you're describing

Re: Answering your questions about Heroku routing and web performance

#142

Post by latchkey »

pytrin wrote: post #141
latchkey wrote: post #93

I am working for a fairly large heroku app running Node on ~50-100 web dyno's with another 20-50 backends. Here are the problems as I see it: We get H12's all the time. Randomly. The only suggestion we get from Heroku is to make the requests process as fast as possible. Thus, we've spent considerable amount of time going through everything we can possibly do to make all requests respond as fast as possible. I've give…

With that kind of money invested in hosting and scaling, why not get a dedicated professional to handle devops on your team, and go with more traditional hosting solutions? I'm interested in hearing why people still use Heroku at the scale you're describing

That's a really good question. Heroku is offering to handle the hosting, devops and scaling issues for us so that we can focus our energy on building a killer product. When considering the costs of hiring a devops team and someone to wear a pager 24/7 when servers go down, paying the premium for using Heroku becomes a lot more attractive.

'why not get a dedicated professional to handle devops on your team'

It sure is easy to write that, but the reality isn't as rosy. I've gone through the process at two companies to try to do that, interviewing ~50-80 people and it has been a nightmare. It is really really difficult to find quality devops people. Again, this makes PaaS like AWS, Heroku and AppEngine a lot more attractive. They are betting their entire business on being able to hire good devops people, so they tend to attract better talent.

Re: Answering your questions about Heroku routing and web performance

#143

Post by peterwoo »

latchkey wrote: post #125
adamwiggins wrote: post #121

Earlier quoted context omitted.

This comes back to visibility: knowing where the problem lies (especially when you're using a variety of add-on services or calling external APIs) and being able to understand what's happening, or what happened in retrospect. Visibility is hard no matter where you run your app. But this is an area where Heroku can get a lot better, and we intend to.

Visibility is one part of the problem. 50k requests/min is only ~833/s. The reality is that a single dyno should be able to more than handle that sort of load, especially if it is a simple app. People are doing 10k connections on a single laptop, 833s should be a piece of cake. So, yes, visibility is a big issue here because you have no idea if you need 10, 11, 12 or 20 dyno's to serve 50k requests/min. You just gues…

How could one single-threaded dino serve more than 833 requests / sec?

Re: Answering your questions about Heroku routing and web performance

#144

Post by latchkey »

peterwoo wrote: post #143
latchkey wrote: post #125

Earlier quoted context omitted.

Visibility is one part of the problem. 50k requests/min is only ~833/s. The reality is that a single dyno should be able to more than handle that sort of load, especially if it is a simple app. People are doing 10k connections on a single laptop, 833s should be a piece of cake. So, yes, visibility is a big issue here because you have no idea if you need 10, 11, 12 or 20 dyno's to serve 50k requests/min. You just gues…

How could one single-threaded dino serve more than 833 requests / sec?

Where do you get that dyno's are single threaded? Please read: https://devcenter.heroku.com/articles/dynos#dynos-and-reques...

Re: Answering your questions about Heroku routing and web performance

#145

Post by peterwoo »

latchkey wrote: post #144
peterwoo wrote: post #143

Earlier quoted context omitted.

How could one single-threaded dino serve more than 833 requests / sec?

Where do you get that dyno's are single threaded? Please read: https://devcenter.heroku.com/articles/dynos#dynos-and-reques...

500-800req/sec over 10-20 servers with ~30ms response times. 1 req / server seemed plausible.

Thanks. I admit I'm not familiar with the platform.

Re: Answering your questions about Heroku routing and web performance

#146

Post by jules »

tptacek wrote: post #130
jules wrote: post #128

Earlier quoted context omitted.

I certainly already agreed that Rails' architecture is bad (though the reason that it has this problem is its memory usage, and not any of the other reasons you mention). Herokus architecture is bad as well. It's the combination of these that causes the problem. But that does not mean that it's impossible, or even hard, to solve the problem at Herokus end. > I'm not sure there are such providers, and if there aren't,…

I understand that one approach to dispatching requests at the load balancer is superior to the other, just as I understand that one way of absorbing requests at the app server is better than the other. Most things are inferior to other substitutable things! :)

That's a mild way of putting it. With the current way of dispatching requests you need exponentially many servers to handle the same load at the same queuing time, if your application uses too much memory to run multiple instances concurrently on a single server.

Re: Answering your questions about Heroku routing and web performance

#147

Post by jules »

friism wrote: post #122
jules wrote: post #115

Earlier quoted context omitted.

I'm not sure I agree. Yes, it's a Rails problem that it is using large amounts of memory (on the other hand (2) isn't Rails specific at all, it applies equally to e.g. Node). But it's a Heroku problem that it gives Dynos just 512MB of memory. It's a Heroku problem that it doesn't have a good load balancer. Heroku is in the business of providing painless app hosting, and part of that is painless request routing. These…

I work at Heroku. To address you concerns about memory limitations, know that we're fast-tracking 2X dynos (this is also mentioned in the FAQ blog post). Extra memory will make it easier to get more concurrency out of each dyno.

Yes, that will be a huge improvement!

Re: Answering your questions about Heroku routing and web performance

#148

Post by Cabal »

stickfigure wrote: post #134
deedubaya wrote: post #76

Haters gonna hate. I wonder how many people bitching here are actual customers who are having problems that haven't been address with a solution. I'm guessing that number is low. Oh, you're a potential customer? That's why you're bitching? About a problem you may or may not have if you actually choose the product? Think about that argument for a second. I've never seen such a transparent response and follow up as I h…

I'm a real customer with real problems. Grep this page for latchkey's description of it. The funny thing is, I don't have much sympathy for Rails users. Scaling problems with a single-threaded, serial request-processing architecture? No surprise there. But we have inexplicable H12 problems with Node.js. There's something broken in the system and it isn't random routing.

You're talking like Node offers real concurrency. It doesn't.

Re: Answering your questions about Heroku routing and web performance

#149

Post by smackfu »

dangrossman wrote: post #47
res0nat0r wrote: post #34

Earlier quoted context omitted.

Because they weren't victims of fraud.

Is promising one service but delivering another not fraud? Promises from the Heroku website pre-Rap Genius posts: "Incoming web traffic is automatically routed to web dynos, with intelligent distribution of load instantly" "Intelligent routing: The routing mesh tracks the availability of each dyno and balances load accordingly. Requests are routed to a dyno only once it becomes available. If a dyno is tied up due to…

Seems like false advertising at least.

Re: Answering your questions about Heroku routing and web performance

#150

Post by twic »

marshray wrote: post #139
adamwiggins wrote: post #88

Earlier quoted context omitted.

Oh, got it. How's this: In the early days, Heroku only had a single routing node that sat out front. So it wasn't a distributed systems problem at that point. You could argue that Heroku circa 2009 was more of a prototype or a toy than a scalable piece of infrastructure. You couldn't run background workers, or large databases. We weren't even charging money yet. Implementing a single global queue in a single node is…

> So to answer your question, there was never some crazy-awesome implementation of a distributed global queue that we got rid of. So it was an oversimplified system that worked great but wasn't scalable and was at some point going to completely fall over under increasing load. IMExp, this is not a wrong thing to build initially and it's not wrong to replace it either. But the replacement is going to have a hard time…

Not a wrong thing to build initially, but perhaps a wrong thing to advertise a feature based on, unless you have a plan for how to continue to deliver that feature as you scale up.
Post reply on HN