Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Answering your questions about Heroku routing and web performance

blog.heroku.com

Post reply on HN
Print view View original

161–168 of 168 posts

Re: Answering your questions about Heroku routing and web performance

#161

Post by adamwiggins »

siong1987 wrote: post #127

Since we are on the question of visibility of Heroku dyno, what is the amount of CPU power each dyno has? What about 2Xdyno?

This is a tough area. If you go look at various types of infrastructure providers (e.g. EC2, Linode, Rackspace) you'll see that they always end up making up vague units of measurement (e.g. "cores") and then showing all the resources in reference to whatever the base unit is. So there's really no good way to talk about CPU power like there is with memory.

That said, I can say that a 1X dyno is not very powerful compared to, say, any server you'd purchase for your own datacenter. Our intention is that 2X dynos will provide twice the CPU horsepower, although CPU and I/O are harder to allocate reliability in virtualized environments.

Re: Answering your questions about Heroku routing and web performance

#162

Post by moe »

adamwiggins wrote: post #154
moe wrote: post #131

Earlier quoted context omitted.

most of our customers tell us that they're not willing to trade lower availability and higher latency per request What's the constraint that prevents you from having your dynos register with the loadbalancer cluster and then having the latter perform leastconn balancing per application? Also why would that mean "lower availability" or "higher latency"? Did you look into zookeeper?

> What's the constraint that prevents you from having your dynos register with the loadbalancer cluster and then having the latter perform leastconn balancing per application? This is how it works. Dynos register their presence into a dyno manager which publishes the results into a feed, and then all the routing nodes subscribe to that feed. But dyno presence is not the rapidly-changing data which is subject to CAP c…

Hm. Until now I thought dyno-presence is your issue, but now I realize you're talking about the actual "leastconn" part, i.e. the requests queueing up on the dynos itself?

If that's what you actually mean then I'd ask: Can't the dynos reject requests when they're busy ("back pressure")?

AFAIK that's the traditional solution to distributing the "leastconn" constraint.

In practice we've implemented this either with the iptables maxconn rule (reject if count >= worker_threads), or by having the server immediately close the connection.

What happens is that when a loadbalancer hits an overloaded dyno the connection is rejected and it immediately retries the request on a different backend.

Consequently the affected request incurs an additional roundtrip per overloaded dyno, but that is normally much less of an issue than queueing up requests on a busy backend (~20ms retry vs potentially a multi-second wait).

PS: Do you seriously consider Zookeeper "really slow"?! http://zookeeper.apache.org/doc/r3.1.2/zookeeperOver.html#Pe...

Re: Answering your questions about Heroku routing and web performance

#163

Post by [deleted] »

adamwiggins wrote: post #159
j-kidd wrote: post #137

From the article: > Q. Did the Bamboo router degrade? > A. Yes. Our older router was built and designed during the early years of Heroku to support the Aspen and later the Bamboo stack. These stacks did not support concurrent backends, and thus the router was designed with a per-app global request queue. This worked as designed originally, but then degraded slowly over the course of the next two years. From Adam's me…

There are two separate issues here, and it's easy to get them confused. One is the slow degradation on Bamboo without any change to the routing algorithm code, and the other was the explicit product choice for Cedar with a different code path in the router. Both are described fully here: https://blog.heroku.com/archives/2013/2/16/routing_performan... The reason it's easy to confuse these two is also part of what conf…

[deleted]

Re: Answering your questions about Heroku routing and web performance

#164

Post by j-kidd »

adamwiggins wrote: post #159
j-kidd wrote: post #137

From the article: > Q. Did the Bamboo router degrade? > A. Yes. Our older router was built and designed during the early years of Heroku to support the Aspen and later the Bamboo stack. These stacks did not support concurrent backends, and thus the router was designed with a per-app global request queue. This worked as designed originally, but then degraded slowly over the course of the next two years. From Adam's me…

There are two separate issues here, and it's easy to get them confused. One is the slow degradation on Bamboo without any change to the routing algorithm code, and the other was the explicit product choice for Cedar with a different code path in the router. Both are described fully here: https://blog.heroku.com/archives/2013/2/16/routing_performan... The reason it's easy to confuse these two is also part of what conf…

Sorry about the accusation. I read the Bamboo's issue wrongly. The article from Feb 2013 seems to imply that the slow degradation happened from 2011 to 2013. It starts with "Over the past couple of years", I guess that's what got me confused. The FAQ clarifies that the slow degradation happened from 2009 to 2011.

Re: Answering your questions about Heroku routing and web performance

#165

Post by stickfigure »

Cabal wrote: post #148
stickfigure wrote: post #134

Earlier quoted context omitted.

I'm a real customer with real problems. Grep this page for latchkey's description of it. The funny thing is, I don't have much sympathy for Rails users. Scaling problems with a single-threaded, serial request-processing architecture? No surprise there. But we have inexplicable H12 problems with Node.js. There's something broken in the system and it isn't random routing.

You're talking like Node offers real concurrency. It doesn't.

There's nothing wrong with Node's concurrency. Our app, like most webapps, is I/O bound. Any individual instance should be able to handle thousands of concurrent requests as long as they are all blocked on I/O.

Being able to process more than one concurrent request (as Node can) is "real concurrency". Java-style native threading is a step above and beyond this, and unnecessary for most web applications.

Re: Answering your questions about Heroku routing and web performance

#166

Post by davidkatz »

adamwiggins wrote: post #91
davidkatz wrote: post #80

Earlier quoted context omitted.

My company has been building apps for startups for years, and I can confirm that Heroku is consistently perceived as a "I never have to worry about scaling" solution.

Very useful observation. I'd love to figure out how we can better communicate that while we aim to make scaling fast and easy, "you never have to worry about scaling" is much too absolute.

Your marketing materials clearly use phrases like "forget about servers", "easily scale to millions of users", "scale effortlessly" and so forth. You're making it very easy to misunderstand you.

Re: Answering your questions about Heroku routing and web performance

#167

Post by omfg »

latchkey wrote: post #119
omfg wrote: post #118

Earlier quoted context omitted.

Have you checked out Elastic Beanstalk yet? http://aws.amazon.com/elasticbeanstalk/ Any thoughts on that. It offers a Heroku'esque deploy.

EBS just recently added support for Node. It's definitely on the list as the next PaaS to try out. AppFog is another one that we've done a JVM deployment to and like a lot except for the fact that it just feels very alpha quality. Their website is painfully slow and under documented and the 'af' command isn't nearly as cool as just doing git deploy remote master. I should also add that one thing that Heroku did get 1…

Oh wow, never knew about that. That's very helpful thanks.

Re: Answering your questions about Heroku routing and web performance

#168

Post by jholman »

adamwiggins wrote: post #155
jholman wrote: post #124

I'd like to start by acknowledging that I'm one of the "non-customers who are watching from the sidelines". I think Adam's right that this is an important distinction. Adam, there's something that confuses me about this. I'm no expert in routing theory, nor have I done the experiments, so forgive me if my reasoning misses something. I understand why RapGenius took you up on your original promises of "intelligent rout…

> It seems to me that if you split into two (or more) tiers, and random-load-balance in the front tier (hit first by the customer), and then at the second tier only send requests to unloaded clients [...] I'm unclear how you'd think introducing a second tier changes things. That tier would need to track dyno availability and then you're right back to the same distributed state problem. Perhaps you mean if the second…

Sorry, yes, I'm supposing that the second tier serves fewer dynos; sufficiently few that your solutions from 2009 (that motivated you to advertise intelligent routing in the first place) are still usable.

> As a rule of thumb, I find it's best to avoid one-shots (or "specials").

Absolutely, and I would never suggest that. However, it's not just RG that has this problem, right? If I understand correctly, isn't it every single customer who believed your advertising and followed your suggested strategy to use single-threaded Rails, and doesn't want to switch?

So it's not about short or medium term; it's about letting customers take the latency hit (as you note), in order to get the scaling properties that they already paid for.

Post reply on HN