Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

rapgenius.com

Post reply on HN
Print view View original

61–70 of 437 posts

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#61

Post by joevandyk »

pvh wrote: post #44
bifrost wrote: post #2

I am only going to suggest a small edit -> s/Postgres can’t/Heroku's Postgres can't/ PG can scale up pretty well on a single box, but scaling PG on AWS can be problematic due to the disk io issue, so I suspect they just don't do it. I'd love to be corrected :)

Data Dep't here. Postgres scales great on Heroku, AWS, and in general. We've got users doing many thousands of query per second, and terabytes of data. Not a problem. The issue with the number of connections is that each connection creates a process on the server. We cap the connections at 500, because at that point you start to see problems with O(n^2) data structures in the Postgres internals that start to make all…

Each connection forks a new process on the server, not a thread.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#62

Post by jhspaybar »

aristus wrote: post #24
mattj wrote: post #12

So the issue here is two-fold: - It's very hard to do 'intelligent routing' at scale. - Random routing plays poorly with request times with a really bad tail (median is 50ms, 99th is 3 seconds) The solution here is to figure out why your 99th is 3 seconds. Once you solve that, randomized routing won't hurt you anymore. You hit this exact same problem in a non-preemptive multi-tasking system (like gevent or golang).

I do perf work at Facebook, and over time I've become more and more convinced that the most crucial metric is the width of the latency histogram. Narrowing your latency band --even if it makes the average case worse -- makes so many systems problems better (top of the list: load balancing) it's not even funny.

I can chime in here that I have had similar experiences at another large scale place :). Some requests would take a second or more to complete with the vast majority finishing in under 100MS. A solution was put in place that added about 5 MS to the average request, but also crushed the long tail(it just doesn't even exist anymore) and everything is hugely more stable and responsive.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#63

Post by gleb »

pvh wrote: post #44
bifrost wrote: post #2

I am only going to suggest a small edit -> s/Postgres can’t/Heroku's Postgres can't/ PG can scale up pretty well on a single box, but scaling PG on AWS can be problematic due to the disk io issue, so I suspect they just don't do it. I'd love to be corrected :)

Data Dep't here. Postgres scales great on Heroku, AWS, and in general. We've got users doing many thousands of query per second, and terabytes of data. Not a problem. The issue with the number of connections is that each connection creates a process on the server. We cap the connections at 500, because at that point you start to see problems with O(n^2) data structures in the Postgres internals that start to make all…

Shouldn't this be s/thread/process/ ?

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#64

Post by fatbird »

joevandyk wrote: post #55
fatbird wrote: post #45

Maybe this is a dumb question, but wouldn't straightforward Round Robin routing by Heroku restore their "one dyno = one more concurrent request" promise without incurring the scaling liabilities of tracking load across an arbitrarily large number of dynos?

Nope, requests could still get queued behind a dyno that's busy with a long request.

Sure, but the real issue the article identifies is that, under random routing, they need to keep doubling the number of dynos to halve the odds of bad queueing, which leads to absurd factor of 50 requirements to get back to what they had before. With round robin, the increase should be much more linear.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#67

Post by kawsper »

anon640 wrote: post #52

"For a Rails app, each dyno is capable of serving one request at a time." Is this a deliberate design choice on Heroku's part, or is this just how Ruby and Rails work? It sounds bizarre that you would need multiple virtual OS instances just to serve multiple requests at the same time. What are the advantages of this over standard server fork()/threaded accept designs?

It is how Rails server behaves in itself, but that is also how Heroku tells you to do it.

Rails can be served with Unicorn ( http://unicorn.bogomips.org/ ) which is a forking app-server.

I do believe there was a trick a while back where you could get Heroku to run a Unicorn process on a dyno to get more requests out of it. The process is described here: http://blog.codeship.io/2012/05/06/Unicorn-on-Heroku.html

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#70

Post by chc »

OK, maybe I'm missing something here, but it seems to me that the OP's real problem is that he's artificially limiting himself to one request per dyno. They now allow a dyno to serve more than one request at a time, and he's presenting that as a bad thing! It seems to me that the answer to Rap Genius' problems is not "rage at Heroku," but rather "gem 'unicorn'".
Post reply on HN