Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

rapgenius.com

Post reply on HN
Print view View original

121–130 of 437 posts

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#121

Post by zeeg »

If this is such a problem for you, why are you still on Heroku? It's not a be-all end-all solution.

I got started on Heroku for a project, and I also ran into limitations of the platform. I think it can work for some types of projects, but it's really not that expensive to host 15m uniques/month on your own hardware. You can do just about anything on Heroku, but as your organization and company grow it makes sense to do what's right for the product, and not necessarily whats easy anymore.

FYI I wrote up several posts about it, though my reasons were different (and my use-case is quite a bit different from a traditional app):

* http://justcramer.com/2012/06/02/the-cloud-is-not-for-you/

* http://justcramer.com/2012/08/30/how-noops-works-for-sentry/

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#123

Post by jaggederest »

toast76 wrote: post #31

Wow. This is explains a lot. We've always been of the opinion that queues were happening on the router, not on the dyno. We consistently see performance problems that, whilst we could tie down to a particular user request (file uploads for example, now moved to S3 direct), we could never figure out why this would result in queuing requests given Heroku's advertised "intelligent routing". We mistakenly thought the occ…

It's interesting, because initially the way that queue time detection worked within New Relic was via timestamps.

Currently, though, I believe it's just fed as a number of milliseconds: https://github.com/newrelic/rpm/blame/master/lib/new_relic/a...

This solves the issue of the application seeing out-of-whack queue times if there's clock skew between the front-end routing framework and the actual dyno box, but misses all the queued time spent in the dyno-queue per rap genius's post.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#124

Post by kawsper »

anon640 wrote: post #109
kawsper wrote: post #67

Earlier quoted context omitted.

It is how Rails server behaves in itself, but that is also how Heroku tells you to do it. Rails can be served with Unicorn ( http://unicorn.bogomips.org/ ) which is a forking app-server. I do believe there was a trick a while back where you could get Heroku to run a Unicorn process on a dyno to get more requests out of it. The process is described here: http://blog.codeship.io/2012/05/06/Unicorn-on-Heroku.html

Isn't this kind of a step backwards? Is Rails really that great that people are willing endure these kinds of limitations just to use it?

Rails server is mostly for development mode. When deployed I think most people use either Unicorn, or throw their applications on JRuby that runs on the JVM with some kind of appserver.

JRuby have this advantage of being multithreaded, so you can parallelize within a single process, and don't rely on forking. Stock Ruby with MRI have a GIL, and as far as I know only runs on one core.

The limitations of stock Ruby is being worked on, but there is still a long way.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#125

Post by barmstrong »

We were very surprised to discover Heroku no longer has a global request queue, and spent a good bit of time debugging performance issues to find this was the culprit.

Heroku is a great company, and I imagine there was some technical reason they did it (not an evil plot to make more money). But not having a global request queue (or "intelligent routing") definitely makes their platform less useful. Moving to Unicorn helped a bit in the short term, but is not a complete solution.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#126

Post by awj »

anon640 wrote: post #52

"For a Rails app, each dyno is capable of serving one request at a time." Is this a deliberate design choice on Heroku's part, or is this just how Ruby and Rails work? It sounds bizarre that you would need multiple virtual OS instances just to serve multiple requests at the same time. What are the advantages of this over standard server fork()/threaded accept designs?

Rails is commonly run as one or more application servers behind an http server that proxies requests to them. Rails itself doesn't manage threads or forked processes for accepting requests, so the only way it fits into Heroku's dyno model is as an app server per dyno.

> What are the advantages of this over standard server fork()/threaded accept designs?

It's simple to build and manage in that you don't have to worry about thread safety and can use the already built and tested proxy capabilities of existing web servers to distribute traffic.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#127

Post by jemfinch »

zenazn wrote: post #13

Randomized routing isn't all bad. In fact, if Heroku were to switch from purely random routing to minimum-of-two random routing, they'd perform asymptotically better [1]. [1]: http://www.eecs.harvard.edu/~michaelm/postscripts/mythesis.p...

If Heroku had the data needed to do minimum-of-two random routing, they'd have the data needed to do intelligent routing. The problem is not the algorithm itself: "decrement and reheap" isn't going to be a performance bottleneck. The problem is tracking the number of requests queued on the dyno.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#128

Post by richcollins »

kennystone wrote: post #95
nthj wrote: post #71

I'm inclined to wait until Heroku weighs in to render judgement. Specifically, because their argument depends on this premise: > But elsewhere in their current docs, they make the same old statement loud and clear: > The heroku.com stack only supports single threaded requests. Even if your applicaExplaintion were to fork and support handling multiple requests at once, the routing mesh will never serve more than a sin…

If you have 2 unicorn servers and you happen to get 3 slow requests routed to it, you are still screwed, right? Seems to me like it will still queue on that dyno.

how does it decide to queue at the dyno level anyway? Does it check for connection refusal at the TCP level?

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#129

Post by richcollins »

wwarnerandrew wrote: post #105
nthj wrote: post #71

I'm inclined to wait until Heroku weighs in to render judgement. Specifically, because their argument depends on this premise: > But elsewhere in their current docs, they make the same old statement loud and clear: > The heroku.com stack only supports single threaded requests. Even if your applicaExplaintion were to fork and support handling multiple requests at once, the routing mesh will never serve more than a sin…

Yes, it's true that the Cedar stack supports forking web servers like unicorn, and that an individual dyno can run multiple workers and therefore serve multiple requests at the same time. However, dumb routing is still very problematic – even if your dyno can work on two requests simultaneously it's still bad for it to get sent a third request when there are other open dynos. Also, for apps with a large-ish memory fo…

Right if the apps internal queue is full and it stops accepting connections I'm assuming it will still queue at the dyno level anyway.
Post reply on HN