Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

rapgenius.com

Post reply on HN
Print view View original

351–360 of 437 posts

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#351

Post by jonnycat »

This might be the case "out of the box", but it's very simple to go multithreaded on the Cedar stack and avoid this issue (provided that your app is threadsafe, of course).

You can do this pretty easily with a Procfile and thin: bundle exec thin -p $PORT -e $RACK_ENV --threaded start

And then config.threadsafe! in the app

Regarding Rails app threadsafety, there are some gotchas around class-level configuration and certain gems, but by and large these issues are easily manageable if you watch out for them during app development.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#352

Post by vicks711 »

humbledrone wrote: post #318
vicks711 wrote: post #287

Earlier quoted context omitted.

I cant stop myself from saying this. You wrote all this instead of doing what?

Normally I would just downvote you and move on, but in this case your comment is frustrating enough that I have to say something. I found the comment you responded to (by nsrivast) quite fascinating. A well-written but brief analysis of the problem, with sources attached for further reading -- what's not to like? In-depth and thoughtful comments like that are what keep me coming back to this site, and are what make t…

really?

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#353

Post by mononcqc »

vidarh wrote: post #336
themgt wrote: post #274

Earlier quoted context omitted.

I agree there should be a better way - it's just important to understand than Rails doesn't get any special treatment on a PaaS done correctly, so it's important to come up with a generic solution. I think part of the solution would be customizable option(i.e.. how many requests can each dyno handle simultaneously), probably combined with intelligently monitoring/balancing proxy load so new requests always go to the…

Rails doesn't need to be treated specially. All that is needed is a "maximum number of simultaneous connections to pass to this backend" setting coupled with load balancing by available slots rather than purely randomly. The issue here isn't that Rails needs to be treated specially - this problem applies to various extent in any type of backend where some types of requests might turn out to be computationally heavy o…

I'm sure OP could prefer the extra 10ms, but then everyone else who can deal with random dispatching right now has to pay a 10ms penalty because OP built his stuff on a technology that can deal with only one request at a time on a server, which boggles the mind to begin with.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#354

Post by drostie »

nsrivast wrote: post #262

OP is a friend of mine, and when I first heard of his problem I wondered if there might be an analytical solution to quantify the difference between intelligent vs naive routing. I took this problem as an opportunity to teach myself a bit of Queueing Theory[1], which is a fascinating topic! I'm still very much a beginner, so bear with me and I'd love to get any feedback or suggestions for further study. For this exam…

You are certainly right that it becomes an M/G/1 model and thus that M/M/1 and M/D/1 will give reasonable approximations. The reason that it's an M/?/1 model is partly what you're saying, but also partly because the random assignment of the incoming requests acts as a random filter on a Poisson process. Poisson processes are just defined by not having history, and the random filter is not history-dependent either, so the output from the filter -- the input to any node -- is still a Poisson process.

What's interesting to me here is: Suppose rather than doing this with a random variable, you do it with a summing filter, a simple counter:

    on request r: 
        node[n].handle(r)
        n = (n + 1) % node.length
That's a really simple solution but it should tend to average out the Poisson input, which gives you something closer to a D/G/1 problem.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#355

Post by neilmiddleton »

antihero wrote: post #340
teich wrote: post #263

This is Oren Teich, I run Heroku. I've read through the OP, and all of the comments here. Our job at Heroku is to make you successful and we want every single customer to feel that Heroku is transparent and responsive. Getting to the bottom of this situation and giving you a clear understanding of what we’re going to do to make it right is our top priority. I am committing to the community to provide more information…

Can "Dynos" serve multiple requests simultaneously? That's the question, really.

That's up to the process you have running on the dyno

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#357

Post by Giszmo »

I searched for variance and apparently nobody mentioned this before: They talk of Mean request time: 306ms Median request time: 46ms Which indicates a very high variance, so don't take for granted that an x50 increase of performance would result from intelligent routing. The problem is that the fast tasks suffer from being queued after the slow tasks, so each such fast task takes an extra latency. If the variance is lower, the random routing will be favorable at some point as the delay of getting the task from the router queue to the dyno is not zero neither. In the case of no variance, "intelligent routing" would always add that delay as soon as all dynos are at their limit. Before that, the router would simply keep a list of idle dynos and send work there without delay.

Sure if you never hit 100% load, intelligent routing is cheap and comes at no delay. Imagine 40ms jobs getting all dynos to 100% load. Now the dynos would be idle for the duration of the ping that it takes to report being idle. let that be 4ms. That is 10% less throughput than with items queuing up on the dyno.

The router being the bottleneck would therefore justify to make it stateless and give the dynos a chance to use these last 10% of processing power as well, ultimately increasing the throughput by 10%. Sure, a serious project would not run its servers at 120% load hoping to eventually get back to 100% within time, so all this being said I would always favor intelligent routing to get responsive servers, add dynos in rush hours and only opt for dyno-queuing for stuff that may come with a delay (scientific number crunching, …)

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#358

Post by miloshadzic »

NatW wrote: post #273
ylansegal wrote: post #247

Earlier quoted context omitted.

I did some performance analysis on puma vs unicorn vs thin a while ago: http://ylan.segal-family.com/blog/2012/08/20/better-performa... Although as noted in the comments, I neglected to run threadsafe! and should have probably tried rubinius or jruby. I have been meaning to redo. Take with a grain of salt

You seem have written one of the very few articles I've seen benchmarking this. I'd love to know more about how Puma compares to Unicorn (especially unicorn configurations mentioned by some in this conversation) and Thin for serving rails on Heroku. Many of the Unicorners pushing their solution don't appear to be aware of Puma and its potential benefits. I'm curious if Puma with MRI has benefits, too. Thanks!

AFAIK Puma should have a lower memory footprint on MRI than Thin but I haven't done benchmarks myself.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#359

Post by toyg »

doktrin wrote: post #341
toyg wrote: post #334

Earlier quoted context omitted.

Maybe it doesn't need "managing", Oren might just want to talk with whoever was responsible for the change and see what the best way forward is. I don't think panicked, knee-jerk reactions like "OMG we were wrong and will revert that commit pronto!" are beneficial in situations as complex as this.

You're assuming that the change was actually made. Until we hear definitively from Heroku, the only evidence is an (admittedly, well documented) blog post.

Yeah, absolutely. I'm just saying we can't expect a to manager to immediately respond to a highly-technical issue questioning subtle changes in internal behaviour which might have been introduced years ago.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#360

Post by badgar »

aneth4 wrote: post #305
badgar wrote: post #144

Earlier quoted context omitted.

> That's one reason customers are throwing money at you to solve it, Heroku. People are throwing money at Heroku because it's really easy to use, not because it's the best long-term technology choice. Seriously - what percentage of Heroku paying users do you think actually read up on the finest technical details like routing algorithms before they put in their credit card? Heroku knows. They know you can't even build…

So heroku doesn't want these $20,000/mo accounts, just technically understaffed startups paying $1000/mo? I think heroku does want to be a long-term technology choice.

> So heroku doesn't want these $20,000/mo accounts, just technically understaffed startups paying $1000/mo?

> I think heroku does want to be a long-term technology choice.

Oh, I'm sure Heroku wants to be a long-term technology choice. That doesn't mean they're trying to be one with their current product offerings.

Consider their product choices since launch: they've added dozens of small value-added technology integrations. Features for a few bucks a month like New Relic to upsell their smallest customers. The price drop was also a big move to reduce barriers to using their platform - which also targets smaller customers. They launched Clojure as their third supported language! Meanwhile, they're singly-homed and have had several protracted outages, and have no announced plans to build a multi-homed product. Scalability has gotten worse with this random routing development.

I think Heroku has known for a long time that they don't have a long-term platform product and that they can't keep big accounts until they build one.

Post reply on HN