Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

rapgenius.com

Post reply on HN
Print view View original

331–340 of 437 posts

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#331

Post by sylvinus »

avodonosov wrote: post #279
teich wrote: post #263

This is Oren Teich, I run Heroku. I've read through the OP, and all of the comments here. Our job at Heroku is to make you successful and we want every single customer to feel that Heroku is transparent and responsive. Getting to the bottom of this situation and giving you a clear understanding of what we’re going to do to make it right is our top priority. I am committing to the community to provide more information…

I hope the solution will not break the possibility for multithreaded apps to receive several requests

Me too. I see this as a side-effect of Rails single-threaded craziness, our "modern" Node.js apps run faster than ever.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#332

Post by Djehngo »

benihana wrote: post #94

I really like Rap Genius, but I wish they would tone down the blackness of the background. Reading #CCC text on #000 background makes my eyes bug out.

For this blog-post where the annotations aren't such a big deal the readability bookmarklet[1] works quite well.

[1] http://readability.com/bookmarklets

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#334

Post by toyg »

doktrin wrote: post #306
teich wrote: post #263

This is Oren Teich, I run Heroku. I've read through the OP, and all of the comments here. Our job at Heroku is to make you successful and we want every single customer to feel that Heroku is transparent and responsive. Getting to the bottom of this situation and giving you a clear understanding of what we’re going to do to make it right is our top priority. I am committing to the community to provide more information…

Thanks for the response, but I have to admit that the lack of a clear-cut answer here is a little worrisome. Anyone who wants to like Heroku would hope that the OP is flat out, 100%, wrong. The fact that Heroku's official answer requires a bit of managing implies otherwise. On a related tangent, I would also encourage future public statements to be a little less opaque than some Heroku has put out previously. For ins…

Maybe it doesn't need "managing", Oren might just want to talk with whoever was responsible for the change and see what the best way forward is. I don't think panicked, knee-jerk reactions like "OMG we were wrong and will revert that commit pronto!" are beneficial in situations as complex as this.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#335

Post by ajanuary »

DeepDuh wrote: post #234
DigitalJack wrote: post #219

Earlier quoted context omitted.

I'm sure it's because the idea of heroku is so great.

As others here have pointed out in different discussions: Ideas aren't worth much by themselves. It's the subset that's implementable of which it's the subset that has gotten implemented that's worth anything. If we just love ideas we should better become philosophers. Sorry to be a bit harsh, but I find it a bit shocking how even in this field where we can basically play god and do whatever we want and what we think…

They want to love it because the idea is great. They don't love it because it doesn't deliver. Sounds like you both agree.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#336

Post by vidarh »

themgt wrote: post #274
pdenya wrote: post #272

Earlier quoted context omitted.

We're discussing Rails on Heroku specifically which, non-unicorn, should be a "next available checkout counter" situation. Ideally it should be possible to make this an optional behavior that you can choose to turn on for Rails apps.

I agree there should be a better way - it's just important to understand than Rails doesn't get any special treatment on a PaaS done correctly, so it's important to come up with a generic solution. I think part of the solution would be customizable option(i.e.. how many requests can each dyno handle simultaneously), probably combined with intelligently monitoring/balancing proxy load so new requests always go to the…

Rails doesn't need to be treated specially. All that is needed is a "maximum number of simultaneous connections to pass to this backend" setting coupled with load balancing by available slots rather than purely randomly.

The issue here isn't that Rails needs to be treated specially - this problem applies to various extent in any type of backend where some types of requests might turn out to be computationally heavy or require lots of IO. You can't magic away that: A request that takes 8 CPU seconds will take 8 CPU seconds. If you start piling more requests onto that server, response times will increase, even if some will keep responding, and if another 8 CPU second request hits too soon, chances increase that a third one will, and a fourth, and before you know it you might have a pileup where available resources for new requests on a specific instance are rapidly diminishing and response times shoot through the roof.

Pure random distribution is horrible for that reason pretty much regardless.

Now, doing "intelligent" routing is a lot easier for servers with some concurrency, as you can "just" have check requests and measure latency for the response and pick servers based on current low latency and get 90% there and that will be enough for most applications. Sure, the lower the concurrency, the more you risk having multiple heavy queries hit the same server and slow things down, and this request grows dramatically with the number of load balancers randomly receiving inbound requests to pass on, but at least you escape the total pileup more often.

But that's also a clue to one possible approach for non-concurrent servers: group them into buckets handled by a single active load balancer at a time and have front ends that identifies the right second layer load balancers. Shared state is now reduced to having the front end load balancers know which second layer load balancers are the currently active ones for each type of backend. It costs you an extra load balancer layer with according overhead. But don't you think OP would prefer an extra 10ms per request over the behaviour he's seen?

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#337

Post by jhuckestein »

Watch out, this affects small rails applications with few dynos as well.

If you hit the wall with one dyno and add another one, you won't get twice the throughput even though you pay twice the price.

I've always had suspicions about this on some smaller apps but never really looked into it. You can configure New Relic to measure round-trip response times on the client side. At peak loads those would be unreasonably high. Much higher than can be explained by huge latencies even.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#338

Post by skrebbel »

themgt wrote: post #313
skrebbel wrote: post #299

Earlier quoted context omitted.

That's like saying Craigslist did it correctly and AirBnB didn't because AirBnB is only tailoring to a specific segment of the world's supply and demand market. Rails is very widely used. How can you consider that an edge case?

It's like saying EC2 should tailor its virtualization to Fedora 16, or Mac OS X should tailor its windowing system to Photoshop CS4, or Apache should tailor mod_proxy to Joomla. There may be specific attributes of popular applications that need to be adapted to, but those adaptations need to be built in a generic way and exposed through a standard API. Since even many Rails apps now do not follow a single threaded re…

> or Mac OS X should tailor its windowing system to Photoshop CS4

I'm taking this example out specifically, because it illustrates my point quite nicely.

If there were a sizeable community of people who only wanted to use a computer for Photoshop, and tailoring the windowing system to them made it a significant usability improvement for those people, then it would be a completely imaginable situation that upon first opening your brand new Mac, it'd ask you whether you're one of those Photoshop people and want the special windowing system setup.

Well, ok, haha Apple and customizing anything for anyone, ever. But many other vendors might make such a choice.

The apparent stubborn refusal of many PaaS services, including Heroku and yours, to particularly tailor to a very common configuration of Rails sounds like a hole in the market, to me. As a customer, I don't care whether this is "incorrect" because Rails does not conform to some yet-to-be-defined standard or simply because the PaaS doesn't have their shit together. The customer experience is the same: I'm running a blog tutorial app, and the performance sucks.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#339

Post by antihero »

arcatek wrote: post #153
chc wrote: post #115

Earlier quoted context omitted.

> But really, throwing in the towel at intelligent routing and replacing it with "random routing" is horrific, if true. The thing is, their old "intelligent routing" was really just "we will only route one request at a time to a dyno." In other words, what changed is that they now allow dynos to serve multiple requests at a time. When you put it that way, it doesn't sound as horrific, does it?

The requests will not be served at the same time , that's the whole point. If a request is routed to a busy dyno, you will have to wait that the previous job finish before being able to start yours.

Thing is though, it says that a Dyno is a Ubuntu virtual machine. In what sort of horrendous configuration can an ENTIRE VM serve only a SINGLE REQUEST AT A TIME?!

That is utter madness, and the validity of the argument depends on whether it's the Heroku or this dude's fault that the VM is serving only a single request at a time (and it taking >1sec to handle a request).

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#340

Post by antihero »

teich wrote: post #263

This is Oren Teich, I run Heroku. I've read through the OP, and all of the comments here. Our job at Heroku is to make you successful and we want every single customer to feel that Heroku is transparent and responsive. Getting to the bottom of this situation and giving you a clear understanding of what we’re going to do to make it right is our top priority. I am committing to the community to provide more information…

Can "Dynos" serve multiple requests simultaneously? That's the question, really.
Post reply on HN