Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

rapgenius.com

Post reply on HN
Print view View original

341–350 of 437 posts

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#341

Post by doktrin »

toyg wrote: post #334
doktrin wrote: post #306

Earlier quoted context omitted.

Thanks for the response, but I have to admit that the lack of a clear-cut answer here is a little worrisome. Anyone who wants to like Heroku would hope that the OP is flat out, 100%, wrong. The fact that Heroku's official answer requires a bit of managing implies otherwise. On a related tangent, I would also encourage future public statements to be a little less opaque than some Heroku has put out previously. For ins…

Maybe it doesn't need "managing", Oren might just want to talk with whoever was responsible for the change and see what the best way forward is. I don't think panicked, knee-jerk reactions like "OMG we were wrong and will revert that commit pronto!" are beneficial in situations as complex as this.

You're assuming that the change was actually made. Until we hear definitively from Heroku, the only evidence is an (admittedly, well documented) blog post.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#342

Post by vidarh »

adrr wrote: post #255
kennystone wrote: post #95

Earlier quoted context omitted.

If you have 2 unicorn servers and you happen to get 3 slow requests routed to it, you are still screwed, right? Seems to me like it will still queue on that dyno.

Maybe this is a stupid question, but with unicorn it forks the request and can process multiple requests at the same time. Previously it seems that only one request could be handled by the dyno so requests had to queue on the dynamic routing layer but with multiple request support with unicorn or whatever, wouldn't it be more efficient to dump all the requests to dynos? Followup question, also how would intelligent r…

> Previously it seems that only one request could be handled by the dyno so requests had to queue on the dynamic routing layer but with multiple request support with unicorn or whatever, wouldn't it be more efficient to dump all the requests to dynos?

It would be if all requests were equal. If all your requests always take 100ms, spreading them equally would work fine.

But consider if one of them takes longer. Doesn't have to be much, but the effect will be much more severe if you e.g. have a request that grinds the disk for a few seconds.

Even if each dyno can handle more than one requests, since those requests share resources, if some of them slows down due to some long running request, response times for the other requests are likely to increase, and as response times increase, it's queue is likely to increase further, and it gets more likely to pile up more long running requests.

> Followup question, also how would intelligent routing work if it just previously checked to see if which dyno had no requests? That seems like an easy thing to do, now you would have to check CPU/IO whatever and route based on load. Not specifically targeted at you but to everyone reading the thread.

There is no perfect answer. Just routing by least connections is one option. it will hurt some queries that will end up being piled up on servers processing a heavy request in high load situations, but pretty soon any heavily loaded servers will have enough connections all the time that most new requests will go to lighter loaded servers.

Adding "buckets" of servers for different types of requests is one option to improve it, if you can easily tell by url which requests will be slow.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#343

Post by regularfry »

skrebbel wrote: post #338
themgt wrote: post #313

Earlier quoted context omitted.

It's like saying EC2 should tailor its virtualization to Fedora 16, or Mac OS X should tailor its windowing system to Photoshop CS4, or Apache should tailor mod_proxy to Joomla. There may be specific attributes of popular applications that need to be adapted to, but those adaptations need to be built in a generic way and exposed through a standard API. Since even many Rails apps now do not follow a single threaded re…

> or Mac OS X should tailor its windowing system to Photoshop CS4 I'm taking this example out specifically, because it illustrates my point quite nicely. If there were a sizeable community of people who only wanted to use a computer for Photoshop, and tailoring the windowing system to them made it a significant usability improvement for those people, then it would be a completely imaginable situation that upon first…

Microsoft have definitely done exactly this. Windows versions have famously been "rebroken" in development to keep bug-for-bug compatibility so they didn't break big third party applications.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#344

Post by jules »

sabat wrote: post #183
jules wrote: post #51

Why not get yourself ONE beefy server? (or two) That should be able to handle your 150 requests per second, simplify your architecture a lot, and buying it would be cheaper than 1 month on Heroku (at $20,000/month).

Because when that one beefy server goes tits-up, you're out of business. Same with two.

Why? The second is idling until the first fails, then the second takes over. Unless both fail simultaneously of course, for example due to power outage, but then your 40 servers will also fail simultaneously. Not to mention that with just one running server there are a lot fewer failure scenarios.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#345

Post by Uchikoma »

~$20,000 sounds like a lot of money, they would need at least $100M a year in revenue to justify this number. This will be a major challenge if they want to grow profitable after the $15M VC money runs out. I'd assume they'd get the same for $5000 in rented servers which would free up enough money - outside of the valley - to have an DevOps and another developer.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#346

Post by vemv »

antihero wrote: post #339
arcatek wrote: post #153

Earlier quoted context omitted.

The requests will not be served at the same time , that's the whole point. If a request is routed to a busy dyno, you will have to wait that the previous job finish before being able to start yours.

Thing is though, it says that a Dyno is a Ubuntu virtual machine. In what sort of horrendous configuration can an ENTIRE VM serve only a SINGLE REQUEST AT A TIME?! That is utter madness, and the validity of the argument depends on whether it's the Heroku or this dude's fault that the VM is serving only a single request at a time (and it taking >1sec to handle a request).

Not Heroku's fault in this case, Rails (and any other single-threaded environment) can handle a single request at a time.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#347

Post by Terretta »

Uchikoma wrote: post #345

~$20,000 sounds like a lot of money, they would need at least $100M a year in revenue to justify this number. This will be a major challenge if they want to grow profitable after the $15M VC money runs out. I'd assume they'd get the same for $5000 in rented servers which would free up enough money - outside of the valley - to have an DevOps and another developer.

Not following. Why do you say they need $100 million a year in revenue to justify $240K (00.25% of revenue) in hosting expenses?

For an online biz, 10% - 50% isn't uncommon for profitable businesses. Many "virtual" companies (online only) do fine at 80%.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#348

Post by zen_boy »

Uchikoma wrote: post #345

~$20,000 sounds like a lot of money, they would need at least $100M a year in revenue to justify this number. This will be a major challenge if they want to grow profitable after the $15M VC money runs out. I'd assume they'd get the same for $5000 in rented servers which would free up enough money - outside of the valley - to have an DevOps and another developer.

How do you need $100M a year in revenue to cover $20k/mo or $240k/year?

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#349

Post by filvdg »

You can Model Queues and calculate the service level when you do inteligent routing

The Erlang C formula expresses the probability that an arriving customer will need to queue (as opposed to immediately being served).

http://owenduffy.net/traffic/erlangc.htm

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#350

Post by krutulis »

I can't help but wonder if this kind of surreptitious change to the platform might in any way be connected to Byron Sebastian's sudden resignation last September from Salesforce. Is that nutty of me?

http://gigaom.com/2012/09/05/heroku-loses-a-star-as-ceo-and-...

Post reply on HN