Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

rapgenius.com

Post reply on HN
Print view View original

401–410 of 437 posts

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#401

Post by JPKab »

teraflop wrote: post #197
damian2000 wrote: post #171

Earlier quoted context omitted.

I believe the article is saying that this wait will only occur with Rails due to it being single threaded? Reactor pattern frameworks like node don't suffer the same issue.

They will if the request is CPU-bound. In that case, throwing more concurrent requests at a server than it has cores just slows all of them down.

You are exactly right, but I would hope that someone wouldn't be using Node at all if CPU bound requests are even remotely commoneplace on their app. But of course, if they built their app to spawn processes when this happens, you end up with a case of a need for intelligent routing.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#402

Post by Dylan16807 »

chc wrote: post #399
Dylan16807 wrote: post #242

Earlier quoted context omitted.

Yes it does sound as horrific. It should never route more than N requests to a dyno, where N is the number of requests it can handle simultaneously. It doesn't matter if N is 1, 2, or 10. And it should always send new requests to the emptiest dyno.

While I agree that routing to the emptiest dyno would be a good thing, I think it's a little melodramatic to describe giving every dyno at least three times and up to several thousand times as much power as "horrific."

Power, what?

The problem is when you send a dyno that has all its threads stuck on long-running computations a new request, because it won't be able to even start processing it. The power is orthogonal to the problem.

The only mitigation is that if a dyno can handle a large number of threads, it probably won't get clogged. But if it can only handle 3 and gets new requests at random, you're in a bad place.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#403

Post by Dylan16807 »

chc wrote: post #393
vidarh wrote: post #388

Earlier quoted context omitted.

Doesn't matter. At some point you hit memory limits, disk IO limits, or simply a connection limit. It doesn't matter what limit: If you have some requests that are longer running than others, random load balancing will make them start piling up once you reach some traffic threshold. You can increase the threshold by adding more backends or increasing the capacity of each backend (by optimizing, or picking beefier har…

> But no matter what you do, you end up having to allocate more spare resources at it than what you would need with more intelligent routing. I think we have to remember that the "intelligent routing" in question here is actually marketing-speak for "one request per server." Are you saying that when your servers can only receive one request at a time, you will necessarily need fewer than if your servers can handle th…

No it's not. "Intelligent routing" means giving each server exactly as many requests as it can handle, and no more. If your servers can handle 3 requests each, using intelligent routing to make sure they never get more than 3 will blow the latency of the random method out of the water.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#405

Post by Scramblejams »

baudehlo wrote: post #395
encoderer wrote: post #251

Earlier quoted context omitted.

Three thoughts: 1. Employees are expensive. A good ops guy who believes in your cause and wants to work at an early stage startup can be had for $100k. (Maybe NYC is much cheaper than the bay area, but I'll use bay area numbers for now because it's what I know). That's base. Now add benefits, payroll taxes, 401k match, and the cost of his options. So what... $133k?. That's one guy who can then never go on vacation or…

Your last sentence hides another option which is the route we took: outsource devops. It cost us about $2000 in consultant fees for a fully setup system, easy to expand and add hardware to, and much more cost effective long term than AWS or Heroku. Our guy runs a small ops company who have 24/7 on-call. It's really the perfect solution.

I may have a use for this. Link to your ops provider?

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#406

Post by joeblau »

JPKab wrote: post #400
lkrubner wrote: post #143

Good lord!!!!! Percentage of the requests served within a certain time (ms) 50% 844 66% 2977 75% 5032 80% 7575 90% 16052 95% 20069 98% 29282 99% 30029 100% 30029 (longest request) Those numbers are amazingly awful. If I ever run ab and see 4 digits I assume I need to optimize my software or server. But 5 digits? Why in the world would a company spend $20,000 a month for service this awful?

I've been scanning through the comments, and I have yet to see anything written by a Heroku engineer to defend the company. I'm hoping its in here and I missed it. I have a feeling that this all might be absolutely true, and they have lawyers/PR trying to think of a damage control plan. I suspect that the reason they'be been pushed to do this is financial, and it makes me think that Nodejitsu's model of simply not pr…

Nodejitsu actually does have a free plan if your project is Open Source.

http://blog.nodejitsu.com/opensource-dot-jitsu

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#407

Post by mcguire »

shabble wrote: post #368
bambax wrote: post #325

Earlier quoted context omitted.

What's the point of posting a link to the front page of your blog, where the most recent article is 15 days old (4 hours after the comment above)? What we want to know: - is the OP right or wrong? That is, did you switch from smart to naive routing, for all platforms, and without telling your existing or future customers? - if you did switch from smart to naive routing, what was the rationale behind it? (The OP is li…

>> I am committing to the community to provide more information as soon as possible, including a blog post on http://blog.heroku.com > What's the point of posting a link to the front page of your blog, where the most recent article is 15 days old (4 hours after the comment above)? I think OP is saying 'I am going to investigate the situation; when I am finished here [the blog] is where I will post my response', not t…

Note: I think we have different referents for "OP" here; bombax's is, I think, the whining customer; while shabble's is the pompous CEO.

Me, I'm the swarthy pirate. Arrrh.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#408

Post by sergiotapia »

marcamillion wrote: post #256
benihana wrote: post #94

I really like Rap Genius, but I wish they would tone down the blackness of the background. Reading #CCC text on #000 background makes my eyes bug out.

Perspective is a hell of a thing....The way this comment reads, I thought this was going to get racist real quick - but was relieved when I finished reading and did agree with you :)

I think that speaks more on your dormant racism than anyone elses.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#409

Post by argarg »

ylansegal wrote: post #247
spoiler wrote: post #155

Earlier quoted context omitted.

Slightly off topic, but what are everyone's. experience and thoughts about Puma[1]? I am using it on a small production environment with Heroku and I like it, but when we officially launch the app, should we switch to Unicorn? [1] http://puma.io/

I did some performance analysis on puma vs unicorn vs thin a while ago: http://ylan.segal-family.com/blog/2012/08/20/better-performa... Although as noted in the comments, I neglected to run threadsafe! and should have probably tried rubinius or jruby. I have been meaning to redo. Take with a grain of salt

You should give a try to puma 2.0 currently in beta 6 (https://rubygems.org/gems/puma). Lots of performance improvements. I haven't benchmarked it but my guess is it outperforms unicorn.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#410

Post by themgt »

vidarh wrote: post #389
DigitalJack wrote: post #386

Earlier quoted context omitted.

"The system could easily be built so that it by default only aggregates those services where the configuration indicates they can handle a concurrency below a certain level, and does random balancing of everything else." Let me know when you are done with that.

I've built fairly large haproxy based infrastructures, thank you very much. Doing this is not particularly challenging. Actually what I'd probably do for a setup like this would be to balance by the Host: header, and simply have the second layer be a suitable set of haproxy instances balancing each by least connections. Immediately vastly better than random.

Haproxy doesn't support dynamic configurations as far as I know, which is a serious problem if you're letting lots of people add/change domains and scale backends up/down dynamically. A Heroku haproxy would probably need to be restarted multiple times a second due to config changes. Nginx can do dynamic backends with lua & redis, but it can't use the built-in upstream backend balancing/failover logic if you do.
Post reply on HN