Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku Blog: Routing Performance Update

blog.heroku.com

Post reply on HN
Print view View original

131–140 of 197 posts

Re: Heroku Blog: Routing Performance Update

#131

Post by spitfire »

antirez wrote: post #90

Come on startups, you should be technically skilled and able to optimize in order to spend little money . If you sum EC2 and Heroku you are going to pay like 10x what it takes to run the same machines power in a dedicated server, all this because you can't handle the operations? This is absurd IMHO. Also people that want to start a business, there is a huge opportunity here , create software that makes managing Apach…

I'm honestly surprised there aren't tools available now replicating what Heroku does.

We should have puppet scripts to deploy, instrument and manage all the popular infrastructure choices by now.

The same way originally Linux was a build it yourself box of parts, we ought to have "cloud infrastructure" distributions, from bootup to app deployment.

The neat thing here is that as people improve the distribution you get cumulative savings. Back in the 90's you needed a skilled individual to setup a Unix/Linux system. Now, even an MBA can do it. The same could happen with infrastructure on a higher level.

Re: Heroku Blog: Routing Performance Update

#132

Post by derefr »

nthj wrote: post #116
derefr wrote: post #115

Earlier quoted context omitted.

No, presumably the pitch is: clone Heroku's software stack as an OSS project, then get the server/VPS sellers to let people opt to install it as a package when they rent out a node, the same way things like Wordpress can automatically be installed.

Sure, see OpenStack: http://www.openstack.org/ [that said, comparing automated scaling of a Ruby on Rails SaaS app with 100K+ users to a one-click WordPress install is kinda humorous. Even WordPress has a market for expert DevOps guys to manage your site: http://wpengine.com ]

Well, Herouk's stack may indeed be way more impressive in sheer scope, but for any one individual user, there's no difference.

As long as bringing up more than one node in your "application" gets them to start talking to one-another and auto-distributing load between them [including sometimes just electing one of them to serve as a router node that doesn't actually have apps running on it], your experience will be identical to paying Heroku, save for the fact that when one of the machines goes down--although the load will be redirected away from it--you'll have to "manually" fix it if you want it back.

In the default case, this just means dropping the IaaS node responsible and provisioning a new one [possibly on a different IaaS provider, if there's a lot of failure in one provider]. Though if you did pay for physical hardware, there can be some pain here. ;)

Basically, what I'm suggesting is a software stack that is to computation as Tahoe-LAFS is to storage. Just give it a set of machines that resemble Unix boxen--with more separate providers and regions = more better--and it'll give you something that looks like Heroku in exchange.

It could even have an optional provisioner service running atop it, that globally aggregates known IaaS providers and gives them ratings based on machine response times from the perspective of various other components, then lets you just say "okay, you're allowed $N/mo more to scale me up" and it'll find places around the globe where your users are being underserved, find well-rated API-compatible IaaS providers in those places, and provision nodes in those locations, adding them to your mesh. And then dropping them back out for alternatives if they start to suck :)

--you know, if this doesn't already exist, I might be willing to put in some time...

Re: Heroku Blog: Routing Performance Update

#133

Post by bgentry »

beambot wrote: post #17

This is still not ideal. Even if you're running unicorn, you're still susceptible to queueing spikes due to random load balancing. The concurrency just gives you a small buffer and/or some smoothing on 95th percentile responses. Right? At least there's a commitment to update the reporting tools... getting bad data in New Relic was (IMHO) the worst -- even worse than out-of-date docs.

Actually, no. Using unicorn with only 2 workers makes a tremendous difference, not just incremental. RapGenius' own statistical model demonstrates this.

Picture each individual dyno in that case as its own "intelligent router". Since it's not distributed and this requires no network coordination, the job of knowing which workers are available becomes trivial.

If you're inclined to read up on queuing theory, you'll see that having at least 2 processes per worker makes the problem much simpler.

Re: Heroku Blog: Routing Performance Update

#134

Post by badgar »

antirez wrote: post #121
ig1 wrote: post #107

Earlier quoted context omitted.

Let say your average http request takes 200ms to serve, then with 5 heroku worker dynos you can serve about 10 million requests a day which should comfortably cover the requirements of most startups. How much do those dynos cost ? - $143/month. That's less than three hours salary for a developer. If your monthly budget is hundreds of dollars I'm guessing your traffic is low enough you can just use the free tiers of h…

10,000,000 requests each day are 115 requests per second. I don't follow your math here with a 200ms CPU time per request.

Heroku customers don't typically do math when they do capacity planning. That's part of the Operations toolchest they're buying. They just up/down dynos.

Re: Heroku Blog: Routing Performance Update

#135

Post by karterk »

antirez wrote: post #122
nthj wrote: post #103

Earlier quoted context omitted.

> Also people that want to start a business, there is a huge opportunity here, create software that makes managing Apache, Redis, PostgreSQL, ..., in dedicated servers very easy and robust. Traget a popular and robust non-commercial distribution like Ubuntu LTE, and provide all is needed to deploy web nodes, database nodes, with backups, monitoring, and everything else trivial. So the pitch is: clone Heroku, which ha…

That's what I think about your reasoning. 1) Once you develop a culture of PaaS the reality is, if you check the history of most startups, you'll continue along this way, and the money you'll burn will be massive. I suspect that if you build a sysop culture earlier, later you'll be able to just hire a full time sysop instead of spending a lot more. 2) If you don't understand very well your platform from the point of…

Unless you selected a platform that is too complex to start with. I think that many of the PaaS companies I see today are here mainly for two reasons: Rails & Java frameworks.

Can't speak for Heroku but with EC2 you get the ability to dynamically add/remove instances. While this seems like it's needed only for certain kinds of startups, the reality is that most background processing is batch oriented and it's extremely cost effective to just spin up instances when I need them, crunch the data and terminate them after that. And, if you go for spot instances (and use a fault tolerant framework like Hadoop to distribute jobs), it's becomes even more cheaper.

Re: Heroku Blog: Routing Performance Update

#138

Post by pytrin »

antirez wrote: post #90

Come on startups, you should be technically skilled and able to optimize in order to spend little money . If you sum EC2 and Heroku you are going to pay like 10x what it takes to run the same machines power in a dedicated server, all this because you can't handle the operations? This is absurd IMHO. Also people that want to start a business, there is a huge opportunity here , create software that makes managing Apach…

My thoughts exactly. Like you don't outsource dev, you can't outsource sysops, it's a core competency to a tech startup.

Re: Heroku Blog: Routing Performance Update

#139

Post by sciurus »

nthj wrote: post #116
derefr wrote: post #115

Earlier quoted context omitted.

No, presumably the pitch is: clone Heroku's software stack as an OSS project, then get the server/VPS sellers to let people opt to install it as a package when they rent out a node, the same way things like Wordpress can automatically be installed.

Sure, see OpenStack: http://www.openstack.org/ [that said, comparing automated scaling of a Ruby on Rails SaaS app with 100K+ users to a one-click WordPress install is kinda humorous. Even WordPress has a market for expert DevOps guys to manage your site: http://wpengine.com ]

I don't think OpenStack is the right comparison to make to Heroku. In the common parlance, OpenStack is infrastructure as a service, while Heroku is platform as a service. Cloud Foundry http://cloudfoundry.com/ and OpenShift https://openshift.redhat.com/app/ are more comparable. They're both open source.

Re: Heroku Blog: Routing Performance Update

#140

Post by latchkey »

neebz wrote: post #78
latchkey wrote: post #49

I'm watching all this debate with great interest. The company I'm consulting for has ~150 dynos, a paid support contract, etc. We're on Cedar... NodeJS latest (which just got upgraded to 0.8.19 after being ignored since October), with Mongo, Rabbit and Redis hosted by third party addon companies. After a ton of H12 errors, they helped us find out some slow points and optimize some things that were relatively slow. On…

150 dynos with Node.js and you are still getting H12 errors? The issue so far reported is that if we ever get multiple requests on a single dyno then we'll have a queuing delay because Rails is essentially single-threaded. But with Node.js I think it would be a fairly large amount of requests on a single dyno I suspect before we get any queuing delays

Yes, it is a heavy traffic app. It is a backend for a very popular iphone app. The thing is that it doesn't matter if we have it set to 200, 300 or 70. We still get H12's. Sure, less now that we spent weeks tuning the absolute fuck out of our app and moving most of the processing of data offline to backends (using Rabbit as the queue), but the H12's still happen in the web tier on a regular basis (about 1-2 a minute instead of the 100's we were getting before).

The amazing part here to me is that the guys at Heroku don't even have a clear picture themselves of why H12 errors occur. They use internal tools like Splunk to provide visibility into their app, but they don't expose those to the customers. There really should be better reporting for all this stuff.

Post reply on HN