Live data from Hacker News

Show HN: Dynosaur, An autoscaler for Heroku using Google Analytics Live

engineering.harrys.com

11–20 of 24 posts

Re: Show HN: Dynosaur, An autoscaler for Heroku using Google Analytics Live

#11

So I'm curious, how many users per dyno are needed for a typical CRUD rails app?

That really depends on your app and your caching strategy. If you are doing something complex or memory intensive (or your app is poorly optimized), or if your users are highly engaged, it could be in the low tens of users! If you are doing a lot of caching you should be able to serve static pages on the order of several hundred or more per dyno.

Re: Show HN: Dynosaur, An autoscaler for Heroku using Google Analytics Live

#14
post #7

Earlier quoted context omitted.

Have you done any comparisons/benchmarks vs Adept Scale, Bounscale, or HireFire? Also why Google Analytics default as opposed to some other service?

Just speaking off the top of my head here, but services like HireFire ping the Heroku API to poll for information on when to scale. The Heroku API is a little rate limited. You'll probably get a more responsive result using GA.

HireFire uses New Relic. I would think that's more responsive than GA.

Re: Show HN: Dynosaur, An autoscaler for Heroku using Google Analytics Live

#15

Doesn't it make more sense to scale based on New Relic response times? Why are you guys using # of users? Depending on the page they are requesting and how the requests are clustered that could produce vastly inferior results.

Performance engineer here.

Scaling should occur on actual traffic/user throughput statistics not on response time. Response times can increase for a variety of reasons, of which only one of is increased traffic load. For example, response times can increase based on back-end database contention or thread contention.

Of course whichever configuration is chosen, it is important to load test your application so you understand its performance profile and that all contention points are understood.

Re: Show HN: Dynosaur, An autoscaler for Heroku using Google Analytics Live

#16

Doesn't it make more sense to scale based on New Relic response times? Why are you guys using # of users? Depending on the page they are requesting and how the requests are clustered that could produce vastly inferior results.

We find that neither New Relic nor Analytics gives the full picture: some of our pages are heavily cached, others (e.g checkout processing) are computationally expensive, database heavy and communicate with other systems (e.g payment processors) that can be a big bottleneck. Both New Relic and GA tend to just average those together (although with GA you can create new views that focus on specific pages). You are right that 'number of visitors on site' does not reflect our site performance in every respect.

We first conceived of Dynosaur as a plugin-based autoscaler (with GA and New Relic plugins to start with), but we've found the times we really need to scale fast are the times we have a lot of traffic generated from press stories etc (like this from today, if you will excuse the shameless plug: http://dealbook.nytimes.com/2014/01/21/a-start-up-run-by-fri...) and using the analytics live API allows us to react a little quicker than if we waited for New Relic to tell us our response times are getting slow. So far, we're happy enough with just a Google Analytics plugin.

One possible improvement would be to scale differently based on different traffic / performance metrics across the site. I think New Relic or other performance instrumentation would be very useful for that.

Re: Show HN: Dynosaur, An autoscaler for Heroku using Google Analytics Live

#17
I'm curious why people put so much effort into scaling Heroku when you could outperform the maximum number of Dynos with only a handful of AWS instances for a fraction the price. It doesn't make sense. We're literally working around a problem that shouldn't exist.

Re: Show HN: Dynosaur, An autoscaler for Heroku using Google Analytics Live

#18

I'm curious why people put so much effort into scaling Heroku when you could outperform the maximum number of Dynos with only a handful of AWS instances for a fraction the price. It doesn't make sense. We're literally working around a problem that shouldn't exist.

This is exactly what I'm wondering too. It's ingenious, but isn't Heroku + GA Live + Dynosaur both more complex and more expensive than just using AWS Elastic Beanstalk, which has Auto Scaling? What am I missing?

Re: Show HN: Dynosaur, An autoscaler for Heroku using Google Analytics Live

#20
post #2

Dynosaur author here, if anyone has questions.

Looks great. Wouldn't work well for us though because our main enemy is over-aggressive spiders that don't trigger Analytics trackers.

Same might apply to an overloaded api / anything not serving html pages to browsers.

Heroku provides log-runtime-metrics, which includes current cpu load / pending cpu tasks. The librato addon also shows request throughput in its dashboard UI. I'm not sure where it gets this data from. If not logging then perhaps new relic?

Post reply on HN