Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

rapgenius.com

Post reply on HN
Print view View original

21–30 of 437 posts

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#22

Post by lil_tee »

andrewcooke wrote: post #9

i don't think some details of the argument hold. it alleges that you need more dynos to get the same throughput. but that's not true once you have sufficient demand to keep a queue of about sqrt(n) (i think - someone who knows more theory than me can correct me) in size on the dyno (where you have n dynos). because at that point all dynos will be running continuously, and the throughput will be the same with either r…

But you never want to have a queue on any of your dynos! A queued request means that a user is waiting with no response. If your goal is to have 0 (or less than epsilon) requests queued, it takes far fewer dynos if the requests are routed intelligently

If you have 10 dynos and 1000 simultaneous requests, the difference between naive and intelligent might well be reduced, but that's also a scenario in which your end user response times would be horrendously slow and so you'd need more dynos either way

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#24

Post by aristus »

mattj wrote: post #12

So the issue here is two-fold: - It's very hard to do 'intelligent routing' at scale. - Random routing plays poorly with request times with a really bad tail (median is 50ms, 99th is 3 seconds) The solution here is to figure out why your 99th is 3 seconds. Once you solve that, randomized routing won't hurt you anymore. You hit this exact same problem in a non-preemptive multi-tasking system (like gevent or golang).

I do perf work at Facebook, and over time I've become more and more convinced that the most crucial metric is the width of the latency histogram. Narrowing your latency band --even if it makes the average case worse-- makes so many systems problems better (top of the list: load balancing) it's not even funny.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#26

Post by htsh »

Why not hire a devops guy & rack your own hardware? Or get some massive computing units at amazon (just as good but more expensive)?

This reminds me of the excellent 5 stages of hosting story shared on here from a while back:

http://blog.pinboard.in/2012/01/the_five_stages_of_hosting/

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#27

Post by zende »

rdl wrote: post #20

Wow. I suspect Rap Genius has the dollars now where it's totally feasible for them to go beyond Heroku, but it still might not be the best use of their time. But if they have to do it, they have to do it. OTOH, having a customer have a serious problem like this AND still say "we love your product! We want to remain on your platform", just asking you to fix something, is a pretty ringing endorsement. If you had a marg…

Rap Genius is limited more by time than by money if anything. It would make more sense to throw money at the problem instead of people.

Re: Heroku's Ugly Secret: The story of how the cloud-king turned its back on Rails

#30

Post by gleb »

bifrost wrote: post #2

I am only going to suggest a small edit -> s/Postgres can’t/Heroku's Postgres can't/ PG can scale up pretty well on a single box, but scaling PG on AWS can be problematic due to the disk io issue, so I suspect they just don't do it. I'd love to be corrected :)

Postgres has a limitation on number of open connections. This is because 1 connection = 1 process. MySQL uses threads, which scales better but has other downsides. The thread-based approach is also possible with Postgres using 3rd party connection pooling apps, e.g. pgbouncer.
Post reply on HN