Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

axonflux.com

Post reply on HN
Print view View original

51–56 of 56 posts

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#51

Post by blader »

rantfoil wrote: post #50
blader wrote: post #48

Rails scales just fine - the problem is that it's just really expensive to do it. Here's some of our learnings: 1. Use query_trace to trace your DB calls and query_analyzer to automatically run EXPLAIN on each call: http://github.com/ntalbott/query_trace/tree/master http://github.com/jeberly/query-analyzer/tree/master 2. Use our patches to mongrel proc_title to troubleshoot slow queries: http://asemanfar.com/Request-…

I dunno about saving a couple engineer's worth of money -- that's why Rails is massively viable in the first place. You absolutely can iterate faster with fewer people and less wasted time. Servers are cheap. People (salaries) are expensive. (Well.. until servers become expensive due to straight up load/scale, that is.)

Right now we're estimating that our Rails premium = 4X the salary of a San Francisco engineer. But we do have a lot of servers.

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#52

Post by jasonkester »

cperciva wrote: post #11

Sad town happens. 1 in 4 requests after Request A will go to port 8000, and all of those requests will wait in line as that mongrel chugs away at the slow request... I don't use Rails, or even Ruby at all if I can avoid it, so I'm sure I'm missing something obvious here, but... why in the world would anyone want to use a web server which can only handle one request at once?

I get that feeling too every time I read an article like this.

I develop 90% of my stuff on the Microsoft stack, and it just plain handles anything you can throw at it out of the box. We see 2,000,000 pageviews a day steady state, and very seldom lift the CPU off of zero, with nothing more than bread and butter index and stored procedure tuning.

With that as a baseline, I just don't see why all these little startups are having such problems just keeping their website up under a little traffic. I can't believe that these scripted ORMs are really that inefficient that you'd need to spend this amount of effort bolting 3rd party stuff onto them just to keep them alive.

There has to be something else going on. What, exactly am I missing here?

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#53

Post by charlesju »

blader wrote: post #47
charlesju wrote: post #31

I hate when people tell me that Rails doesn't scale because I have scaled Rails and it has nothing to do with the framework. The reason why websites can't scale is ALWAYS the DB. To fix the DB it is always master/slave, memcached, then sharding. For EVERY language. Good article, sums up a lot of the tools I used to scale my stuff. I'd like to add a little more. 1. Turn on slow-query logging in your DB and tail -f the…

"5. If you're a serious startup, use Engine Yard, they're life savers." - If you're a serious startup that doesn't monetize off of display ads, use Engine Yard. Otherwise, you can't afford them.

If you're a serious startup, you're not monetizing off display ads.

:)

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#54

Post by rantfoil »

shafqat wrote: post #42

Can someone shed some extra light about the point re: reducing the number of requests to the DB for a dynamic page. When left unoptimized, Rails (and a lot of other frameworks) often result in 100 DB queries for a page. One obvious way around this is to ensure the DB joins are done correctly. But the article mentions batching/grouping up the requests. How does that work?

Check out the :include parameter to find method calls in the ActiveRecord documentation.

Say you have 30 blog posts, and your views reference associations to the blog post's owners. Well, views are dumb and if you call post.user on each one within a loop, you end up calling User.find 30 times.

But if you do Post.find(..., :include => [:user]), Rails will know to eager load all users -- and User.find never gets called 10 times.

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#55

Post by pwk »

teej wrote: post #18

> You're only as fast as your slowest query.... Use HAProxy. Take the time to configure it correctly. It's absolutely the most stable and useful loadbalancer I've used. It also solves this issue by talking to your backends.

We use pen, which can be configured to avoid this issue as well. Just specify the maximum number of connections to be the same as the number of backend servers (via the -x option), and it will queue connections beyond that number and hand them to the first backend that frees up.

We've had zero issues with pen, it's been rock solid, but we're still looking at moving to Passenger at some point: seems like it will be more flexible and efficient than our current pack of mongrels.

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#56

Post by teej »

pwk wrote: post #55
teej wrote: post #18

> You're only as fast as your slowest query.... Use HAProxy. Take the time to configure it correctly. It's absolutely the most stable and useful loadbalancer I've used. It also solves this issue by talking to your backends.

We use pen, which can be configured to avoid this issue as well. Just specify the maximum number of connections to be the same as the number of backend servers (via the -x option), and it will queue connections beyond that number and hand them to the first backend that frees up. We've had zero issues with pen, it's been rock solid, but we're still looking at moving to Passenger at some point: seems like it will be mo…

I attempted to switch to pen when I was having issues with Perlbal. It was segfaulting under light load (light for me is heavy for most people) comparable to Perlbal, so I switched back. I can't say I reccomend it at all.
Post reply on HN