Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

axonflux.com

Post reply on HN
Print view View original

41–50 of 56 posts

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#41

Post by aaronblohowiak »

danielrhodes wrote: post #35
charlesju wrote: post #31

I hate when people tell me that Rails doesn't scale because I have scaled Rails and it has nothing to do with the framework. The reason why websites can't scale is ALWAYS the DB. To fix the DB it is always master/slave, memcached, then sharding. For EVERY language. Good article, sums up a lot of the tools I used to scale my stuff. I'd like to add a little more. 1. Turn on slow-query logging in your DB and tail -f the…

Actually, this is generally good advice regardless of what language you're on. The database is almost always at fault. My general rule is that a query should generally take less than 0.001 seconds to execute. If you can get it down to that through proper indexing and whatnot, MySQL will execute the queries fast enough so that they don't get backed up and eventually kill the server. Don't start caching until you can w…

... and be less afraid of de-normalizing your data!

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#42

Post by shafqat »

Can someone shed some extra light about the point re: reducing the number of requests to the DB for a dynamic page. When left unoptimized, Rails (and a lot of other frameworks) often result in 100 DB queries for a page.

One obvious way around this is to ensure the DB joins are done correctly. But the article mentions batching/grouping up the requests. How does that work?

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#43

Post by tdavis »

reconbot wrote: post #39
tdavis wrote: post #5

I don't know jack about Rails, but there is some good general advise here too. I would have liked to see some DB commentary that didn't choose MySQL as a foregone conclusion. I can't think of many instances where I would recommend it in general.

May I ask what you do use exactly? MySQL has always been my goto for simple db needs. What do you normally use and what's your "general" cases where you wouldn't use it?

For "simple [rdms] needs" I would recommend SQLite. For anything more, PostgreSQL. I find MySQL too buggy and it diverts from the SQL standard too often (or doesn't implement enough of it) for my tastes. A properly configured postgre install (granted, not exactly trivial) will perform at least as good if not better than MySQL and its advanced functionality is extremely mature and robust, unlike MySQL's which has largely been tacked on in the current major version (views, templates, triggers, etc.)

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#44

Post by technoweenie »

rantfoil wrote: post #7
tptacek wrote: post #4

Great article. Background/deferred job processing has been immensely painful for us, and I have no idea what the accepted best job queue is. We're still stuck on backgroundrb, which is a nightmare. Nanite looks like overkill, and has too many deps.

Workling has worked well for us -- Rany Keddo is a phenomenal open source guy. Tobi's deferred jobs also looked totally solid to me. That one's proven because it drives Shopify. We use his liquid plugin, which is also stellar. That underlines a big issue with Rails dev even today -- its really hard to know what's good / what works, and what is just some weekend project for someone.

Delayed-Job works great for Lighthouse/Tender from us, and for Github too. It's definitely a great starter queue until your needs necessitate something more heavy duty like nanite.

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#45

Post by railsguy »

tptacek wrote: post #4

Great article. Background/deferred job processing has been immensely painful for us, and I have no idea what the accepted best job queue is. We're still stuck on backgroundrb, which is a nightmare. Nanite looks like overkill, and has too many deps.

the async plugin is quite nice. you can replace

>> myobject.very_slow_method

with

>> myobject.async :very_slow_method

and it's executed in background

http://github.com/lassej/async/tree/master

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#46

Post by ezmobius »

tptacek wrote: post #12
rantfoil wrote: post #7

Earlier quoted context omitted.

Workling has worked well for us -- Rany Keddo is a phenomenal open source guy. Tobi's deferred jobs also looked totally solid to me. That one's proven because it drives Shopify. We use his liquid plugin, which is also stellar. That underlines a big issue with Rails dev even today -- its really hard to know what's good / what works, and what is just some weekend project for someone.

Found Tobi's delayed_job (not deferred) --- I think I like the design better than Workling; fewer moving parts, just a database backend end a rake script. The biggest problem we have with backgroundrb (which again: nightmare) is not really ever knowing the state of current running jobs.

yeah backgroundrb sucks, who wrote that shit? (me and i wish i never did)

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#47

Post by blader »

charlesju wrote: post #31

I hate when people tell me that Rails doesn't scale because I have scaled Rails and it has nothing to do with the framework. The reason why websites can't scale is ALWAYS the DB. To fix the DB it is always master/slave, memcached, then sharding. For EVERY language. Good article, sums up a lot of the tools I used to scale my stuff. I'd like to add a little more. 1. Turn on slow-query logging in your DB and tail -f the…

"5. If you're a serious startup, use Engine Yard, they're life savers."

- If you're a serious startup that doesn't monetize off of display ads, use Engine Yard. Otherwise, you can't afford them.

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#48

Post by blader »

Rails scales just fine - the problem is that it's just really expensive to do it.

Here's some of our learnings:

1. Use query_trace to trace your DB calls and query_analyzer to automatically run EXPLAIN on each call:

http://github.com/ntalbott/query_trace/tree/master http://github.com/jeberly/query-analyzer/tree/master

2. Use our patches to mongrel proc_title to troubleshoot slow queries: http://asemanfar.com/Request-Queue-via-Mongrel-Proctitle

3. Don't use ActiveRecord.

4. Don't use any link or url helpers in Rails.

5. For that matter, don't use Rails. Rewrite your most hit components in something faster.

I love Ruby, but the simple truth of it is that we'd be saving a couple of engineer's worth of money if we weren't on Rails.

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#49

Post by charlesju »

pcowans wrote: post #34
charlesju wrote: post #31

I hate when people tell me that Rails doesn't scale because I have scaled Rails and it has nothing to do with the framework. The reason why websites can't scale is ALWAYS the DB. To fix the DB it is always master/slave, memcached, then sharding. For EVERY language. Good article, sums up a lot of the tools I used to scale my stuff. I'd like to add a little more. 1. Turn on slow-query logging in your DB and tail -f the…

Rails running on vanilla Ruby does have particular scaling issues due to the limitations of the Ruby garbage collector. You hit a GC cycle after every 8MB of allocation, which typically takes around 150ms to run, so can dominate the runtime of your requests. We've had 80% of the runtime in GC, even when you include database time. This isn't necessarily a killer, but it means there's sometimes more work than you might…

That's an application bottleneck which is fixed by more application servers. This has no effect on scaling what-so-ever.

And by scaling here, we're talking about 1 M to 10 M, not 10,000 to 100,000.

Re: Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)

#50

Post by rantfoil »

blader wrote: post #48

Rails scales just fine - the problem is that it's just really expensive to do it. Here's some of our learnings: 1. Use query_trace to trace your DB calls and query_analyzer to automatically run EXPLAIN on each call: http://github.com/ntalbott/query_trace/tree/master http://github.com/jeberly/query-analyzer/tree/master 2. Use our patches to mongrel proc_title to troubleshoot slow queries: http://asemanfar.com/Request-…

I dunno about saving a couple engineer's worth of money -- that's why Rails is massively viable in the first place.

You absolutely can iterate faster with fewer people and less wasted time.

Servers are cheap. People (salaries) are expensive.

(Well.. until servers become expensive due to straight up load/scale, that is.)

Post reply on HN