LinkedIn Mobile Moved from Rails to Node: 27 Servers Cut and Up to 20x Faster

Post by gnufied » Thu, Oct 04, 2012, 8:49 PM UTC

justinjlynn wrote: post #26

gnufied wrote: post #19

Earlier quoted context omitted.

From original article[1] (Not the highscalability link spam): >This led them to a model where the application is essentially piping all of its data through a single connection that is held open for as long as it is needed. So it sounds like from using a request/response driven architecture, they adopted streaming architecture. Also, they moved away from Rails and adopted an Event driven approach. It is well known fac…

Indeed, shifting to a streaming architecture can vastly reduce the amount of churn your application sees. This makes great sense if it fits the use case of your product. However, one need not throw out a huge existing code/knowledge base in order to accomplish this shift. I think that it's perfectly acceptable to shift to something like EventMachine + some HTTP layers to implement a streaming server using a good chun…

Post by rhizome » Thu, Oct 04, 2012, 9:02 PM UTC

gnufied wrote: post #27

rhizome wrote: post #21

Earlier quoted context omitted.

Sure, but is it unreasonable to wonder why they need such an architecture? LinkedIn is basically a CRUD app, and while they have a wall now and yadda yadda, I wonder how much of this rearch was really necessary, over simple refactorings and sysadmin attention to the nuts and bolts.

Ability to stream stuff from an Event driven reactor loop makes lot of sense when it comes to raw performance actually. If you throw in Postgres database in mix which supports asynchronous queries - one can pretty much beat 20 passenger instances serving similar request. The problem again though is, doing non-blocking IO does not reduce database load. So, if likely they re-architected that bit as well.

Post by pdelgallego » Thu, Oct 04, 2012, 9:12 PM UTC

Smudge wrote: post #10

SoftwareMaven wrote: post #3

If you are thinking about using node.js for this reason[1] on most sites, you are optimizing poorly. LinkedIn didn't worry about this until after they were a public company. If Python/Djando or Ruby/Rails can get your app out the door and into customer hands faster, it is almost always the right thing to use. 1. There are certainly other, very valid, technical reasons for choosing node.js over other technologies earl…

I'm sure LinkedIn could have cut servers without switching to node. Take your first, crappy implementation and rewrite it in the same language and you'll probably still see at least 10x improvement, if not 20.

Post by erichocean » Thu, Oct 04, 2012, 9:16 PM UTC

astrodust wrote: post #28

lumberjack wrote: post #11

Earlier quoted context omitted.

To be fair, Node.js isn't exactly harder to work with than Django or Rails. I myself had to choose between these about a year ago and despite the asynchronous paradigm I still found it relatively easy to start using in comparison to Rails and Django.

Node.js is extremely hard to work with compared to Rails and moderately hard to work with compared to Django. The Node.js ecosystem has a lot of potential, but the variety of off-the-shelf add-ons is severely limited compared to either of those more mature frameworks. If you think Node.js is easy, you've never really experimented enough to understand what makes Rails so effortless. It's a lot easier to produce a prod…

Post by markmm » Thu, Oct 04, 2012, 9:20 PM UTC

SoftwareMaven wrote: post #3

If you are thinking about using node.js for this reason[1] on most sites, you are optimizing poorly. LinkedIn didn't worry about this until after they were a public company. If Python/Djando or Ruby/Rails can get your app out the door and into customer hands faster, it is almost always the right thing to use. 1. There are certainly other, very valid, technical reasons for choosing node.js over other technologies earl…