Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LinkedIn Mobile Moved from Rails to Node: 27 Servers Cut and Up to 20x Faster

highscalability.com

Post reply on HN
Print view View original

111–120 of 140 posts

Re: LinkedIn Mobile Moved from Rails to Node: 27 Servers Cut and Up to 20x Faster

#111

Post by lsh123 »

The performance improvements have probably NOTHING to do with node.js but with the re-architecture goals set by the team:

"For our inevitable rearchitecting and rewrite, we want to cache content aggressively, store templates client-side (with the ability to invalidate and update them from the server) and keep all state purely client side."

Better understanding of the problem and experience running the system were probably key for building the new high-performance architecture. Obviously, old one lacked these big advantages.

Re: LinkedIn Mobile Moved from Rails to Node: 27 Servers Cut and Up to 20x Faster

#112

Post by yxhuvud »

markmm wrote: post #39
SoftwareMaven wrote: post #3

If you are thinking about using node.js for this reason[1] on most sites, you are optimizing poorly. LinkedIn didn't worry about this until after they were a public company. If Python/Djando or Ruby/Rails can get your app out the door and into customer hands faster, it is almost always the right thing to use. 1. There are certainly other, very valid, technical reasons for choosing node.js over other technologies earl…

I would spend a little extra time upfront and save many many expensive rewrites latter on. Use C from the start, you won't need to rewrite just scale with hardware. This argument that using Ruby (or any language that makes programming easy but runs like treacle) to get something out the door a few months earlier is bullshit, I would rather release something a few months later that I didn't have to totally rewrite dow…

What makes you think you know the problem space enough to make a C implementation that you won't have to rewrite 10 times? Writing in C certainly doesn't stop any rewrites at least where I work.

Re: LinkedIn Mobile Moved from Rails to Node: 27 Servers Cut and Up to 20x Faster

#114

Post by jes5199 »

pdelgallego wrote: post #35
Smudge wrote: post #10

Earlier quoted context omitted.

I'm sure LinkedIn could have cut servers without switching to node. Take your first, crappy implementation and rewrite it in the same language and you'll probably still see at least 10x improvement, if not 20.

>> Take your first, crappy implementation and rewrite it in the same language and you'll probably still see at least 10x improvement. You will probably will get the same mess. There is plenty of literature about that. I.e: A complete rewrite is what killed Netscape

Don't rewrite the whole thing at once! Cut off logical sections and fix them one at a time.

Re: LinkedIn Mobile Moved from Rails to Node: 27 Servers Cut and Up to 20x Faster

#115

Post by statictype »

benologist wrote: post #110
Smudge wrote: post #10

Earlier quoted context omitted.

I'm sure LinkedIn could have cut servers without switching to node. Take your first, crappy implementation and rewrite it in the same language and you'll probably still see at least 10x improvement, if not 20.

Node has some pretty unique benefits, I save a good $1000/month from switching from .NET on dedicateds to NodeJS on a PaaS (Heroku), and that was after 2 - 3 years of writing and rewriting and optimizing the .NET stuff. Biggest improvements came from persistant connections to redis/mongodb and polling for updated information independently of requests so there was no cached-or-fetch shenanigans at all in some areas.

What were the benefits you saw in switching from .NET to Node? Was it in basic code structure/complexity or performance?

I'd actually be very surprised if NodeJS running on Heroku (which is built on EC2) performs better than compiled .NET code running on dedicated Windows hardware.

Re: LinkedIn Mobile Moved from Rails to Node: 27 Servers Cut and Up to 20x Faster

#116

Post by statictype »

markmm wrote: post #68
andybak wrote: post #60

Earlier quoted context omitted.

I look forward to seeing C as the go-to web-development language. Really? C for a problem that is largely string manipulation and database access?

Why are they moving from Ruby if it's just "string manipulation and database access"? The problem is folks, as soon as you get a large number of users every compromise you made by using a toy language or database is magnified 1000 times. Google didn't implement in some scripting language to get to the market a few months early.

Google didn't implement in some scripting language to get to the market a few months early.

That's actually exactly what they did.

Re: LinkedIn Mobile Moved from Rails to Node: 27 Servers Cut and Up to 20x Faster

#118

Post by cmccabe »

wardb wrote: post #5

Reading this like: "Hey, our previous backend was a total turd, technically speaking." It might be trivial to speed up your own crappy first implementation 20x with some extra TLC.

yeah, but it's easier politically to rewrite it in a newer language. If you rewrite it in the same language, then you have to explain your rewrite as cleaning up someone's braindead mistakes-- someone who might still work there.

If you rewrite it in a different language, then you get to blame the old language and framework for all the problems. No harm, no foul-- and you get to put the new shiny thing on your resume.

Re: LinkedIn Mobile Moved from Rails to Node: 27 Servers Cut and Up to 20x Faster

#119

Post by benologist »

statictype wrote: post #115
benologist wrote: post #110

Earlier quoted context omitted.

Node has some pretty unique benefits, I save a good $1000/month from switching from .NET on dedicateds to NodeJS on a PaaS (Heroku), and that was after 2 - 3 years of writing and rewriting and optimizing the .NET stuff. Biggest improvements came from persistant connections to redis/mongodb and polling for updated information independently of requests so there was no cached-or-fetch shenanigans at all in some areas.

What were the benefits you saw in switching from .NET to Node? Was it in basic code structure/complexity or performance? I'd actually be very surprised if NodeJS running on Heroku (which is built on EC2) performs better than compiled .NET code running on dedicated Windows hardware.

The major benefits were persistant connections and background fetching of data - a lot of my requests serve data directly from local memory instead of hitting local or shared caches and databases.

The equivalent in .NET I guess would be BackgroundWorkers that are independently prefetching the data required most of the time but I could never get them to Just Work.

Specifically for my use case 99+ percent of requests receive some data, do some light manipulations and then push the data to redis about 8,000 to 12,000 times a second. With .NET I could only push to locally running software instances because anything remote couldn't keep up with the connection volume (without throwing even more hardware at the problem).

Re: LinkedIn Mobile Moved from Rails to Node: 27 Servers Cut and Up to 20x Faster

#120

Post by nicholassmith »

omaranto wrote: post #34
nicholassmith wrote: post #20

Earlier quoted context omitted.

This times a thousand. When you first write the software you're basing it on expectations, no matter how well you plan, but when you rewrite it you're coming at it with real world knowledge of the pain points so of course it's going to be better in terms of performance.

"This times a thousand" would be an improvement by a factor of 10000. The improvement should definitely not be that large.

I included an extra bit of improvement to deal with inflation rates.
Post reply on HN