Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is NodeJS stable enough to build the next Twitter?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 60 posts

Re: Ask HN: Is NodeJS stable enough to build the next Twitter?

#31

Post by mathrawka »

masterkain wrote: post #30
mathrawka wrote: post #23

Yes, it is stable enough. BUT, that assumes you know how to write code that will work under various conditions that can and do arise. The main thing that can cause shit to hit the fan is not properly handling errors. I highly suggest using domains, and that when an error occurs, if you can, that you gracefully exit. If not, then all other requests will just abort and that isn't very user friendly. You will also want…

Domains are 0.10 specific I believe, and 0.10 is still bugged in few areas. I'd stay on 0.8.23 for quite some time since we are talking about stability.

I'm using 0.10 in production and haven't had any bugs... what bugs are you referring to?

Also domains are in 0.8

Re: Ask HN: Is NodeJS stable enough to build the next Twitter?

#32

Post by don_draper »

Torn wrote: post #19
russell_h wrote: post #16

We've built the control plane to Rackspace Cloud Monitoring in Node.js, and overall the experience has been positive. A few things to look out for: 1. Error handling. In node you can't just wrap your code in a try/catch. And even if you register a listener for uncaught exceptions, you almost certainly have state shared between requests (for example, a database connection pool), which makes trying to handle such excep…

Have you tried a Promises implementation like deferred https://github.com/medikoo/deferred to manage callback spaghetti?

Or q? And wouldn't using promises help handle exceptions?

Re: Ask HN: Is NodeJS stable enough to build the next Twitter?

#33

Post by vemv »

mh- wrote: post #14
weego wrote: post #7

Whatever you write now would never scale to be Twitter as it is now, there's no point even considering that as you aren't thinking on anything like that scale conceptually. I doubt most developers can. But then writing something that could scale to Twitter scale when you don't have a business or users or revenue would be pointless. Specifically to answer you question, no. Node could work as a thin publishing veneer o…

Node could work as a thin publishing veneer on a much larger stack but you just don't get what you need from Node.js end-to-end. I'd be interested in you expanding on that claim. I'm indifferent towards node, but this is my area of concern. I don't see anything inherent to node/v8/js that would be limiting.

    I don't see anything inherent to node/v8/js that would be limiting.
The concurrency model.

Re: Ask HN: Is NodeJS stable enough to build the next Twitter?

#34

Post by js4all »

I am also working on a similar project using node.js and redis. Please make sure that you are using a perfectly fitting data model. I bet most of the load will come from the database and not from the application servers. We went for an in-memory solution for all timelines and graphs.

Re: Ask HN: Is NodeJS stable enough to build the next Twitter?

#35

Post by fooyc »

Node.js itself is fine.

The real problem with node.js is the libraries. Just don't use them.

A huge portion of existing libraries is full of hidden bugs, shortcomings, race conditions, edge cases, security issues, unscalable, or unmaintainable (and unmaintained).

This is exacerbated by the fact that npm makes it really easy to publish a library.

Many small buggy libraries.

Core modules are too low level (e.g. http), and you really don't want to use an overlay library.

Not to mention that doing something not trivial fully asynchronously is not as fun as it sounds. You will spend a significant time tracking bugs, fixing edge cases, and making your code stable.

There is still no way to make async code better in core (no promises); and there are a handful of incompatible promises implementations.

Oh, and node.js is not really fine actually. It's not doing everything using asynchronous I/O as you would expect. Node.js uses a thread pool for things like DNS resolution and disk I/O. Only 4 threads by default, for all those things with very different latencies. This means that 4 DNS queries can occupy node.js's 4 hidden worker threads, and block your disk I/O for minutes.

Re: Ask HN: Is NodeJS stable enough to build the next Twitter?

#36

Post by danneu »

mh- wrote: post #9

I'm not sure how you're defining 'stable' here, but I'll comment on another aspect of your concern. Twitter started with Rails, and at some point decided it was more efficient to do an incremental rewrite on the JVM. I can rewrite the entire backend in a weekend in Python. I'd bet it took Twitter a bit longer to replace their infrastructure, and they survived just fine. Build in whatever is rapid, for you. At this ve…

    > cromulent
I had to look that word up on urbandictionary, wiktionary, and tvtropes (apparently a Simpsons reference).

From what I gather, if that's the word you meant, then it doesn't seem that you believe Node.js is a viable platform. By default, of course, I'll assume the expression was just beyond me!

Re: Ask HN: Is NodeJS stable enough to build the next Twitter?

#38

Post by masterkain »

mathrawka wrote: post #31
masterkain wrote: post #30

Earlier quoted context omitted.

Domains are 0.10 specific I believe, and 0.10 is still bugged in few areas. I'd stay on 0.8.23 for quite some time since we are talking about stability.

I'm using 0.10 in production and haven't had any bugs... what bugs are you referring to? Also domains are in 0.8

Is yours a large or small app? Do you use streaming/piping/remote services?

Regarding bugs, there are too many to list, just see the tracker and spend some time browsing through them https://github.com/joyent/node/issues?direction=desc&pag... I'm fairly sure you can find pretty much anything.

I'm considering removing all the npm modules from my project and go raw, since most of the obscure bugs are in the ecosystem anyway.

Somehow I missed domains are also in 0.8, thanks!

Re: Ask HN: Is NodeJS stable enough to build the next Twitter?

#39

Post by olegp »

russell_h wrote: post #16

We've built the control plane to Rackspace Cloud Monitoring in Node.js, and overall the experience has been positive. A few things to look out for: 1. Error handling. In node you can't just wrap your code in a try/catch. And even if you register a listener for uncaught exceptions, you almost certainly have state shared between requests (for example, a database connection pool), which makes trying to handle such excep…

We are using Node at https://starthq.com and I agree with most of the points above.

Error handling is the major issue, because you need to handle all errors manually, i.e. you can't use try catch to trap all errors further down the stack. If you don't handle all errors your Node process will terminate and you may lose some state. Even if you're confident in the stability of your code, I strongly advise that you use a watchdog process like supervisor to start a new process if the current one terminates.

We've handled this issue and kept business logic code simple by using https://github.com/olegp/common-node which uses fibers to present synchronous APIs, allowing us to use exceptions for error handling.

Be very careful when choosing third party packages, since if they don't handle all errors, again your process will terminate and there's nothing you're able to do about it, even if you're using fibers.

One last issue is changes to the core APIs. Since some of them are still in flux, it is advisable to provide an abstraction layer above them so as to be able to weather any changes. For example when streams2 came out, we only needed to upgrade Common Node, with no changes to the application itself.

Re: Ask HN: Is NodeJS stable enough to build the next Twitter?

#40

Post by CoryG89 »

mh- wrote: post #14
weego wrote: post #7

Whatever you write now would never scale to be Twitter as it is now, there's no point even considering that as you aren't thinking on anything like that scale conceptually. I doubt most developers can. But then writing something that could scale to Twitter scale when you don't have a business or users or revenue would be pointless. Specifically to answer you question, no. Node could work as a thin publishing veneer o…

Node could work as a thin publishing veneer on a much larger stack but you just don't get what you need from Node.js end-to-end. I'd be interested in you expanding on that claim. I'm indifferent towards node, but this is my area of concern. I don't see anything inherent to node/v8/js that would be limiting.

There are definitely going to be things that Node is slower for, and others that it excels at.. just like everything else. I would recommend anyone that is wondering about this to check out this great presentation given by Ryan Dahl, the creator of Node, in which it talks about the concurrency model, how it is acheived, and what the consequences of this are. The short story is that Node performs well under high concurrency and and IO bound workload with lots of small files. It is not going to excel so much for a computationally bounded or at serving huge files. It's always going to be about knowing which is the best technology for the job at hand.

http://nodejs.org/jsconf2010.pdf

Post reply on HN