Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On problems with threads in Node.js

future-processing.pl

Post reply on HN
Print view View original

31–40 of 66 posts

Re: On problems with threads in Node.js

#31

Post by lordbusiness »

eklavya wrote: post #25
lordbusiness wrote: post #16

Earlier quoted context omitted.

Do you realize that comments like this are what is breaking the community here at HN? Are you aware that people like you are destroying something that was once brilliant? HN is going through an Eternal September.

Calm down man, nobody's breaking anything. Everyone is entitled to his/her opinion. You can express your thinking on the matter by up/down voting. Relax :)

Heh - maybe you're right, and I wish the world were as you describe. :-)

There is however a deeper underlying issue; decorum is important and communities that exhibit genuine 'niceness' are nice. Communities that allow, or worse, overlook dark behaviour degenerate.

Flagging and down voting is one part of the solution, but when the nastiness reaches a level that the nice people start to disengage and go elsewhere, it's clear to me that we need another element of control. Perhaps algorithmically detecting repeat offenders? Perhaps more granularity with down votes?

There's are differences between a down vote because one disagrees with the author, and a down vote because one believes the author is ill-informed and spreading misinformation, and a down vote because the author is being downright juvenile.

A number of hits on the third case against a given author on multiple comments could conceivably constitute an automatic warning and / or banning system.

I don't want people to be unable to express their views, but when the mean-spirited people who contribute nothing but nonsense start to represent a large percentage of a community, it's reasonable to see if anything can be done.

Re: On problems with threads in Node.js

#32

Post by inquisitiveio »

albertzeyer wrote: post #23

Is there a detailed overview about which functions in libuv (Nodejs) rely on blocking primitives and thus are using the thread pool to work async? From [here]( http://docs.libuv.org/en/latest/design.html ), it sounds like all file IO is always based on blocking primitives, and native async file IO primitives are not used, although such async file IO primitives do exists and were tried out in libtorrent ( http://blog.…

The article you link to touches on a few of the Linux AIO shortcomings.

But if you have also read the following blunt criticism from Linus http://yarchive.net/comp/linux/o_direct.html he also outlines a better alternative way of implementing asynchronous disk io on Linux at least.

Re: On problems with threads in Node.js

#33

Post by 72deluxe »

lordbusiness wrote: post #31
eklavya wrote: post #25

Earlier quoted context omitted.

Calm down man, nobody's breaking anything. Everyone is entitled to his/her opinion. You can express your thinking on the matter by up/down voting. Relax :)

Heh - maybe you're right, and I wish the world were as you describe. :-) There is however a deeper underlying issue; decorum is important and communities that exhibit genuine 'niceness' are nice. Communities that allow, or worse, overlook dark behaviour degenerate. Flagging and down voting is one part of the solution, but when the nastiness reaches a level that the nice people start to disengage and go elsewhere, it'…

Perhaps the should split the upvote/downvote buttons into those 3 categories? I know the downvoting because of disagreement is a really really annoying state of affairs, particularly when they don't tell you WHY they disagree but instead just downvote you.

Re: On problems with threads in Node.js

#34

Post by simpleigh »

pdpi wrote: post #6
babuskov wrote: post #5

To sum it up: node.js runs only 4 background threads. Be aware of this. No big deal really. I have multiple servers running node.js under load for 3+ years and never had an issue with this. In fact, I found it helpful. If your database is under load and is already running 4 heavy queries, not giving it any more jobs is actually a good thing.

On the other hand, not being able to hit the filesystem because you have those four queries running is... not so helpful. In general, this looks like it could be a performance bottleneck for highly-concurrent applications.

I agree with this - the right place to restrict the number of concurrent database queries is the database, not the whole IO layer!

Re: On problems with threads in Node.js

#35

Post by spion »

Not sure if it was intentional, but the article is quite misleading.

The thread pool in node is only used for a limited number of APIs. Pretty much all networking uses native async IO and is unaffected by the size of the thread pool. Things like Oracle's driver are rare exceptions: the typical MySQL/PostgreSQL/redis etc drivers all use native async IO and are unaffected by this.

The author only glosses over this briefly. As a result this article leaves the impression that the problem described is the norm, which is not the case.

Re: On problems with threads in Node.js

#36

Post by amelius »

babuskov wrote: post #5

To sum it up: node.js runs only 4 background threads. Be aware of this. No big deal really. I have multiple servers running node.js under load for 3+ years and never had an issue with this. In fact, I found it helpful. If your database is under load and is already running 4 heavy queries, not giving it any more jobs is actually a good thing.

> If your database is under load and is already running 4 heavy queries, not giving it any more jobs is actually a good thing.

User-interfaces 101: don't sacrifice the latency of short-lived jobs for finishing lengthy jobs.

Also, let the OS figure it out. That's what it was made for.

Re: On problems with threads in Node.js

#38

Post by jrochkind1 »

justincormack wrote: post #22
imaginenore wrote: post #17

4 seems like a ridiculously low default size for the thread pool.

It is only for file system operations, and mostly these do not actually block, as the results are available in cache. So it is probably reasonable for casual use. Now if you want to get good performance on an SSD (ie the rated iops) you will need a decent queue depth, like 32 or so, so it wont work but thats a specialist use case.

SSD's are a specialist use case these days? Or getting good performance on them is? What do you mean?

Re: On problems with threads in Node.js

#39

Post by laggyluke »

vinceyuan wrote: post #37

Does it mean the async functions about file system operations are not really asynchronous in Node.js? The whole node.js server will be blocked if the number of file system operations is bigger than the thread pool size. :-(

Not the whole server - most of the networking IO will work just fine, only the operations that use libuv thread pool will be queued.

Re: On problems with threads in Node.js

#40

Post by paraboul »

eknkc wrote: post #20
laggyluke wrote: post #9

It's worth noting that DB drivers that actually integrate with libuv are a minority - most DB divers use Node-level network APIs and are unaffected by such thread pool limits.

Why would a DB driver need to integrate with libuv in the first place?

In the case where the driver (for instance the official libmysql) doesn't expose its network interface (e.g. its sockets can't be used with an external event loop). In that case, it would block on IO operation, and there is no other choice but to push it to another thread.

That said, using a driver in another thread leads to various complexities that can be avoided when running in the same application (javascript/v8) thread/event loop.

Post reply on HN