Live data from Hacker News

On problems with threads in Node.js

future-processing.pl

51–60 of 66 posts

Re: On problems with threads in Node.js

#51
post #35

Not sure if it was intentional, but the article is quite misleading. The thread pool in node is only used for a limited number of APIs. Pretty much all networking uses native async IO and is unaffected by the size of the thread pool. Things like Oracle's driver are rare exceptions: the typical MySQL/PostgreSQL/redis etc drivers all use native async IO and are unaffected by this. The author only glosses over this brie…

[deleted]

Re: On problems with threads in Node.js

#52
post #35

Not sure if it was intentional, but the article is quite misleading. The thread pool in node is only used for a limited number of APIs. Pretty much all networking uses native async IO and is unaffected by the size of the thread pool. Things like Oracle's driver are rare exceptions: the typical MySQL/PostgreSQL/redis etc drivers all use native async IO and are unaffected by this. The author only glosses over this brie…

> Pretty much all networking uses native async IO and is unaffected by the size of the thread pool

Are they still using threads to get the "magic" working? I'm referring to this sentence: "But how did that happen? To the best of my knowledge node.js, is not powered by magic and fairy dust and things don’t just get done on their own."

Re: On problems with threads in Node.js

#53
post #52
post #35

Not sure if it was intentional, but the article is quite misleading. The thread pool in node is only used for a limited number of APIs. Pretty much all networking uses native async IO and is unaffected by the size of the thread pool. Things like Oracle's driver are rare exceptions: the typical MySQL/PostgreSQL/redis etc drivers all use native async IO and are unaffected by this. The author only glosses over this brie…

> Pretty much all networking uses native async IO and is unaffected by the size of the thread pool Are they still using threads to get the "magic" working? I'm referring to this sentence: "But how did that happen? To the best of my knowledge node.js, is not powered by magic and fairy dust and things don’t just get done on their own."

Not magic, but APIs like these: https://en.wikipedia.org/wiki/Epoll https://en.wikipedia.org/wiki/Kqueue

Pretty much all event loop based programs work the same way: instead of blocking on a single request for IO, they use system calls (e.g. epool_wait) that block until any of the many descriptors (sockets) has some event (data to be read, client connecting, etc). It gets a bit complicated when there are queued tasks for the thread pool and timers involved too, but its the same principle.

Re: On problems with threads in Node.js

#54
post #35

Not sure if it was intentional, but the article is quite misleading. The thread pool in node is only used for a limited number of APIs. Pretty much all networking uses native async IO and is unaffected by the size of the thread pool. Things like Oracle's driver are rare exceptions: the typical MySQL/PostgreSQL/redis etc drivers all use native async IO and are unaffected by this. The author only glosses over this brie…

It's a completely unscientific method, but searching through one of our large applications (`npm ls|wc -l` -> ~2000 dependencies), the only modules I can find using `uv_queue_work` are: * kerberos, unused (dependency of mongodb) * protobuf, for serializing data * snappy, for compression kerberos isn't actually used in our app, so it doesn't matter, but we send a lot of data through protobuf and snappy, so it may be w…

You can also experiment with different values for the env variable `UV_THREADPOOL_SIZE`; last time I checked this can even be set from the JS code via `process.env['UV_THREADPOOL_SIZE']` if you make sure to do it before you call something that uses the threadpool.

Re: On problems with threads in Node.js

#55
post #43

I actually ran into an issue recently with CPU intensive tasks blocking my web server. It turns out that "querystring" (used to parse request bodies in web applications) is an asynchronous, blocking request. You'd never notice much slowness, until your request bodies are massive (think 50 nested JSON objects and some base64 image data for good measure) and you have multiple per second. Now, every request is blocked u…

I always use the following replacements written by petkaanotonov, the author of bluebird :) https://www.npmjs.com/package/querystringparser for query string parsing. Depending on content, you may get massive improvements (5x-20x) https://www.npmjs.com/package/fast-url-parser for url parsing (the built in url parser is the main reason why node is so far behind on the TechEmpower benchmark - with this replacement the b…

These look promising, thanks!

Re: On problems with threads in Node.js

#56
post #25

Earlier quoted context omitted.

Calm down man, nobody's breaking anything. Everyone is entitled to his/her opinion. You can express your thinking on the matter by up/down voting. Relax :)

Heh - maybe you're right, and I wish the world were as you describe. :-) There is however a deeper underlying issue; decorum is important and communities that exhibit genuine 'niceness' are nice. Communities that allow, or worse, overlook dark behaviour degenerate. Flagging and down voting is one part of the solution, but when the nastiness reaches a level that the nice people start to disengage and go elsewhere, it'…

I would down vote much more often if it weren't so ambiguous. If I had more options. Sometimes I want to down vote just 0.1, just to say Walter, you're not wrong: you're just an asshole.

I also think pointed, honest replies like yours above go a long way. The best way to get people to assume good faith is to show it. (Reyk's Second Law)

Re: On problems with threads in Node.js

#57
post #53
post #52

Earlier quoted context omitted.

> Pretty much all networking uses native async IO and is unaffected by the size of the thread pool Are they still using threads to get the "magic" working? I'm referring to this sentence: "But how did that happen? To the best of my knowledge node.js, is not powered by magic and fairy dust and things don’t just get done on their own."

Not magic, but APIs like these: https://en.wikipedia.org/wiki/Epoll https://en.wikipedia.org/wiki/Kqueue Pretty much all event loop based programs work the same way: instead of blocking on a single request for IO, they use system calls (e.g. epool_wait) that block until any of the many descriptors (sockets) has some event (data to be read, client connecting, etc). It gets a bit complicated when there are queued tasks…

Don't most computer have separate processors for network, hard drive, etc. So, even if you have a single core processor, you are still running a multi-processor environment? Can anyone give me details on this? Someone told me something like this once and it has confused me ever since...

Re: On problems with threads in Node.js

#58
post #2

Piece of badly engineered software conquering the world. As if more JavaScript was really something worth pursuing.

Do you realize that comments like this are what is breaking the community here at HN? Are you aware that people like you are destroying something that was once brilliant? HN is going through an Eternal September.

You mean destroying the echo chamber that is HN?

Re: On problems with threads in Node.js

#59
post #35

Not sure if it was intentional, but the article is quite misleading. The thread pool in node is only used for a limited number of APIs. Pretty much all networking uses native async IO and is unaffected by the size of the thread pool. Things like Oracle's driver are rare exceptions: the typical MySQL/PostgreSQL/redis etc drivers all use native async IO and are unaffected by this. The author only glosses over this brie…

Aren't there some commonly used system calls that don't have asynchronous equivalents, such as open() or stat() or access()? Are threads used for those?
Post reply on HN