Is there an easy way to identify the number of tasks queued in uv's work queue?
On problems with threads in Node.js
21–30 of 66 posts
Re: On problems with threads in Node.js
#224 seems like a ridiculously low default size for the thread pool.
It is only for file system operations, and mostly these do not actually block, as the results are available in cache. So it is probably reasonable for casual use.
Now if you want to get good performance on an SSD (ie the rated iops) you will need a decent queue depth, like 32 or so, so it wont work but thats a specialist use case.
Re: On problems with threads in Node.js
#23Is there a detailed overview about which functions in libuv (Nodejs) rely on blocking primitives and thus are using the thread pool to work async?
From [here](http://docs.libuv.org/en/latest/design.html), it sounds like all file IO is always based on blocking primitives, and native async file IO primitives are not used, although such async file IO primitives do exists and were tried out in libtorrent (http://blog.libtorrent.org/2012/10/asynchronous-disk-io/). The result of that experiment however was mostly that the thread pool solution was simpler to code (I guess).
From the libuv design doc, the overview is:
* Filesystem operations
* DNS functions (getaddrinfo and getnameinfo)
* User specified code via uv_queue_work()
I wonder whether this is really the best solution of if some combination of a thread pool and native async disk IO primitives could perform better.
Re: On problems with threads in Node.js
#24It's worth noting that DB drivers that actually integrate with libuv are a minority - most DB divers use Node-level network APIs and are unaffected by such thread pool limits.
Why would a DB driver need to integrate with libuv in the first place?
To be evented and get the respective performance gains in a Node setting?
Re: On problems with threads in Node.js
#25Piece of badly engineered software conquering the world. As if more JavaScript was really something worth pursuing.
Do you realize that comments like this are what is breaking the community here at HN? Are you aware that people like you are destroying something that was once brilliant? HN is going through an Eternal September.
Calm down man, nobody's breaking anything. Everyone is entitled to his/her opinion. You can express your thinking on the matter by up/down voting. Relax :)
Re: On problems with threads in Node.js
#26Is there a detailed overview about which functions in libuv (Nodejs) rely on blocking primitives and thus are using the thread pool to work async? From [here]( http://docs.libuv.org/en/latest/design.html ), it sounds like all file IO is always based on blocking primitives, and native async file IO primitives are not used, although such async file IO primitives do exists and were tried out in libtorrent ( http://blog.…
[deleted]
Re: On problems with threads in Node.js
#27[deleted]
Re: On problems with threads in Node.js
#28Is there a detailed overview about which functions in libuv (Nodejs) rely on blocking primitives and thus are using the thread pool to work async? From [here]( http://docs.libuv.org/en/latest/design.html ), it sounds like all file IO is always based on blocking primitives, and native async file IO primitives are not used, although such async file IO primitives do exists and were tried out in libtorrent ( http://blog.…
> The result of that experiment however was mostly that the thread pool solution was simpler to code (I guess).
and uniformly asynchronous (native async operations may not cover e.g. file copy or filesystem operations, furthermore filesystems may block during submission of IO ops which makes the operations effectively synchronous) and have higher throughput (they support read/write vectors).
Re: On problems with threads in Node.js
#29Is there a detailed overview about which functions in libuv (Nodejs) rely on blocking primitives and thus are using the thread pool to work async? From [here]( http://docs.libuv.org/en/latest/design.html ), it sounds like all file IO is always based on blocking primitives, and native async file IO primitives are not used, although such async file IO primitives do exists and were tried out in libtorrent ( http://blog.…
[deleted]
Yes, just read that (the libtorrent post). I somehow feel that this is a sad state. Shouldn't we try to improve the disk AIO API then, if it is not useful at the moment? Or maybe that has changed also? The article is from 2012. Or maybe we could at least partly use it? Of course, code complexity will be a problem then in any case. But for applications depending on a lot of concurrent disk IO, maybe it could be worth it.
Re: On problems with threads in Node.js
#30Is there a detailed overview about which functions in libuv (Nodejs) rely on blocking primitives and thus are using the thread pool to work async? From [here]( http://docs.libuv.org/en/latest/design.html ), it sounds like all file IO is always based on blocking primitives, and native async file IO primitives are not used, although such async file IO primitives do exists and were tried out in libtorrent ( http://blog.…
> The result of that experiment however was mostly that the thread pool solution was simpler to code (I guess). and uniformly asynchronous (native async operations may not cover e.g. file copy or filesystem operations, furthermore filesystems may block during submission of IO ops which makes the operations effectively synchronous) and have higher throughput (they support read/write vectors).
We could use the thread pool for blocking primitives and otherwise use the native AIO primitives, couldn't we?
And the higher throughput seemed only to be a problem on MacOSX, so we could fallback to the thread pool there, but use the async IO on Windows and Linux.