Live data from Hacker News

Dyad: Minimal, portable async networking library for C

github.com

11–20 of 47 posts

Re: Dyad: Minimal, portable async networking library for C

#11
dyad_writef is certainly not 'async' when using the 'r' input specifier (which seems to redirect input from a FILE* to the output stream). And the way dyad_writef achieves 'async' is to buffer the entire contents of the output in RAM.

So for 'r', why bother using the FILE* formatted IO if all you're going to do is pull single characters at a time? You can just use file descriptors.

Re: Dyad: Minimal, portable async networking library for C

#12

It uses select, which is notorious for poor performance (although is relatively portable). Try not to use this project for anything high-load.

How are you supposed to do socket programming without select?

> http://linux.die.net/man/7/socket

create a socket, bind, listen, connect, send etc.

Why do you need select?

Re: Dyad: Minimal, portable async networking library for C

#15
post #12

Earlier quoted context omitted.

How are you supposed to do socket programming without select?

> http://linux.die.net/man/7/socket create a socket, bind, listen, connect, send etc. Why do you need select?

You need select or a select-like function to know which sockets are readable/writable without tying up one thread per socket

Re: Dyad: Minimal, portable async networking library for C

#16
post #15
post #12

Earlier quoted context omitted.

> http://linux.die.net/man/7/socket create a socket, bind, listen, connect, send etc. Why do you need select?

You need select or a select-like function to know which sockets are readable/writable without tying up one thread per socket

I know what select/epoll and friends are for.

What's wrong with tying up one thread per socket?

Re: Dyad: Minimal, portable async networking library for C

#17
post #16
post #15

Earlier quoted context omitted.

You need select or a select-like function to know which sockets are readable/writable without tying up one thread per socket

I know what select/epoll and friends are for. What's wrong with tying up one thread per socket?

"Context switching is expensive. My rule of thumb is that it'll cost you about 30µs of CPU overhead. This seems to be a good worst-case approximation. Applications that create too many threads that are constantly fighting for CPU time (such as Apache's HTTPd or many Java applications) can waste considerable amounts of CPU cycles just to switch back and forth between different threads."

http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-ma...

On 32-bit systems, you can also easily run out of address space for your thread stacks.

Re: Dyad: Minimal, portable async networking library for C

#18

It uses select, which is notorious for poor performance (although is relatively portable). Try not to use this project for anything high-load.

How are you supposed to do socket programming without select?

this competes with libevent, libev, and libuv... all of which use the best method for the platform where it's installed.. so kqueue on BSD, epoll on Linux, etc.

That's one of the big reasons to use a lib for this.. so you get the best performance, without having to change your code to get it (or bother detecting which is best, etc).

Re: Dyad: Minimal, portable async networking library for C

#19
post #17
post #16

Earlier quoted context omitted.

I know what select/epoll and friends are for. What's wrong with tying up one thread per socket?

"Context switching is expensive. My rule of thumb is that it'll cost you about 30µs of CPU overhead. This seems to be a good worst-case approximation. Applications that create too many threads that are constantly fighting for CPU time (such as Apache's HTTPd or many Java applications) can waste considerable amounts of CPU cycles just to switch back and forth between different threads." http://blog.tsunanet.net/2010/1…

I don't trust those measurements. They never fully explored what is due to cache effects and what is due to context switching. You'd have heavy cache effects if you switched data between client contexts after select returns, too. So just because spinning on a futex show a max 30us wasted CPU doesn't mean you won't waste that with select as well.

Also in case of an IO bound thread, they are not just spinning on a futex aimlessly. There is a different mechanism, so should really benchmark with a more characteristic workload.

Speaking of characteristic workload, they should have probably also measured on a tickless kernel since I saw they complained about time quanta and HZ=100. Well recent kernels are tickless so they'll behave differently. (Might be worse even).

> On 32-bit systems, you can also easily run out of address space for your thread stacks.

Well don't run large servers with so many threads on 32 bit systems ;-) Many database vendors don't even package for or support 32 bit versions of Linux.

Sorry, I haven't bought into the whole "async is always better" trend. Some (ex?) Senior Google engineer (Paul Tyma) agrees with me:

http://www.mailinator.com/tymaPaulMultithreaded.pdf

Async / select pattern is usually good where there is very little business logic. Like a router, proxy or simple web server and so on. In a large application having a giant dispatch call at the center of it, with callbacks branching out is not a healthy pattern.

Re: Dyad: Minimal, portable async networking library for C

#20
post #5

It uses select, which is notorious for poor performance (although is relatively portable). Try not to use this project for anything high-load.

And anything that requires more than 1024 fds...

You can increase the define if you need more than 1024 fds but then the performance is getting very poor.
Post reply on HN