Live data from Hacker News

Dyad: Minimal, portable async networking library for C

github.com

21–30 of 47 posts

Re: Dyad: Minimal, portable async networking library for C

#21
post #8

Very neat! Although after using C#'s TAP, callbacks sound like hell :)

Callbacks can be avoided using co-routines. I started a project that uses asynchronous sockets but they appear synchronous. With a bit of abstraction, it's really easy to develop network applications even in C: https://github.com/reginaldl/librinoo

Your approach looks very interesting - the examples are definitely much easier to read and understand over most async C code I have seen. How does it work internally?

Re: Dyad: Minimal, portable async networking library for C

#22
post #8

Very neat! Although after using C#'s TAP, callbacks sound like hell :)

Callbacks can be avoided using co-routines. I started a project that uses asynchronous sockets but they appear synchronous. With a bit of abstraction, it's really easy to develop network applications even in C: https://github.com/reginaldl/librinoo

Didn't find this one before.

I wrote my own user-space-threading library called libwire. Mostly for not liking the malloc-everywhere approach so common everywhere. The tradeoffs are different and the code is more verbose at times but I like the fact that there are no mallocs in code, at least not very explicit ones. I do provided a memory pool so it allocates memory but it is bounded in size.

libwire: https://github.com/baruch/libwire

list of coroutine/user-space-threading libraries: https://github.com/baruch/libwire/wiki/Other-coroutine-libra...

Re: Dyad: Minimal, portable async networking library for C

#23

Earlier quoted context omitted.

How are you supposed to do socket programming without select?

poll is fairly portable, but still has scaling problems. To get beyond scaling problems, you usually need something OS-specific, such as epoll (Linux), kqueue (BSDs of some kind), etc. One of the things I'd expect of a cross-platform networking library would be to use the best available abstraction, especially on major platforms such as linux, and perhaps fall back to select on odd platforms. Of course, your docs nee…

yeah...epoll and kqueue in my experience are easily interchangeable. I built a server on FreeBSD and the port to Linux was straightforward. My first event-based socket usage was on Windows NT around 1999. When we ported the server parts to Linux, replacing with epoll was also straightforward.

Re: Dyad: Minimal, portable async networking library for C

#24
post #8

Earlier quoted context omitted.

Callbacks can be avoided using co-routines. I started a project that uses asynchronous sockets but they appear synchronous. With a bit of abstraction, it's really easy to develop network applications even in C: https://github.com/reginaldl/librinoo

Your approach looks very interesting - the examples are definitely much easier to read and understand over most async C code I have seen. How does it work internally?

All of the different coroutine/user-space-threads work by switching the stack and the the instruction pointer when the code is about to block on an async system call. So one would find that to read from the socket would block (by getting EWOULDBLOCK in errno) register to wait for this fd to become readable and switch to another coroutine. There is always one coroutine that gets scheduled from time to time and users select/poll/epoll to get all the fds that got data in them and wake up and schedule the coroutines that are waiting on each of these fds.

It's a very neat concept that allows for cleaner code (sequential rather than callback-hell) and use one or a few threads for many actions but without the overhead of thread-per-socket.

Re: Dyad: Minimal, portable async networking library for C

#25
post #8

Earlier quoted context omitted.

Callbacks can be avoided using co-routines. I started a project that uses asynchronous sockets but they appear synchronous. With a bit of abstraction, it's really easy to develop network applications even in C: https://github.com/reginaldl/librinoo

Your approach looks very interesting - the examples are definitely much easier to read and understand over most async C code I have seen. How does it work internally?

It uses "user-space threads" similar to ucontext (swapcontext(3)). I have my own version: https://github.com/reginaldl/fcontext

Once the stack allocated for a thread, context switches are almost as cheap as a function call.

RiNOO has an event driven scheduler, based on epoll, which resume/release these user-space threads (that I call tasks) according to pending IOs. The library provides with IO functions (read, write...) which use the RiNOO scheduler.

As a bonus, real threading is quite easy: just need to run a scheduler per thread (see examples with multi-threading).

Re: Dyad: Minimal, portable async networking library for C

#27
post #25

Earlier quoted context omitted.

Your approach looks very interesting - the examples are definitely much easier to read and understand over most async C code I have seen. How does it work internally?

It uses "user-space threads" similar to ucontext (swapcontext(3)). I have my own version: https://github.com/reginaldl/fcontext Once the stack allocated for a thread, context switches are almost as cheap as a function call. RiNOO has an event driven scheduler, based on epoll, which resume/release these user-space threads (that I call tasks) according to pending IOs. The library provides with IO functions (read, write…

That's really interesting - thanks.

Re: Dyad: Minimal, portable async networking library for C

#28
post #24

Earlier quoted context omitted.

Your approach looks very interesting - the examples are definitely much easier to read and understand over most async C code I have seen. How does it work internally?

All of the different coroutine/user-space-threads work by switching the stack and the the instruction pointer when the code is about to block on an async system call. So one would find that to read from the socket would block (by getting EWOULDBLOCK in errno) register to wait for this fd to become readable and switch to another coroutine. There is always one coroutine that gets scheduled from time to time and users s…

Isn't it effectively thread-per-socket except that the switches now only happen on an async system call? The difference, I guess, is you can optimize the context switch to be very small (and have smaller thread stacks) at the cost of giving up pre-emptive switching.

Re: Dyad: Minimal, portable async networking library for C

#30
post #24

Earlier quoted context omitted.

All of the different coroutine/user-space-threads work by switching the stack and the the instruction pointer when the code is about to block on an async system call. So one would find that to read from the socket would block (by getting EWOULDBLOCK in errno) register to wait for this fd to become readable and switch to another coroutine. There is always one coroutine that gets scheduled from time to time and users s…

Isn't it effectively thread-per-socket except that the switches now only happen on an async system call? The difference, I guess, is you can optimize the context switch to be very small (and have smaller thread stacks) at the cost of giving up pre-emptive switching.

It reduces context switches in the kernel which are more expensive and reduces the memory resources needed for kernel threads which are larger.

There are also some hidden gains in terms of TLB caches and other costs of kernel threads switching.

An additional advantage is that between user-space-threads you have fewer locking problem since they implicitly lock out each other between context switch points so you only need locks when you need to protect an area across several context switch points.

Post reply on HN