Lthread is a multicore/multithread coroutine library written in C • Hacker News Board

Re: Lthread is a multicore/multithread coroutine library written in C

#21

Post by loeg » Mon, Apr 23, 2012, 8:34 AM UTC

wsc981 wrote: post #17

I wonder how this library compares agains Apple's Grand Central Dispatch. I guess they don't contain 1:1 functionality, but both libraries seem to be meant to make threading easier. I guess GCD is a more complete solution due to the addition of queues.

They're different beasts entirely. One key distinction is that GCD requires kernel support, while lthread and friends simply sit on top of any pthread-compatible system. As I understand it, GCD is based on an old systems idea (1991):

https://en.wikipedia.org/wiki/Scheduler_activations

It also has some auxiliary stuff (compiler-level support for C closures (Clang blocks), etc…) to "make threading easier," like you say.

Re: Lthread is a multicore/multithread coroutine library written in C

#22

Post by tomp » Mon, Apr 23, 2012, 8:44 AM UTC

loeg wrote: post #5

So, last time this was posted I commented that I was interested, but couldn't use this at work because of the GPL license. Since then I implemented my own M:N userspace:kernel threading library (which is what lthread boils down to) based on Russ Cox's BSD-licensed libtask. You know what I found? 1) Many libc routines require a surprisingly large amount of stack. 64kiB was the smallest power of 2 I could find where I…

Since you can consider libc routines as blocking code, you could switch back to the C stack (of the original pthread), so you would need only one 64kB stack per pthread.

Re: Lthread is a multicore/multithread coroutine library written in C

#23

Post by alpb » Mon, Apr 23, 2012, 9:06 AM UTC

This is previously submitted to HN several times. http://news.ycombinator.com/item?id=3331474

Re: Lthread is a multicore/multithread coroutine library written in C

#24

Post by pm215 » Mon, Apr 23, 2012, 10:43 AM UTC

The trouble with this sort of C coroutine library in my experience is portability. Lthread seems to have taken the "write a bit of inline asm" approach, which means it's x86/x86-64 only. If you avoid the assembly and try to do things only with C library functions, you end up with something like QEMU's coroutines, which have four different backends: makecontext/setcontext based, win32 fibers, the nasty sigaltstack trick used by GNU Pth[1], and a last-ditch fallback using a separate GThread per coroutine. Multiple backends means more code, and the less-used backends are more liable to bitrot and undetected bugs, which is the last thing you want in a key bit of infrastructure.

Also some libc implementations don't take kindly to programs messing with the stack pointer behind their backs. Early Linux NPTL implementations put thread-local-storage just above the stack and used "round ESP up to 2MB boundary" to access it, which meant you had to switch back to the libc-created thread stack before calling just about any libc function. I hear at least one of the BSDs still does something similar.

All things considered I'd really rather just use threads...

[1] http://www.gnu.org/software/pth/rse-pmt.ps

Re: Lthread is a multicore/multithread coroutine library written in C

#25

Post by Hoff » Mon, Apr 23, 2012, 11:16 AM UTC

Any thoughts on this HP research paper

http://www.hpl.hp.com/techreports/2004/HPL-2004-209.html

"Threads Cannot be Implemented as a Library"?

That paper states that library-based threading implementations that don't also involve the compiler can't guarantee correctness of the resulting threading.

Re: Lthread is a multicore/multithread coroutine library written in C

#26

Post by qznc » Mon, Apr 23, 2012, 11:41 AM UTC

Hoff wrote: post #25

Any thoughts on this HP research paper http://www.hpl.hp.com/techreports/2004/HPL-2004-209.html "Threads Cannot be Implemented as a Library"? That paper states that library-based threading implementations that don't also involve the compiler can't guarantee correctness of the resulting threading.

Not relevant here.

The paper talks about correctness for parallel kernel-level threads. In contrast, lthreads are about concurrency. These user-level threads are about software architecture, because they are more elegant than an event-loop. Parallelism (exploiting multicore hardware for a speedup) is a secondary goal, but it is much easier to parallelize a user-level thread program compared to an event loop program.