When I saw this repost, I thought the license had changed. Too bad it hasn't yet.
What's wrong with the BSD license?
Lthread is a multicore/multithread coroutine library written in C
11–20 of 26 posts
Re: Lthread is a multicore/multithread coroutine library written in C
#12Re: Lthread is a multicore/multithread coroutine library written in C
#13So, last time this was posted I commented that I was interested, but couldn't use this at work because of the GPL license. Since then I implemented my own M:N userspace:kernel threading library (which is what lthread boils down to) based on Russ Cox's BSD-licensed libtask. You know what I found? 1) Many libc routines require a surprisingly large amount of stack. 64kiB was the smallest power of 2 I could find where I…
Out of interest, what implementation of libc was that?
Re: Lthread is a multicore/multithread coroutine library written in C
#14So, last time this was posted I commented that I was interested, but couldn't use this at work because of the GPL license. Since then I implemented my own M:N userspace:kernel threading library (which is what lthread boils down to) based on Russ Cox's BSD-licensed libtask. You know what I found? 1) Many libc routines require a surprisingly large amount of stack. 64kiB was the smallest power of 2 I could find where I…
Out of interest, what implementation of libc was that?
A hacked up FreeBSD libc (we make a FreeBSD-derived operating system). Some of the authentication/credentials code, for example, creates a bunch of large stack frames.
Re: Lthread is a multicore/multithread coroutine library written in C
#15So, last time this was posted I commented that I was interested, but couldn't use this at work because of the GPL license. Since then I implemented my own M:N userspace:kernel threading library (which is what lthread boils down to) based on Russ Cox's BSD-licensed libtask. You know what I found? 1) Many libc routines require a surprisingly large amount of stack. 64kiB was the smallest power of 2 I could find where I…
Since I can't edit this post anymore, here's the relevant code:
https://github.com/cemeyer/taskmn
(Buyer beware: this is basically a snapshot of it at the point we decided to drop it entirely; I poked it enough to get it to compile under GCC / linux, but haven't verified functionality at all.)
More edits:
4) Re: Stack-copying; I found that without any attempt at optimizing memory use, my "light" threads were consuming on the order of ~600 bytes of stack (when de-scheduled); with a stack copying approach and a run-time stack of 128kiB, calling large-stack-using libc functions was fine.
5) Re: per-task dedicated stack memory; the default amount of memory allocated per-pthread on our platform is 128kiB, so 64kiB isn't great savings. I detected stack corruption at 32kiB and below by setting a canary value (at the top of the allocated stack) in the scheduler before running a ready "lthread."
Re: Lthread is a multicore/multithread coroutine library written in C
#16See also State Threads:
Re: Lthread is a multicore/multithread coroutine library written in C
#17I wonder how this library compares agains Apple's Grand Central Dispatch.
I guess they don't contain 1:1 functionality, but both libraries seem to be meant to make threading easier. I guess GCD is a more complete solution due to the addition of queues.
Re: Lthread is a multicore/multithread coroutine library written in C
#18So, last time this was posted I commented that I was interested, but couldn't use this at work because of the GPL license. Since then I implemented my own M:N userspace:kernel threading library (which is what lthread boils down to) based on Russ Cox's BSD-licensed libtask. You know what I found? 1) Many libc routines require a surprisingly large amount of stack. 64kiB was the smallest power of 2 I could find where I…
Since I can't edit this post anymore, here's the relevant code: https://github.com/cemeyer/taskmn (Buyer beware: this is basically a snapshot of it at the point we decided to drop it entirely; I poked it enough to get it to compile under GCC / linux, but haven't verified functionality at all.) More edits: 4) Re: Stack-copying; I found that without any attempt at optimizing memory use, my "light" threads were consumin…
Thanks for sharing your code and experience.
600 bytes per de-scheduled thread sounds excellent; It makes a million mostly-dormant threads feasible in less than a gig of memory -- I think that's the use case lthread-style threads shine in. You wouldn't be able to do that with blocking kernel threads (you'd need 64gig of ram just for stack, and that assumes the kernel scales well enough to handle that).
I think overall lower efficiency per thread, even if it's 50% slower, is acceptable in such a use case, given the memory requirements reduction.
Re: Lthread is a multicore/multithread coroutine library written in C
#19Earlier quoted context omitted.
Since I can't edit this post anymore, here's the relevant code: https://github.com/cemeyer/taskmn (Buyer beware: this is basically a snapshot of it at the point we decided to drop it entirely; I poked it enough to get it to compile under GCC / linux, but haven't verified functionality at all.) More edits: 4) Re: Stack-copying; I found that without any attempt at optimizing memory use, my "light" threads were consumin…
Thanks for sharing your code and experience. 600 bytes per de-scheduled thread sounds excellent; It makes a million mostly-dormant threads feasible in less than a gig of memory -- I think that's the use case lthread-style threads shine in. You wouldn't be able to do that with blocking kernel threads (you'd need 64gig of ram just for stack, and that assumes the kernel scales well enough to handle that). I think overal…
No problem :-).
Re: "lower efficiency… is acceptable in such a use case": Maybe — it really depends on your use case. In the case you describe, sure. In my particular (specialized) case:
1) My clients are connected by 1Gb or 10Gb switched ethernet, and they are typically on firewalled local networks. I don't worry about trickle-style DoS attacks. (In other words: my threads are mostly-active, not mostly-dormant.)
2) My clients really only care about sustained streaming throughput. So, if they can max out the underlying disk with 20-30 simultaneous connections, they won't bother throwing 1000s of connections at my server. pthread-per-connection works great for this circumstance (with a pre-allocated thread pool).
Use the right tool for the job ;-).
Re: Lthread is a multicore/multithread coroutine library written in C
#20See also State Threads: http://state-threads.sourceforge.net/
According to SF, it was last updated in 2009 (is it actively developed?). I don't see much in the way of examples on the SF page, which is a bit off-putting. I guess I'm curious about the model it uses and how it performs, but my best guess is that it's not radically different from lthread or any other M:N userspace threading library. Do you know more?