Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thread Pools on the JVM

gist.github.com

Post reply on HN
Print view View original

31–40 of 120 posts

Re: Thread Pools on the JVM

#31

Post by clhodapp »

brokencode wrote: post #28
Spivak wrote: post #22

Earlier quoted context omitted.

You are ignoring the downside to green threads which is that it’s cooperative. If the thread doesn’t yield control back to the event loop then the real OS thread backing the loop is now stuck. Which leads to dirty things like inserting sleep 0 at the top of loops and dealing with really unbalanced scheduling of threads don’t hit yields often enough. Plus with loom it might not be obvious that some function is a yield…

I was under the impression that Loom was implementing preemptable lightweight threads. Is that not the case?

I think that's not quite it:

I believe that loom is implementing cooperative lightweight threads and simultaneously reworking all of the blocking IO operations in the Java standard library to include yields. I guess this means that you could, for example, hold an OS-level thread forever by writing an infinite loop that doesn't do any IO...

Re: Thread Pools on the JVM

#32

Post by mikepurvis »

brokencode wrote: post #28
Spivak wrote: post #22

Earlier quoted context omitted.

You are ignoring the downside to green threads which is that it’s cooperative. If the thread doesn’t yield control back to the event loop then the real OS thread backing the loop is now stuck. Which leads to dirty things like inserting sleep 0 at the top of loops and dealing with really unbalanced scheduling of threads don’t hit yields often enough. Plus with loom it might not be obvious that some function is a yield…

I was under the impression that Loom was implementing preemptable lightweight threads. Is that not the case?

It sounds like it is: https://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1....

But the other side of that is that sometimes non-preemption is also a desirable property— like in JavaScript, or Python asyncio, knowing that you don't need to lock over every little manipulation of some shared data structure because you're never going to yield if you didn't explicitly await.

Re: Thread Pools on the JVM

#33

Post by cogman10 »

Spivak wrote: post #22
cogman10 wrote: post #2

Loom can't land fast enough! The current issue the JVM has is that all threads have a corresponding operating system thread. That, unfortunately, is really heavy memory wise and on the OS context switcher. Loom allows java to have threads as light weight as a goroutine. It's going to change the way everything works. You might still have a dedicated CPU bound thread pool (the common fork join pool exists and probably…

You are ignoring the downside to green threads which is that it’s cooperative. If the thread doesn’t yield control back to the event loop then the real OS thread backing the loop is now stuck. Which leads to dirty things like inserting sleep 0 at the top of loops and dealing with really unbalanced scheduling of threads don’t hit yields often enough. Plus with loom it might not be obvious that some function is a yield…

Which is why the advice would be "Don't use virtual threads for CPU work".

It just so happens that a large number of JVM users are working with IO bound problems. Once you start talking about CPU bound problems the JVM tends not to be the thing most people reach for.

Loom doesn't remove the CPU bound solution by adding the IO solution. Instead, it adds a good IO solution and keeps the old CPU solution when needed.

In fact, there's already a really good pool in the JVM for common CPU bound tasks. `Forkjoin.common()`.

Re: Thread Pools on the JVM

#34

Post by Spivak »

brokencode wrote: post #28
Spivak wrote: post #22

Earlier quoted context omitted.

You are ignoring the downside to green threads which is that it’s cooperative. If the thread doesn’t yield control back to the event loop then the real OS thread backing the loop is now stuck. Which leads to dirty things like inserting sleep 0 at the top of loops and dealing with really unbalanced scheduling of threads don’t hit yields often enough. Plus with loom it might not be obvious that some function is a yield…

I was under the impression that Loom was implementing preemptable lightweight threads. Is that not the case?

So loom uses interesting terminology when talking about this. They say that they’re preemptive and not cooperative because there’s not an explicit await/yield keyword that you call from your code but that isn’t the whole story because threads are only preempted when they perform IO or are synchronized. So you as an author can’t know for sure where the yield points are and aren’t supposed to rely on them but they’re still there. You’re not going to be forcefully preempted in the middle of number crunching.

I think most people would consider this a surprising notion of preemption where it’s out of your control-ish but also not arbitrary like it is for OS threads which still leads to basically the same problems and constraints as cooperative threads.

Re: Thread Pools on the JVM

#35

Post by cogman10 »

Spivak wrote: post #34
brokencode wrote: post #28

Earlier quoted context omitted.

I was under the impression that Loom was implementing preemptable lightweight threads. Is that not the case?

So loom uses interesting terminology when talking about this. They say that they’re preemptive and not cooperative because there’s not an explicit await/yield keyword that you call from your code but that isn’t the whole story because threads are only preempted when they perform IO or are synchronized. So you as an author can’t know for sure where the yield points are and aren’t supposed to rely on them but they’re s…

Yeah... this is a place where I disagree with how the Loom devs define "preemptive". They are basically defining it as "most tasks will give up control when they hit a blocking operation". Yet, it's been my understanding that preemption means the scheduler can stop a currently operating task from running and switch to something else. That's not what happens with loom.

Re: Thread Pools on the JVM

#36

Post by jsmith45 »

cogman10 wrote: post #11
dragontamer wrote: post #6

Earlier quoted context omitted.

Threadpools are probably better on CPU-bound bound (or CPU-ish bound tasks: like RAM-bound) without any I/O. Coroutines / Goroutines and the like are probably better on I/O bound tasks where the CPU-effort in task-switching is significant. -------- For example: Matrix Multiplication is better with a Threadpool. Handling 1000 simultaneous connections when you get Slashdotted (or "Hacker News hug of death") is better s…

I agree. Coroutines MIGHT be more efficient if what you end up building is a statemachine anyways (as that's what most of those coroutines are doing with the compiler). Otherwise, if it's just pure parallel CPU/memory burning with little state transitions/dependence then a dedicated CPU pool fixed to roughly the number of CPU cores on the box will be the most efficient. Heck, it can often even yield benefits to "pin"…

Only stackless co-routines require state machine transformation. Stackfull co-routines based user mode threading generally just change out the IO primitives to issue an asynchronous version of the operation, and immediately calls into the the user mode scheduler to pick some ready-to-resume co-routine to switch the stack to and resume. They might include a preemption facility (beyond just the OS's preemption of the underlying kernel threads), but that is not required and is largely a language/runtime design decision.

The big headaches with stackfull co-routine based user mode threading come from two sources. One is allocating the stack. If your language requires a contiguous stack then you either need to make the stacks small, and risk running out, or make them big which can be a problem on 32-bit platforms (you can run out of address space), or can be a problem on some platforms (those with strict commit-charge based memory accounting). Both can be mitigated by allowing non-contiguous stacks or re-locatable contiguous stacks (to allow small stacks to grown later without headaches), although obviously that can have performance considerations.

The other stackfull co-routine headache is in calling into code from another language (i.e. FFI) which could be making direct blocking system calls, and end up starving you of your OS threads.

I do agree that in purely CPU or memory bound applications a classical thread pool makes better sense. The main advantages of either type of co-routine based user mode threading primarily apply to IO-heavy or mixed workloads.

Re: Thread Pools on the JVM

#37

Post by rzzzt »

jackcviers3 wrote: post #12

Author mentions scala. Both ZIO[1] and Cats-Effect[2] provide fibers (coroutines) over these specific threadpool designs today, without the need for Project Loom, and give the user the capability of selecting the pool type to use without explicit reference. They are unusable from Java, sadly, as the schedulers and ExecutionContexts and runtime are implicitly provided in sealed companion objects and are therefore priv…

Quasar has similar functionality: https://docs.paralleluniverse.co/quasar/

Re: Thread Pools on the JVM

#38

Post by sk5t »

jfoutz wrote: post #9

I'm wary of unbounded thread pools. Production has a funny way of showing that threads always consume resources. A fun example is file descriptors. An unexpected database reboot is often a short outage, but it's crazy how quickly unbounded thread pools can amplify errors and delay recovery. Anyway, they have their place, but if you've got a fancy chain of micro services calling out to wherever, think hard before putt…

And you should be wary! Prefer instead a bounded thread pool with a bounded queue of tasks waiting for service, and also decide explicitly what should happen when the queue fills up or wait times become too high (whatever "too high" means for the application).

Re: Thread Pools on the JVM

#39

Post by cogman10 »

rzzzt wrote: post #37
jackcviers3 wrote: post #12

Author mentions scala. Both ZIO[1] and Cats-Effect[2] provide fibers (coroutines) over these specific threadpool designs today, without the need for Project Loom, and give the user the capability of selecting the pool type to use without explicit reference. They are unusable from Java, sadly, as the schedulers and ExecutionContexts and runtime are implicitly provided in sealed companion objects and are therefore priv…

Quasar has similar functionality: https://docs.paralleluniverse.co/quasar/

Fun fact, one of the primary loom devs wrote quasar.

Re: Thread Pools on the JVM

#40

Post by jeffbee »

cogman10 wrote: post #2

Loom can't land fast enough! The current issue the JVM has is that all threads have a corresponding operating system thread. That, unfortunately, is really heavy memory wise and on the OS context switcher. Loom allows java to have threads as light weight as a goroutine. It's going to change the way everything works. You might still have a dedicated CPU bound thread pool (the common fork join pool exists and probably…

Are you quite certain that a (linux, nptl) thread costs more memory than a goroutine? You've implied that but it's not obviously true.
Post reply on HN