I'm wary of unbounded thread pools. Production has a funny way of showing that threads always consume resources. A fun example is file descriptors. An unexpected database reboot is often a short outage, but it's crazy how quickly unbounded thread pools can amplify errors and delay recovery. Anyway, they have their place, but if you've got a fancy chain of micro services calling out to wherever, think hard before putt…
Thread Pools on the JVM
41–50 of 120 posts
Re: Thread Pools on the JVM
#42Earlier quoted context omitted.
Are we coming full circle going back a variant of the original Java green threads?
This gives some color - https://blogs.oracle.com/javamagazine/going-inside-javas-pro...
Re: Thread Pools on the JVM
#43Earlier quoted context omitted.
Are we coming full circle going back a variant of the original Java green threads?
Yes and no. The new Loom threads will be much lighter weight than the original Java green threads. Further, the entire IO infrastructure of the JVM is being reworked for Loom to make sure the OS doesn't block the VM's thread. What's more, Loom does M:N threading. Same concept, very different implementation.
Re: Thread Pools on the JVM
#44Loom can't land fast enough! The current issue the JVM has is that all threads have a corresponding operating system thread. That, unfortunately, is really heavy memory wise and on the OS context switcher. Loom allows java to have threads as light weight as a goroutine. It's going to change the way everything works. You might still have a dedicated CPU bound thread pool (the common fork join pool exists and probably…
Are you quite certain that a (linux, nptl) thread costs more memory than a goroutine? You've implied that but it's not obviously true.
I don't think goroutines would need such information. A goroutine knows that "int foobar;" is currently being stored in "rbx", and that "int foobar" is currently saved on the stack. Therefore, rbx doesn't need to be saved.
------
Linux/NPTL threads don't know when they are interrupted. So all register state (including AVX512 state if those are being used) needs to be saved. AVX512 x 32 is 2kB alone.
Even if AVX512 isn't being used by a thread (Linux detects all AVX512 registers to be all-zero), RAX through R15 is 128-bytes, plus SSE-registers (another 128-bytes) or ~256 bytes of space that the goroutines don't need. Plus whatever other process-specific information needs to be saved off (CPU time and other such process / thread details that Linux needs to decide which threads to process next)
Re: Thread Pools on the JVM
#45Earlier quoted context omitted.
What benefits does loom provide vs using something like cats-effect fibres?
You can actually debug the code you write because you get a real stacktrace, not few frames that shows the underlying implementation.
Re: Thread Pools on the JVM
#46Loom can't land fast enough! The current issue the JVM has is that all threads have a corresponding operating system thread. That, unfortunately, is really heavy memory wise and on the OS context switcher. Loom allows java to have threads as light weight as a goroutine. It's going to change the way everything works. You might still have a dedicated CPU bound thread pool (the common fork join pool exists and probably…
You are ignoring the downside to green threads which is that it’s cooperative. If the thread doesn’t yield control back to the event loop then the real OS thread backing the loop is now stuck. Which leads to dirty things like inserting sleep 0 at the top of loops and dealing with really unbalanced scheduling of threads don’t hit yields often enough. Plus with loom it might not be obvious that some function is a yield…
Re: Thread Pools on the JVM
#47Earlier quoted context omitted.
I was under the impression that Loom was implementing preemptable lightweight threads. Is that not the case?
So loom uses interesting terminology when talking about this. They say that they’re preemptive and not cooperative because there’s not an explicit await/yield keyword that you call from your code but that isn’t the whole story because threads are only preempted when they perform IO or are synchronized. So you as an author can’t know for sure where the yield points are and aren’t supposed to rely on them but they’re s…
That is a common terminology. Wikipedia says: [1]
The term preemptive multitasking is used to distinguish a multitasking operating system, which permits preemption of tasks, from a cooperative multitasking system wherein processes or tasks must be explicitly programmed to yield when they do not need system resources. ... The term "preemptive multitasking" is sometimes mistakenly used when the intended meaning is more specific, referring instead to the class of scheduling policies known as time-shared scheduling, or time-sharing.
> threads are only preempted when they perform IO or are synchronized
First, they can be preempted by any call, explicit or implicit, to the runtime (or any library, for that matter). For all you know, class loading or even Math.sin might include a scheduling point (although that is unlikely as that's a compiler intrinsic). We make no promises on when scheduling can occur. Not only do threads not explicitly yield, code cannot statically determine where scheduling might occur; I don't believe anyone can consider this "cooperative."
Second, Loom's virtual threads can also be forcibly preempted by the scheduler at any safepoint to implement time sharing. Currently, this capability isn't exposed because we're yet to find a use-case for it (other than one special case that we want to address, but isn't urgent). If you believe you have one, please send it to the loom-dev mailing list.
The reason it's hard to find good use cases for time slicing is as follows:
1. If you have only a small number of threads that are frequently CPU bound. In that case, just make them platform threads and use the OS scheduler. Loom makes it easy to choose which implementation you want for each thread.
2. If you have a great many threads, each of which can infrequently become CPU-bound, then the scheduler takes care of that with work-stealing and other scheduling techniques.
3. If you have a great many threads, each of which is frequently CPU-bound, then your cores are oversubscribed by orders of magnitude -- recall that we're talking about hundreds of thousands or possibly millions of threads -- and no scheduling strategy can help you.
It's possible that there could arise real-world situations where infrequent CPU-boundedness might affect responsiveness, but we'll want to see such cases before deciding to expose the mechanism. Even OSes don't like relying on time-sharing (it happens less frequently than people think on well-tuned servers), and putting that capability in the hands of programmers is an attractive nuisance that will more likely cause a degradation in performance.
[1]: https://en.wikipedia.org/wiki/Preemption_(computing)#Preempt...
Re: Thread Pools on the JVM
#48Earlier quoted context omitted.
You can actually debug the code you write because you get a real stacktrace, not few frames that shows the underlying implementation.
On the other hand, you'll spend a lot more time debugging Loom code, because it reuses the same broken-by-design thread API.
Re: Thread Pools on the JVM
#49Earlier quoted context omitted.
Are you quite certain that a (linux, nptl) thread costs more memory than a goroutine? You've implied that but it's not obviously true.
Wouldn't any linux/nptl thread require at at least the register-state of the entire x86 (or ARM) CPU? I don't think goroutines would need such information. A goroutine knows that "int foobar;" is currently being stored in "rbx", and that "int foobar" is currently saved on the stack. Therefore, rbx doesn't need to be saved. ------ Linux/NPTL threads don't know when they are interrupted. So all register state (includin…
Re: Thread Pools on the JVM
#50Earlier quoted context omitted.
Whats the difference between goroutines and project loom? Is their any?
Terminology mostly :D I've not looked into the goroutine implementation, so I couldn't tell you how it compares to what I've read loom is doing. Loom is looking to have some extremely compact stacks which means each new "virtual thread" as they are calling them will end up having bytes worth of memory allocated. Another thing coming with loom that go lacks is "structured concurrency". It's the notion that you might h…