Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thread Pools on the JVM

gist.github.com

Post reply on HN
Print view View original

101–110 of 120 posts

Re: Thread Pools on the JVM

#101

Post by dikei »

cogman10 wrote: post #2

Loom can't land fast enough! The current issue the JVM has is that all threads have a corresponding operating system thread. That, unfortunately, is really heavy memory wise and on the OS context switcher. Loom allows java to have threads as light weight as a goroutine. It's going to change the way everything works. You might still have a dedicated CPU bound thread pool (the common fork join pool exists and probably…

Yup, Loom will simplify a lot the Producer-Consumer pattern on I/O operation. With virtual threads, it's basically free to block on consumer threads, so you would need only 1 bounded pool for the consumers.

Currently for efficiency, you would need at least 2 pools: 1 small bounded pool for dequeuing the requests and create the IO operation, and 1 unbounded pool for actually executing the IO operation.

Re: Thread Pools on the JVM

#102

Post by an_opabinia »

cbsmith wrote: post #99
vbezhenar wrote: post #79

Earlier quoted context omitted.

Your words might be true, but the world jumped on async wagon long time ago and going all in. Nobody likes threads, everyone wants lightweight threads. Emulating lightweight threads with promises (optionally hidden behind async/await transformations) is very popular. So demand for this feature is here. I don't know why, I, personally, never needed that feature and good old threads were always enough for me. It's weir…

It's madness.

Is it? Benchmarks routinely show the async native database drivers outperforming JDBC ones in Java, and evented (async) IO is king in the only other contenders, C++ and Rust runtimes for RESTful and other server apps.

Re: Thread Pools on the JVM

#103

Post by [deleted] »

geodel wrote: post #93
lmilcin wrote: post #91

Earlier quoted context omitted.

I have discovered ReactiveX for Java and Reactor in particular. I am working with Kafka and MongoDB and it is normal for my app to have a million in flight transactions at various stages of completion. In the past it required a lot of planning (and a lot of code) but Reactor let's me build these processes as pipelines with whatever concurrency or scheduler I desire, at any stage of the processing. We are even doing t…

Loom will help folks who prefer writing straightforward Java code instead of some random reactive library with obscure exception handling and poor to impossible debuggability. Now I get it is hard for many folks to understand that part. Just like at my workplace people think it is impossible to write micro service without SpringBoot. > Loom could help if you had blocking APIs to start, but you get much better results…

[deleted]

Re: Thread Pools on the JVM

#104

Post by cbsmith »

an_opabinia wrote: post #102
cbsmith wrote: post #99

Earlier quoted context omitted.

It's madness.

Is it? Benchmarks routinely show the async native database drivers outperforming JDBC ones in Java, and evented (async) IO is king in the only other contenders, C++ and Rust runtimes for RESTful and other server apps.

...and in how many circumstances are the database drivers the limiting factor in application performance?

As I said in the beginning, you will absolutely win in the extreme cases (and accordingly, those tend to be the drivers that are tuned more for performance). In most cases in won't really make much of a difference one way or the other, and in some cases it will actually inhibit performance.

Re: Thread Pools on the JVM

#105

Post by jjtheblunt »

dragonwriter wrote: post #62
jjtheblunt wrote: post #57

Earlier quoted context omitted.

I worked in Solaris internals for a while at Sun during the early java era, and Solaris threading definitely did multiplexing of userspace onto os, and then os onto cores. Do you have a citation (because I can't find one) specifying your assertion that original Java green threads were not analogous to Solaris user -> os -> hardware multiplexing?

> Do you have a citation (because I can't find one) specifying your assertion that original Java green threads were not analogous to Solaris user -> os -> hardware multiplexing? I was writing from memory of second-hand after-the-fact recitations of the history. Doing some followup research prompted by your question, if I understand this document [0] correctly, Java initially had N:1 green threads on Solaris, then M:N…

thank you!

Re: Thread Pools on the JVM

#106

Post by pron »

lmm wrote: post #96
pron wrote: post #47

Earlier quoted context omitted.

> So loom uses interesting terminology when talking about this. That is a common terminology. Wikipedia says: [1] The term preemptive multitasking is used to distinguish a multitasking operating system, which permits preemption of tasks, from a cooperative multitasking system wherein processes or tasks must be explicitly programmed to yield when they do not need system resources. ... The term "preemptive multitasking…

Isn't the point of preemption to degrade gracefully when the cores are oversubscribed? E.g. the first system I worked on ran potentially CPU-heavy work from various clients, and used per-client threads to isolate them; every so often clients would find ways to get their thread stuck doing a large amount of CPU work (e.g. regex backtracking) and although these were in some sense bugs (and we did fix them), it was very…

There's no doubt forced preemption could help, but I'm still unsure about what the right algorithm is; probably not time sharing.

Suppose you have 100K threads, and only 1% of them become CPU-bound for 100ms. That could take down your 32-core server for 3 seconds, which is bad. But suppose we had 10ms time-slices. Then, those busy threads' latency might go from 100ms to as high as a few minutes, which means effectively taking them down. The scale has a qualitative effect here. So, rather than time-sharing, it might be better to optionally install some other preemption policy -- maybe something that indefinitely suspends threads that behave badly too often and puts them in some collection.

The point is that time-slicing will probably not be helpful in sufficiently many cases, and we don't yet know what will. We'd like to gather more data before offering something. In some other languages/runtimes it might be worthwhile to just expose a capability and see what people do with it, but with Java, within five minutes you'll have twenty libraries doing time-sharing, and thousands of people using them blindly whether it's good or bad for them (just because they say they do time-sharing, and that's good, no?), and now there's just noise and bad habits everywhere. This is nanny-state governance, but we've learned our lesson, and you can't be too careful with an ecosystem this big.

Re: Thread Pools on the JVM

#107

Post by kaba0 »

Spivak wrote: post #22
cogman10 wrote: post #2

Loom can't land fast enough! The current issue the JVM has is that all threads have a corresponding operating system thread. That, unfortunately, is really heavy memory wise and on the OS context switcher. Loom allows java to have threads as light weight as a goroutine. It's going to change the way everything works. You might still have a dedicated CPU bound thread pool (the common fork join pool exists and probably…

You are ignoring the downside to green threads which is that it’s cooperative. If the thread doesn’t yield control back to the event loop then the real OS thread backing the loop is now stuck. Which leads to dirty things like inserting sleep 0 at the top of loops and dealing with really unbalanced scheduling of threads don’t hit yields often enough. Plus with loom it might not be obvious that some function is a yield…

When you have a runtime, you have proper information whether there is work being done on a given virtual thread - So in case of Loom, afaik any blocking call will turn into non-blocking auto-magically (other than FFI, but that is very rare in Java), since the JVM is free to wait on that asynchronously behind the scenes and do some other work in the meantime.

Re: Thread Pools on the JVM

#108

Post by kaba0 »

sudhirj wrote: post #26
Spivak wrote: post #22

Earlier quoted context omitted.

You are ignoring the downside to green threads which is that it’s cooperative. If the thread doesn’t yield control back to the event loop then the real OS thread backing the loop is now stuck. Which leads to dirty things like inserting sleep 0 at the top of loops and dealing with really unbalanced scheduling of threads don’t hit yields often enough. Plus with loom it might not be obvious that some function is a yield…

Sleep 0 sounds like quite a hack, Go has the neater https://pkg.go.dev/runtime#Gosched instead, and I assume there will be a Java equivalent as well. And if most stdlib methods and all blocking methods call it, it's going to be pretty difficult to hang a green thread.

Since there is a runtime that knows everything about the state of the thread, my understanding is that there is no need for explicit yields. Everything will turn automagically into non-blocking (except for FFI)

Re: Thread Pools on the JVM

#109

Post by kaba0 »

clhodapp wrote: post #31
brokencode wrote: post #28

Earlier quoted context omitted.

I was under the impression that Loom was implementing preemptable lightweight threads. Is that not the case?

I think that's not quite it: I believe that loom is implementing cooperative lightweight threads and simultaneously reworking all of the blocking IO operations in the Java standard library to include yields. I guess this means that you could, for example, hold an OS-level thread forever by writing an infinite loop that doesn't do any IO...

I believe both the OS and the JVM is free to reschedule it. Yields are just an explicit way of possibly changing the thread.

Re: Thread Pools on the JVM

#110

Post by samus »

kllrnohj wrote: post #69
cbsmith wrote: post #52

Earlier quoted context omitted.

> That, unfortunately, is really heavy memory wise and on the OS context switcher. So, there was a time where a broad statement like that was pretty solid. These days, I don't think so. The default stack size (on 64-bit Linux) is 1MB, and you can manipulate that to be smaller if you want. That's also the virtual memory. The actually memory usage depends on your application. There was a time where 1MB was a lot of mem…

> So, there was a time where a broad statement like that was pretty solid. That time is approaching 20 years old at this point, too. Native threads haven't been "expensive" for a very, very long time now. Maybe if you're in the camp of disabling overcommit it matters, but otherwise the application of green threads is definitely a specialized niche, not generally useful. > In the broad set of use cases though, switchi…

> I'd go even further and say it'll be a net-loss in most cases, especially with modern complications like heterogeneous compute. If you're use case is specifically spinning up thousands of threads for IO (aka, you're a server & nothing else), then sure. But if you aren't there's no win here, just complications (like times when you need native thread isolation for FFI reasons, like using OpenGL)

Virtual threads are going to be an /option/, not a requirement. Threads have to explicitly created as virtual threads. If this is not done, nothing will change.

Post reply on HN