Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Java Virtual Threads: A Case Study

infoq.com

Post reply on HN
Print view View original

61–70 of 199 posts

Re: Java Virtual Threads: A Case Study

#61

Post by zokier »

hashmash wrote: post #20
packetlost wrote: post #12

Earlier quoted context omitted.

Yeah, and it's generally good to be RAM limited instead of CPU, no? The alternative is blowing a bunch of time on syscalls and OS scheduler overhead. Also the virtual threads run on a "traditional" thread pool to my understanding, so you can just tweak the number of worker threads to cap the total concurrency. The benefit is it's overall more efficient (in the general case) and lets you write linear blocking code (as…

The OS scheduler is still there (for the carrier threads), but now you've added on top of that FJ pool based scheduler overhead. Although virtual threads don't have the syscall overhead when they block, there's a new cost caused by allocating the internal continuation object, and copying state into it. This puts more pressure on the garbage collector. Context switching cost due to CPU cache thrashing doesn't go away…

> I've not yet seen a study that shows that virtual threads offer a huge benefit.

Not exactly Java virtual threads, but a study on how userland threads beat kernel threads.

https://cs.uwaterloo.ca/~mkarsten/papers/sigmetrics2020.html

For quick results, check figures 11 and 15 from the (preprint) paper. Userland threads ("fred") have ~50% higher throughput while having orders of magnitude better latency at high load levels, in a real-world application (memcached).

Re: Java Virtual Threads: A Case Study

#62

Post by bberrry »

LinXitoW wrote: post #57

From my very limited exposure to virtual threads and the older solution (thread pools), the biggest hurdle was the extensive use of ThreadLocals by most popular libraries. In one project I had to basically turn a reactive framework into a one thread per request framework, because passing around the MDC (a kv map of extra logging information) was a horrible pain. Getting it to actually jump ship from thread to thread…

If you are already in a reactive framework, why would you change to virtual threads? Those frameworks pool threads and have their own event loop so I would say they are not suitable for virtual thread migration.

Re: Java Virtual Threads: A Case Study

#63

Post by pron »

Virtual threads do one thing: they allow creating lots of threads. This helps throughput due to Little's law [1]. But because this server here saturates the CPU with only a few threads (it doesn't do the fanout modern servers tend to do), this means that no significant improvements can be provided by virtual threads (or asynchronous programming, which operates on the same principle) while keeping everything else in the system the same, especially since everything else in that server was optimised for over two decades under the constraints of expensive threads (such as the deployment strategy to many small instances with little CPU).

So it looks like their goal was: try adopting a new technology without changing any of the aspects designed for an old technology and optimised around it.

[1]: https://youtu.be/07V08SB1l8c

Re: Java Virtual Threads: A Case Study

#64

Post by dwattttt »

bheadmaster wrote: post #24
nlitened wrote: post #10

Earlier quoted context omitted.

> is far easier for most people I’d say that writing single-threaded code is far easier for _all_ people, even async code experts :) Also, single-threaded code is supported by programming language facilities: you have a proper call stack, thread-local vars, exceptions bubbling up, structured concurrency, simple resource management (RAII, try-with-resources, defer). Easy to reason and debug on language level. Async ru…

> Async runtimes are always complicated, filled with leaky abstractions, it’s like another language that one has to learn in addition, but with a less thought-out, ad-hoc design. Difficult to reason and debug, especially in edge cases Async runtimes themselves are simply attempts to bolt-on green threads on top of a language that doesn't support them on a language level. In JavaScript, async/await uses Promises to en…

> Not sure about Rust, but it probably also uses some Rust macro magic to do something similar.

Much the same as JavaScript I understand, but no macros; the compiler turns them into Futures that can be polled

Re: Java Virtual Threads: A Case Study

#65

Post by pjmlp »

peteri wrote: post #58
taspeotis wrote: post #26

My rough understanding is that this is similar to async/await in .NET? It’s a shame this article paints a neutral (or even negative) experience with virtual threads. We rewrote a boring CRUD app that spent 99% of its time waiting the database to respond to be async/await from top-to-bottom. CPU and memory usage went way down on the web server because so many requests could be handled by far fewer threads.

It's a different model. Microsoft did work on green threads a while ago and decided against continuing. Links: https://github.com/dotnet/runtimelab/issues/2398 https://github.com/dotnet/runtimelab/blob/feature/green-thre...

It should be pointed out, that the main reason they didn't go further was because of added complexity in .NET, when async/await already exists.

> Green threads introduce a completely new async programming model. The interaction between green threads and the existing async model is quite complex for .NET developers. For example, invoking async methods from green thread code requires a sync-over-async code pattern that is a very poor choice if the code is executed on a regular thread.

Also to note that even the current model is complex enough to warrant a FAQ,

https://devblogs.microsoft.com/dotnet/configureawait-faq

https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/b...

Re: Java Virtual Threads: A Case Study

#66

Post by anonymousDan »

frant-hartm wrote: post #55
giamma wrote: post #53

Earlier quoted context omitted.

You will still have to worry, too many virtual threads will imply too much context switching. However, virtual threads will be always interruptable on I/O, as they are not mapped to actual o.s. threads, but rather simulated by the JVM which will executed a number of instructions for each virtual thread. This gives the chance to the JVM to use real threads more efficiently, avoiding that threads remain unused while wa…

What do you mean by context switching? My understanding is that virtual threads mostly eliminate context switching - for N CPUs JVM creates N platform threads and they run virtual threads as needed. There is no real context switching apart from GC and other JVM internal threads. A platform thread picking another virtual thread to run after its current virtual thread is blocked on IO is not a context switch, that is a…

Does Java's implementation of virtual threads perform any kind of work stealing when a particular physical thread has no virtual threads to run (e.g. they are all blocked on I/O)?

Re: Java Virtual Threads: A Case Study

#67

Post by za3faran »

devjab wrote: post #37
taspeotis wrote: post #26

My rough understanding is that this is similar to async/await in .NET? It’s a shame this article paints a neutral (or even negative) experience with virtual threads. We rewrote a boring CRUD app that spent 99% of its time waiting the database to respond to be async/await from top-to-bottom. CPU and memory usage went way down on the web server because so many requests could be handled by far fewer threads.

> My rough understanding is that this is similar to async/await in .NET? Not really. What C# does is sort of similar but it has the disadvantages of splitting your code ecosystem into non-blocking/blocking code. This means you can “accidentally” start your non-blocking code. Something which may cause your relatively simple API to consume a ridiculous amount of resources. It also makes it much more complicated to upda…

I would not argue that golang is the obvious choice for concurrency. Java's approach is actually superior to golang's. It takes it a step further by offering structured concurrency[1].

Kotlin's design had no bearing on Java's or the JVM's implementation.

C# has an interruption model through CancellationToken as far as I'm aware.

[1] https://openjdk.org/jeps/453

Re: Java Virtual Threads: A Case Study

#68

Post by za3faran »

delusional wrote: post #40
devjab wrote: post #37

Earlier quoted context omitted.

> My rough understanding is that this is similar to async/await in .NET? Not really. What C# does is sort of similar but it has the disadvantages of splitting your code ecosystem into non-blocking/blocking code. This means you can “accidentally” start your non-blocking code. Something which may cause your relatively simple API to consume a ridiculous amount of resources. It also makes it much more complicated to upda…

From what I recall, and this is a while ago so bare with me, Java Virtual Threads still have a lot of pitfalls where the promise of concurrency isn't really fulfilled. I seem to remember that is was some pretty basic operations (like maybe read or something) that caused the thread not to unmount, and therefore just block the underlying os thread. At that point you've just invented the world's most complicated thread…

You're referring to thread pinning, and this is being addressed.

Re: Java Virtual Threads: A Case Study

#69

Post by pron »

exabrial wrote: post #2

What is the virtual thread / event loop pattern seeking to optimize? Is it context switching? A number of years ago I remember trying to have a sane discussion about “non blocking” and I remember saying “something” will block eventually no matter what… anything from the buffer being full on the NIC to your cpu being at anything less than 100%. Does it shake out to any real advantage?

No, it optimises hardware utilisation by simply allowing more tasks to concurrently make progress. This allows throughput to reach the maximum the hardware allows. See https://youtu.be/07V08SB1l8c.

Re: Java Virtual Threads: A Case Study

#70

Post by immibis »

gregopet wrote: post #28
immibis wrote: post #8

Earlier quoted context omitted.

So... What is it seeking to optimize? Why did you need a thread pool before but not any more? What resource was exhausted to prevent you from putting every request on a thread?

It's mainly trying to make you not worry about how many threads you create (and not worry about the caveats that come with optimising how many threads you create, which is something you are very often forced to do). You can create a thread in your code and not worry whether that thing will then be some day run in a huge loop or receive thousands of requests and therefore spend all your memory on thread overhead. Go a…

It seems that the answer to the question was "memory". Stack allocations, presumably. You have answered by telling us that virtual threads are better than real threads because real threads suck, but you didn't say why they suck or why virtual threads don't suck in the same way.
Post reply on HN