Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Java Virtual Threads: A Case Study

infoq.com

Post reply on HN
Print view View original

51–60 of 199 posts

Re: Java Virtual Threads: A Case Study

#51

Post by he0001 »

taspeotis wrote: post #26

My rough understanding is that this is similar to async/await in .NET? It’s a shame this article paints a neutral (or even negative) experience with virtual threads. We rewrote a boring CRUD app that spent 99% of its time waiting the database to respond to be async/await from top-to-bottom. CPU and memory usage went way down on the web server because so many requests could be handled by far fewer threads.

> My rough understanding is that this is similar to async/await in .NET?

The biggest difference is that C# async/await code is rewritten by the compiler to be able to be async. This means that you see artifacts in the stack that weren’t there when you wrote the code.

There are no rewrites with virtual threads and the code is presented on the stack just as you write it.

They solve the same problem but in very different ways.

Re: Java Virtual Threads: A Case Study

#52

Post by Nullabillity »

fzeindl wrote: post #4
exabrial wrote: post #2

What is the virtual thread / event loop pattern seeking to optimize? Is it context switching? A number of years ago I remember trying to have a sane discussion about “non blocking” and I remember saying “something” will block eventually no matter what… anything from the buffer being full on the NIC to your cpu being at anything less than 100%. Does it shake out to any real advantage?

Does it shake out to any real advantage? To put it shortly: Writing single-threaded blocking code is far easier for most people and has many other benefits, like more understandable and readable programs: https://www.youtube.com/watch?v=449j7oKQVkc The main reason why non-blocking IO with it's style of intertwining concurrency and algorithms came along is that starting a thread for every request was too expensive. Wi…

> To put it shortly: Writing single-threaded blocking code is far easier for most people. [snip] With virtual threads that problem is eliminated so we can go back to writing blocking code.

This is the core misunderstanding/dishonesty behind the Loom/Virtual Threads hype. Single-threaded blocking code is easy, yes. But that ease comes from being single-threaded, not from not having to await a few Futures.

But Loom doesn't magically solve the threading problem. It hides the Futures, but that just means that you're now writing a multi-threaded program, without the guardrails that modern Future-aware APIs provide. It's the worst of all worlds. It's the scenario that gave multi-threading such a bad reputation for inscrutable failures in the first place.

Re: Java Virtual Threads: A Case Study

#53

Post by giamma »

gregopet wrote: post #28
immibis wrote: post #8

Earlier quoted context omitted.

So... What is it seeking to optimize? Why did you need a thread pool before but not any more? What resource was exhausted to prevent you from putting every request on a thread?

It's mainly trying to make you not worry about how many threads you create (and not worry about the caveats that come with optimising how many threads you create, which is something you are very often forced to do). You can create a thread in your code and not worry whether that thing will then be some day run in a huge loop or receive thousands of requests and therefore spend all your memory on thread overhead. Go a…

You will still have to worry, too many virtual threads will imply too much context switching. However, virtual threads will be always interruptable on I/O, as they are not mapped to actual o.s. threads, but rather simulated by the JVM which will executed a number of instructions for each virtual thread.

This gives the chance to the JVM to use real threads more efficiently, avoiding that threads remain unused while waiting on I/O (e.g. a response from a stream). As soon as the JVM detects that a physical thread is blocked on I/O, a semaphore, a lock or anything, it will reallocate that physical thread to running a new virtual thread. This will reduce latency, context switch time (the switching is done by the JVM that already globally manages the memory of the Java process in its heap) and will avoid or at least largely reduce the chance that a real thread remains allocated but idle as it's blocked on I/O or something else.

Re: Java Virtual Threads: A Case Study

#54

Post by pansa2 »

he0001 wrote: post #51
taspeotis wrote: post #26

My rough understanding is that this is similar to async/await in .NET? It’s a shame this article paints a neutral (or even negative) experience with virtual threads. We rewrote a boring CRUD app that spent 99% of its time waiting the database to respond to be async/await from top-to-bottom. CPU and memory usage went way down on the web server because so many requests could be handled by far fewer threads.

> My rough understanding is that this is similar to async/await in .NET? The biggest difference is that C# async/await code is rewritten by the compiler to be able to be async. This means that you see artifacts in the stack that weren’t there when you wrote the code. There are no rewrites with virtual threads and the code is presented on the stack just as you write it. They solve the same problem but in very differen…

> They solve the same problem but in very different ways.

Yes. Async/await is stackless, which leads to the “coloured functions” problem (because it can only suspend function calls one-by-one). Threads are stackful (the whole stack can be suspended at once), which avoids the issue.

Re: Java Virtual Threads: A Case Study

#55

Post by frant-hartm »

giamma wrote: post #53
gregopet wrote: post #28

Earlier quoted context omitted.

It's mainly trying to make you not worry about how many threads you create (and not worry about the caveats that come with optimising how many threads you create, which is something you are very often forced to do). You can create a thread in your code and not worry whether that thing will then be some day run in a huge loop or receive thousands of requests and therefore spend all your memory on thread overhead. Go a…

You will still have to worry, too many virtual threads will imply too much context switching. However, virtual threads will be always interruptable on I/O, as they are not mapped to actual o.s. threads, but rather simulated by the JVM which will executed a number of instructions for each virtual thread. This gives the chance to the JVM to use real threads more efficiently, avoiding that threads remain unused while wa…

What do you mean by context switching?

My understanding is that virtual threads mostly eliminate context switching - for N CPUs JVM creates N platform threads and they run virtual threads as needed. There is no real context switching apart from GC and other JVM internal threads.

A platform thread picking another virtual thread to run after its current virtual thread is blocked on IO is not a context switch, that is an expensive OS-level operation.

Re: Java Virtual Threads: A Case Study

#56

Post by pragmatick »

gregopet wrote: post #7
exabrial wrote: post #2

What is the virtual thread / event loop pattern seeking to optimize? Is it context switching? A number of years ago I remember trying to have a sane discussion about “non blocking” and I remember saying “something” will block eventually no matter what… anything from the buffer being full on the NIC to your cpu being at anything less than 100%. Does it shake out to any real advantage?

It's a brave attempt to release the programmer from worrying or even thinking about thread pools and blocking code. Java has gone all in - they even cancelled a non-blocking rewrite of their database driver architecture because why have that if you won't have to worry about blocking code? And the JVM really is a marvel of engineering, it's really really good at what it does, so what team to better pull this off? So f…

> although most of not all modern libraries have already adapted

Unfortunately kafka, for example, has not: https://github.com/spring-projects/spring-kafka/commit/ae775...

Re: Java Virtual Threads: A Case Study

#57

Post by LinXitoW »

From my very limited exposure to virtual threads and the older solution (thread pools), the biggest hurdle was the extensive use of ThreadLocals by most popular libraries.

In one project I had to basically turn a reactive framework into a one thread per request framework, because passing around the MDC (a kv map of extra logging information) was a horrible pain. Getting it to actually jump ship from thread to thread AND deleting it at the correct time was basically impossible.

Has that improved yet?

Re: Java Virtual Threads: A Case Study

#58

Post by peteri »

taspeotis wrote: post #26

My rough understanding is that this is similar to async/await in .NET? It’s a shame this article paints a neutral (or even negative) experience with virtual threads. We rewrote a boring CRUD app that spent 99% of its time waiting the database to respond to be async/await from top-to-bottom. CPU and memory usage went way down on the web server because so many requests could be handled by far fewer threads.

It's a different model. Microsoft did work on green threads a while ago and decided against continuing.

Links:

https://github.com/dotnet/runtimelab/issues/2398

https://github.com/dotnet/runtimelab/blob/feature/green-thre...

Re: Java Virtual Threads: A Case Study

#59

Post by davidtos »

I did some similar testing a few days ago[1]. Comparing platform threads to virtual threads doing API calls. They mention the right conditions like having high task delays, but it also depends on what the task is. Threads.sleep(1) performs better on virtual threads than platform threads but a rest call taking a few ms performs worse.

[1] https://davidvlijmincx.com/posts/virtual-thread-performance-...

Re: Java Virtual Threads: A Case Study

#60

Post by szundi »

devjab wrote: post #37
taspeotis wrote: post #26

My rough understanding is that this is similar to async/await in .NET? It’s a shame this article paints a neutral (or even negative) experience with virtual threads. We rewrote a boring CRUD app that spent 99% of its time waiting the database to respond to be async/await from top-to-bottom. CPU and memory usage went way down on the web server because so many requests could be handled by far fewer threads.

> My rough understanding is that this is similar to async/await in .NET? Not really. What C# does is sort of similar but it has the disadvantages of splitting your code ecosystem into non-blocking/blocking code. This means you can “accidentally” start your non-blocking code. Something which may cause your relatively simple API to consume a ridiculous amount of resources. It also makes it much more complicated to upda…

Maybe C# is going to have a new asynv await model but the fragmentation of libs and codes cannot be undone probably.

Java has the power that they make relatively more decisions about the language and the libs that they don’t have to fix later. That’s a great value if you’re not building throw-away software but SaaS or something that has to live long.

Post reply on HN