Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Java Virtual Threads: A Case Study

infoq.com

Post reply on HN
Print view View original

41–50 of 199 posts

Re: Java Virtual Threads: A Case Study

#41

Post by derriz »

bheadmaster wrote: post #24
nlitened wrote: post #10

Earlier quoted context omitted.

> is far easier for most people I’d say that writing single-threaded code is far easier for _all_ people, even async code experts :) Also, single-threaded code is supported by programming language facilities: you have a proper call stack, thread-local vars, exceptions bubbling up, structured concurrency, simple resource management (RAII, try-with-resources, defer). Easy to reason and debug on language level. Async ru…

> Async runtimes are always complicated, filled with leaky abstractions, it’s like another language that one has to learn in addition, but with a less thought-out, ad-hoc design. Difficult to reason and debug, especially in edge cases Async runtimes themselves are simply attempts to bolt-on green threads on top of a language that doesn't support them on a language level. In JavaScript, async/await uses Promises to en…

Indeed. Async runtimes/sytles are attempts to provide a more readable/useable syntax for CPS[1]. CPS originally had nothing to do with blocking/non-blocking or multi-threading but arose as a technique to structure compiler code.

Its attraction for non-blocking coding is that it allows hiding the multi-threaded event dispatching loop. But as the parent comment suggests, this abstraction is extremely leaky. And in addition, CPS in non-functional languages or without syntactic sugar has poor readability. Improving the readability requires compiler changes in the host language - so many languages have added compiler support to further hide the CPS underpinnings of their async model.

I've always felt this was a big mistake in our industry - all this effort not only in compilers but also in debuggers/IDE - building on a leaky abstraction. Adding more layers of leaky abstractions has only made the issue worse. Async code, at first glance, looks simple but is a minefield for inexperienced/non-professional software engineers.

It's annoying that Rust switched to async style - the abstraction leakiness immediately hits you, as the "hidden event dispatching loop" remains a real dependency even if it's not explicit in the code. Thus libraries using asycn cannot generally be used together although last time i looked, tokio seems to have become the de-facto standard.

[1] https://en.wikipedia.org/wiki/Continuation-passing_style

Re: Java Virtual Threads: A Case Study

#42

Post by jillesvangurp »

tzahifadida wrote: post #38
jillesvangurp wrote: post #36

Earlier quoted context omitted.

Kotlin embraced the same thing via co-routines, which are conceptually similar to go routines. It adds a few useful concepts around this though; mainly that of a co-routine context which encapsulates that a tree of co-routine calls needs some notion of failure handling and cancellation. Additionally, co-routines are dispatched to a dispatcher. A dispatcher can be just on the same thread or actually use a thread pool.…

Kotlin is not a language I learned so I will avoid commenting. However, the use of JAVA for me is for admin backend or heavy weight services for enterprises or startups I coded for, so for my taste I can't use it without spring or jboss, etc.. , and in that way I think simplicity went out the window a long long time ago :) It took me years to learn all the quirks of these frameworks... and the worse thing about it is…

Kotlin makes a lot of that stuff easier to deal with and there is also a growing number of things that work without Java libraries. Or even the JVM. I use it with Spring Boot. But we also have a lot of kotlin-js code running in a browser. And I use quite a few multiplatform libraries for Kotlin that work pretty much anywhere. I've even written a few myself. It's pretty easy to write portable code in Kotlin these days.

For example ktor works on the JVM but you can also build native applications with it. And I use ktor client in the browser. When running in the browser it uses the browser fetch API. When running on the jvm you can configure it to use any of a wide range of Java http clients. On native it uses curl.

Re: Java Virtual Threads: A Case Study

#43

Post by xxs »

taspeotis wrote: post #26

My rough understanding is that this is similar to async/await in .NET? It’s a shame this article paints a neutral (or even negative) experience with virtual threads. We rewrote a boring CRUD app that spent 99% of its time waiting the database to respond to be async/await from top-to-bottom. CPU and memory usage went way down on the web server because so many requests could be handled by far fewer threads.

>My rough understanding is that this is similar to async/await in .NET?

No, the I/O is still blocking with respect to the application code.

Re: Java Virtual Threads: A Case Study

#44

Post by [deleted] »

nlitened wrote: post #10
fzeindl wrote: post #4

Earlier quoted context omitted.

Does it shake out to any real advantage? To put it shortly: Writing single-threaded blocking code is far easier for most people and has many other benefits, like more understandable and readable programs: https://www.youtube.com/watch?v=449j7oKQVkc The main reason why non-blocking IO with it's style of intertwining concurrency and algorithms came along is that starting a thread for every request was too expensive. Wi…

> is far easier for most people I’d say that writing single-threaded code is far easier for _all_ people, even async code experts :) Also, single-threaded code is supported by programming language facilities: you have a proper call stack, thread-local vars, exceptions bubbling up, structured concurrency, simple resource management (RAII, try-with-resources, defer). Easy to reason and debug on language level. Async ru…

[deleted]

Re: Java Virtual Threads: A Case Study

#45

Post by zokier »

hashmash wrote: post #20
packetlost wrote: post #12

Earlier quoted context omitted.

Yeah, and it's generally good to be RAM limited instead of CPU, no? The alternative is blowing a bunch of time on syscalls and OS scheduler overhead. Also the virtual threads run on a "traditional" thread pool to my understanding, so you can just tweak the number of worker threads to cap the total concurrency. The benefit is it's overall more efficient (in the general case) and lets you write linear blocking code (as…

The OS scheduler is still there (for the carrier threads), but now you've added on top of that FJ pool based scheduler overhead. Although virtual threads don't have the syscall overhead when they block, there's a new cost caused by allocating the internal continuation object, and copying state into it. This puts more pressure on the garbage collector. Context switching cost due to CPU cache thrashing doesn't go away…

> The OS scheduler is still there (for the carrier threads), but now you've added on top of that FJ pool based scheduler overhead.

Ideally carrier threads would be pinned to isolated cpu cores, which removes most aspects of OS scheduler from the picture

Re: Java Virtual Threads: A Case Study

#46

Post by lichtenberger »

hashmash wrote: post #15
initplus wrote: post #11

Earlier quoted context omitted.

The benefits from virtual threads come from the simple API that it presents to the programmer. It's not a performance optimization.

But that same benefit was always available with platform threads -- a simple API. What is the real gain by using virtual threads? It's either going to be performance or memory utilization.

Throughput. The code can be "suspended" on a blocking call (I/O, where the platform thread usually is wasted, as the CPU has nothing to do during this time). So, the platform thread can do other work in the meantime.

Re: Java Virtual Threads: A Case Study

#47

Post by xxs »

nlitened wrote: post #10
fzeindl wrote: post #4

Earlier quoted context omitted.

Does it shake out to any real advantage? To put it shortly: Writing single-threaded blocking code is far easier for most people and has many other benefits, like more understandable and readable programs: https://www.youtube.com/watch?v=449j7oKQVkc The main reason why non-blocking IO with it's style of intertwining concurrency and algorithms came along is that starting a thread for every request was too expensive. Wi…

> is far easier for most people I’d say that writing single-threaded code is far easier for _all_ people, even async code experts :) Also, single-threaded code is supported by programming language facilities: you have a proper call stack, thread-local vars, exceptions bubbling up, structured concurrency, simple resource management (RAII, try-with-resources, defer). Easy to reason and debug on language level. Async ru…

>I’d say that writing single-threaded code is far easier for _all_ people, even async code experts :)

While 'async' is just a name, underneath it's epoll - and the virtual threads would not perform better than a proper NIO (epoll) server. I dont consider myself an 'async expert' but I have my share of writing NIO code (dare say not terrible at all)

Re: Java Virtual Threads: A Case Study

#48

Post by kimi »

taspeotis wrote: post #26

My rough understanding is that this is similar to async/await in .NET? It’s a shame this article paints a neutral (or even negative) experience with virtual threads. We rewrote a boring CRUD app that spent 99% of its time waiting the database to respond to be async/await from top-to-bottom. CPU and memory usage went way down on the web server because so many requests could be handled by far fewer threads.

It's more like Erlang threads - they appear to be blocking, so existing code will work with zero changes. But you can create a gazillion of them.

Re: Java Virtual Threads: A Case Study

#50

Post by giamma »

pansa2 wrote: post #39

Are these Virtual Threads the feature that was previously known as “Project Loom”? Lightweight threads, more-or-less equivalent to Go’s goroutines?

Yes, at EclipseCon 2022 an Oracle manager working on the Helidon framework presented their results replacing the Helidon core, which was based on Netty (and reactive programming) with Virtual Threads (using imperative programming). [1].

Unfortunately the slides from that presentation were not uploaded to the conference site, but this article summarizes [2] the most significant metrics. The Oracle guy claimed that by using Virtual Threads Oracle was able to implement, using imperative Java, a new engine for Helidon (called Nima) that had identical performance to the old engine based on Netty, which is (at least in Oracle's opinion) the top performing reactive HTTP engine.

The conclusion of the presentation was that based on Oracle's experience imperative code is much easier to write, read and maintain with respect to reactive code. Given the identical performance achieved with Virtual Threads, Oracle was going to abandon reactive programming in favor of imperative programming and virtual threads in all its products.

[1] https://www.eclipsecon.org/2022/sessions/helidon-nima-loom-b...

[2] https://medium.com/helidon/helidon-n%C3%ADma-helidon-on-virt...

Post reply on HN