Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Java Virtual Threads: A Case Study

infoq.com

Post reply on HN
Print view View original

181–190 of 199 posts

Re: Java Virtual Threads: A Case Study

#181

Post by riku_iki »

chipdart wrote: post #173
riku_iki wrote: post #158

Earlier quoted context omitted.

> why should you block a thread if creating gazillion threads on modern hardware is super cheap why not? I have transparency and debuggability what threads are running, can check stacktrace of each and what are they blocked on. virtual threads adds lots of magic under the hood, and if there will be some bug or lib in your infra with no vthreads support it is absolutely not clear how to debug it.

> if creating gazillion threads on modern hardware is super cheap why not? Virtual threads are a performance improvement over threads, no matter how cheap to create threads are. Virtual threads run on threads. If threads become cheaper to create, so do virtual threads. They are not mutually exclusive. Virtual threads are on top of that a developer experience improvement. Code is easier to write and maintain. Virtual…

> Virtual threads are on top of that a developer experience improvement. Code is easier to write and maintain.

except now you need to prove somehow that all 100 libs in your project support virtual threads.

> Virtual threads improve throughput because the moment a task is waiting for anything like IO, the thread is able to service any other task in the queue.

from reading similar discussions, linux for example doesn't have true IO async API, you just push lock of Java thread to lock of thread in the kernel

Re: Java Virtual Threads: A Case Study

#182

Post by troupo »

kbolino wrote: post #170
troupo wrote: post #138

Earlier quoted context omitted.

Erlang's concurrency story isn't green threads. It's (with caveats, of course): - a thread crashing will not bring the system down - a thread cannot hog all processing time as the system ensures all threads get to run. The entire system is re-entrant and execution of each thread can be suspended to let other threads continue - all CPU cores can and will be utilized transparently to the user - you can monitor a thread…

Neither an error nor a recovered-from panic will cause a Go program to crash; only an unrecovered panic does that. The bigger problem with Go in this regard is how easy it is to cause a panic thanks to nil.

In Erlang even a nil will not lead to an unrecovered panic (if it happens in the process aka green thread).

Go made half a step in the right direction with goroutines, but never committed fully

Re: Java Virtual Threads: A Case Study

#183

Post by troupo »

immibis wrote: post #177
troupo wrote: post #143

Earlier quoted context omitted.

Because they don't create 4k real threads, and can be scheduled on n=CPU Cores OS threads

4k "real" threads can also be scheduled on 4 CPU cores. What's the difference?

Real threads are extremely expensive both in terms of memory and CPU time compared to virtual threads. I think the main issue is not even that but context switching when switching threads which is also very expensive.

Virtual threads usually require significantly fewer resources to spawn and run. And, if the underlying system is implemented with them in mind, they can use fewer context switches, and possibly even fewer cache misses etc.

Re: Java Virtual Threads: A Case Study

#184

Post by kbolino »

troupo wrote: post #182
kbolino wrote: post #170

Earlier quoted context omitted.

Neither an error nor a recovered-from panic will cause a Go program to crash; only an unrecovered panic does that. The bigger problem with Go in this regard is how easy it is to cause a panic thanks to nil.

In Erlang even a nil will not lead to an unrecovered panic (if it happens in the process aka green thread). Go made half a step in the right direction with goroutines, but never committed fully

Each has its tradeoffs. I had a case that cropped up more than once where RabbitMQ kept on trucking even though the process for an important queue had crashed; had it propagated all the way to the server itself it may have been easier to diagnose and fix (I'm assuming there's something like defer or finally in Erlang to ensure the mnesia database was synced properly on exit). Instead, I had to monitor for this condition and periodically run some command-line trickery to fix it (without ever really knowing why it happened). This was years ago, maybe RabbitMQ handles that better now.

The Go authors are adamant that goroutines not be addressable (from without) or identifiable (from within). This is diametrically opposed to Erlang, where processes are meant to be addressed/identified. I can't say I've ever found a case where a problem couldn't be solved due to this constraint in Go, but it does complicate some things.

Re: Java Virtual Threads: A Case Study

#185

Post by munksbeer »

brabel wrote: post #85
bberrry wrote: post #62

Earlier quoted context omitted.

If you are already in a reactive framework, why would you change to virtual threads? Those frameworks pool threads and have their own event loop so I would say they are not suitable for virtual thread migration.

Yes, if you're happy with the reactive frameworks there's no reason to migrate. Most people, however, would love to remove their complexities from their code bases. Virtual Threads are much, much easier to program with. There's downsides, like not being able to easily limit concurrency, having to implement your own timeout mechanisms etc. but that will probably be provided by a common lib sooner or later which hopefu…

I've not looked too deeply. We use the eventloop model, and we're guaranteed that data is only mutated by a single unit of work at a time which means you don't need to use any concurrent data types, volatile etc. This is great for micro performance.

Does the same apply to virtual threads?

Edit: I think I answered my own question. Java virtual threads have the same memory model as regular java threads so yes, I need to use the same semantics. That rules replacing the eventloop model for us.

Re: Java Virtual Threads: A Case Study

#186

Post by jandrewrogers »

exabrial wrote: post #172
gregopet wrote: post #7

Earlier quoted context omitted.

It's a brave attempt to release the programmer from worrying or even thinking about thread pools and blocking code. Java has gone all in - they even cancelled a non-blocking rewrite of their database driver architecture because why have that if you won't have to worry about blocking code? And the JVM really is a marvel of engineering, it's really really good at what it does, so what team to better pull this off? So f…

Thank you for a A very candid response I enjoyed reading it! My question is though: Why even do alleged “non-blocking” _at all_? What are people trying to optimize against?

The short answer is that blocking is expensive due to the overhead of the implied context switch and poor locality. As computers become faster, a larger percentage of the CPU time is dedicated to context-switching overhead and non-blocking architectures eliminate that. For applications like databases where this problem is more severe, the difference in throughput between a blocking architecture and a non-blocking architecture can be 10x on the same hardware, so it is a very important optimization if you want your software to have performance that is competitive.

A modern thread-per-core shared-nothing architecture takes this even further and tries to eliminate blocking at the hardware level for the same basic reason.

Re: Java Virtual Threads: A Case Study

#187

Post by nimish »

Jtsummers wrote: post #146
nimish wrote: post #145

Earlier quoted context omitted.

This is a really unfortunate gotcha that's not at all obvious. Does it kick preemption up a layer to the OS then?

The "not at all obvious" gotcha is described in the documentation near the top, under the heading "What is a Virtual Thread?": https://docs.oracle.com/en/java/javase/21/core/virtual-threa... > Like a platform thread, a virtual thread is also an instance of java.lang.Thread. However, a virtual thread isn't tied to a specific OS thread. A virtual thread still runs code on an OS thread. However, when code running in a v…

All that says is the Java runtime will suspend on blocking IO, not that it _only_ suspends on blocking IO.

> what they're intended for

Java prides itself on careful and deliberate changes to eliminate foot guns, but this seems like a pretty major restriction. Usually these kinds of cooperative threads are called fibers or something else to distinguish them from truly preempt-able threads.

Expecting developers to read the minutiae of documentation (there's another restriction around synchronized blocks) is a fool's errand TBH. Principle of least surprise, etc.

Re: Java Virtual Threads: A Case Study

#188

Post by vbezhenar »

merb wrote: post #136
vbezhenar wrote: post #77

Earlier quoted context omitted.

R2DBC allows to efficiently maintain millions of connections to the database. But what database supports millions of connections? Not postgres for sure, and probably no other conventional database. So using reactive JDBC driver makes little sense, if you're going to use 1000 connections, 1000 threads will do just fine and bring little overhead. Those who use Java, don't care about spending 100 more MB of RAM when the…

Reactive drivers were not about 1000 connections, they were about reusing a single connection better, by queuing a little bit more efficient over a single connection. Reactive programming is not about parallelism, it’s about concurrency.

It is not possible to reuse a single connection better, if we're talking about postgres. You must conduct the transaction over a single connection and you cannot mix different transactions simultaneously over a single connection. That's the way postgres wire protocol works. I think that there's some rudimentary async capabilities, but they don't change anything fundamentally.

It might be different for some exotic databases, but I don't see any reason why ordinary JDBC driver couldn't reuse single TCP connection for multiple logical JDBC connections in this case.

Re: Java Virtual Threads: A Case Study

#189

Post by Fulgen »

riku_iki wrote: post #181
chipdart wrote: post #173

Earlier quoted context omitted.

> if creating gazillion threads on modern hardware is super cheap why not? Virtual threads are a performance improvement over threads, no matter how cheap to create threads are. Virtual threads run on threads. If threads become cheaper to create, so do virtual threads. They are not mutually exclusive. Virtual threads are on top of that a developer experience improvement. Code is easier to write and maintain. Virtual…

> Virtual threads are on top of that a developer experience improvement. Code is easier to write and maintain. except now you need to prove somehow that all 100 libs in your project support virtual threads. > Virtual threads improve throughput because the moment a task is waiting for anything like IO, the thread is able to service any other task in the queue. from reading similar discussions, linux for example doesn'…

> linux for example doesn't have true IO async API

io_uring has been around for a few years at this point, with vulnerabilities having been fixed to the point that it's fit for broadband usage.

Re: Java Virtual Threads: A Case Study

#190

Post by riku_iki »

Fulgen wrote: post #189
riku_iki wrote: post #181

Earlier quoted context omitted.

> Virtual threads are on top of that a developer experience improvement. Code is easier to write and maintain. except now you need to prove somehow that all 100 libs in your project support virtual threads. > Virtual threads improve throughput because the moment a task is waiting for anything like IO, the thread is able to service any other task in the queue. from reading similar discussions, linux for example doesn'…

> linux for example doesn't have true IO async API io_uring has been around for a few years at this point, with vulnerabilities having been fixed to the point that it's fit for broadband usage.

In this discussion people claim io_uring is not real async IO, but just another kernel level threadpool: https://news.ycombinator.com/item?id=38919659
Post reply on HN