Live data from Hacker News

Java Virtual Threads: A Case Study

infoq.com

151–160 of 199 posts

Re: Java Virtual Threads: A Case Study

#151
post #141
post #137

Earlier quoted context omitted.

Virtual threads could be scheduled pre-emptively but currently the scheduler will wait for some kind of thread sleep to schedule another virtual thread. That's just a scheduler implementation detail and the spec is such that a time slice scheduler could be implemented.

Yes, but the problem is that the spec is such that preemptive blocking doesn't need to be implemented. That means that Java programmers have to be very careful when writing code, lest they block the entire underlying (OS) thread! Again, Go already went through that experience. It was painful. Java should have learned and implemented it from the start

I don't know. The language already has Thread.Yield. If your use case is such that you have starvation and care about it, it seems trivial to work around.

Still, an annoying gotcha if it hits you unexpectedly.

Re: Java Virtual Threads: A Case Study

#152
post #63

Virtual threads do one thing: they allow creating lots of threads. This helps throughput due to Little's law [1]. But because this server here saturates the CPU with only a few threads (it doesn't do the fanout modern servers tend to do), this means that no significant improvements can be provided by virtual threads (or asynchronous programming, which operates on the same principle) while keeping everything else in t…

This take sounds reasonable to me. But I'm not an expert, and I'd be curious to hear an opposing view if there's one.

Standard/OS threads in Java use about a megabyte of memory per thread, so running 256 threads uses about 256 MB of memory before you've even started allocating things on the heap.

Virtual threads are therefore useful if you're writing something like a proxy server, where you want to allow lots of concurrent connections, and you want to use the familiar thread-per-connection programming model.

Re: Java Virtual Threads: A Case Study

#153
post #63

Virtual threads do one thing: they allow creating lots of threads. This helps throughput due to Little's law [1]. But because this server here saturates the CPU with only a few threads (it doesn't do the fanout modern servers tend to do), this means that no significant improvements can be provided by virtual threads (or asynchronous programming, which operates on the same principle) while keeping everything else in t…

I guess at least their work has confirmed what we probably already knew intuitively: if you have CPU-intensive tasks, without waiting on anything, and you want to execute these concurrently, use traditional threads. The advice "don't use virtual threads for that, it will be inefficient" really does need some evidence. Mildly infuriating though that people may read this and think that somehow the JVM has problems in i…

[deleted]

Re: Java Virtual Threads: A Case Study

#154
post #140

Earlier quoted context omitted.

Greenlets ultimately have to be scheduled onto system threads at the end of the day unless you have a lightweight thread model of some sort supported by the OS, so it’s a little bit misleading depending on how far down the stack you want to think about optimizing for greenlets. You could potentially have a poor implementation of task scheduling for some legacy compatibility reason, however. I guess I’d be curious abo…

Even though yes, in the end you have to map onto system threads, there are still quite a fee things you can do. But this is infeasible for Java, unfortunately. For example, in Erlang the entire VM is built around green threads with a huge amount of guarantees and mechanisms: https://news.ycombinator.com/item?id=40989995 When your entire system is optimized for green threads, the question of "it still needs to map ont…

I really don’t think it’s useful to be this nonspecific. You could give an example of what a Java greenlet cannot do or how it cannot be optimized, for example. If your whole point is actually just “I prefer the semantics of BEAM threads”, then just say that.

Re: Java Virtual Threads: A Case Study

#155

Earlier quoted context omitted.

This take sounds reasonable to me. But I'm not an expert, and I'd be curious to hear an opposing view if there's one.

Standard/OS threads in Java use about a megabyte of memory per thread, so running 256 threads uses about 256 MB of memory before you've even started allocating things on the heap. Virtual threads are therefore useful if you're writing something like a proxy server, where you want to allow lots of concurrent connections, and you want to use the familiar thread-per-connection programming model.

Only address space of 1 MB is reserved (which can still be a problem), actual memory usage is limited to the memory pages that are actually accessed by the program within that address space.

Re: Java Virtual Threads: A Case Study

#156
post #134
post #39

Are these Virtual Threads the feature that was previously known as “Project Loom”? Lightweight threads, more-or-less equivalent to Go’s goroutines?

They're not equivalent to Go's goroutines. Go's goroutines are preemptive (and Go's development team went through a lot of pain to make them such). Java's lightweight threads aren't. Java's repeating the same mistakes that Go made (and learned from) 10 years ago.

I would put it more charitably as "Java Virtual Threads are new and have not seen massive use and optimization yet".

This is crucial, because Java wouldn't necessarily require the same optimizations Go needed.

Making Virtual Threads fully preemptive could be useful, but it's probably not as crucial as it was for Go.

Go does not have a native mechanism to spawn OS threads that are separate from the scheduler pool, so if you want to run a long CPU-heavy task, you can only run it on the same pool as you run your I/O-bound Goroutines. This could lead to starvation, and adding partial preemption and later full preemption was a neat way to solve that issue.

On the other hand, Java still has OS threads, so you can put those long-running CPU-bound tasks on a separate thread-pool. Yes, it means programmers need to be extra careful with the type of code they run on Virtual Threads, but it's not the same situation as Go faced: in Java they DO have a native escape hatch.

I'm not saying a preemptive scheduler won't be helpful at Java, but it just isn't as direly needed as it was with Go. One of the most painful issues with Java Virtual Threads right now is thread pinning when a synchronized method call is executed. Unfortunately, a lot of existing Java code is heavily using synchronized methods[1], so it's very easy to unknowingly introduce a method call that pins an OS thread. Preemeptive could solve this issue, but it's not the only way to solve it.

---

[1] One of my pet peeves with the Java standard library is that almost any class or method that was added before Java 5 is using synchronized methods excessively. One of the best examples is StringBuffer, the precursor of StringBuilder, where all mutating methods are synchronized, as if it was a common use case to build a string across multiple threads. I'm still running into StringBuffers today in legacy codebases, but even newer codebases tend to use synchronized methods over ReentrantLocks or atomic operations, since they're just so easy to use.

Re: Java Virtual Threads: A Case Study

#157
post #63

Virtual threads do one thing: they allow creating lots of threads. This helps throughput due to Little's law [1]. But because this server here saturates the CPU with only a few threads (it doesn't do the fanout modern servers tend to do), this means that no significant improvements can be provided by virtual threads (or asynchronous programming, which operates on the same principle) while keeping everything else in t…

It goes deeper than Little's Law. Every decent textbook on introductory queuing theory has the result that on a normalized basis, fast server > multi-server > multi-queue. That analysis admits almost arbitrary levels of depth of analysis and still holds true.

Your observation that computing architectures have chased fast server for decades is apt. There's a truism in computing that those who build systems are doomed to relearn the lessons of the early ages of networks, whether they studied them in school or not. But kudos to whoever went through the exercise again.

Re: Java Virtual Threads: A Case Study

#158
post #8

Earlier quoted context omitted.

So... What is it seeking to optimize? Why did you need a thread pool before but not any more? What resource was exhausted to prevent you from putting every request on a thread?

> So... What is it seeking to optimize? The goal is to maximize the number of tasks you can run concurrently, while imposing on the developers a low cognitive load to write and maintain the code. > Why did you need a thread pool before but not any more? You still need a thread pool. Except with virtual threads you are no longer bound to run a single task per thread. This is specially desirable when workloads are IO-b…

> why should you block a thread

if creating gazillion threads on modern hardware is super cheap why not? I have transparency and debuggability what threads are running, can check stacktrace of each and what are they blocked on.

virtual threads adds lots of magic under the hood, and if there will be some bug or lib in your infra with no vthreads support it is absolutely not clear how to debug it.

Re: Java Virtual Threads: A Case Study

#159
post #128

Earlier quoted context omitted.

What patterns does async/await solve which virtual threads don’t?

"Green Threads" as implemented in Java is a solution that solves only a single problem - blocking/multiplexing. It does not enable easy concurrency and task/future composition the way C#/JS/Rust do, which offer strictly better and more comprehensive model.

Structured concurrency[1] offers task composition and more.

[1] https://openjdk.org/jeps/453

Re: Java Virtual Threads: A Case Study

#160
post #107
post #37

Earlier quoted context omitted.

> My rough understanding is that this is similar to async/await in .NET? Not really. What C# does is sort of similar but it has the disadvantages of splitting your code ecosystem into non-blocking/blocking code. This means you can “accidentally” start your non-blocking code. Something which may cause your relatively simple API to consume a ridiculous amount of resources. It also makes it much more complicated to upda…

It's foolish to say that green threads are strictly better and ignore async/await as something outdated. It can do a lot that green threads can't. For example, you can actually share a thread with another runtime. Cooperative threading allows for implicit critical sections that can be cumbersome in preemptive threading. Async/await and virtual threads are solving different problems. > What is perhaps worse is that C#…

> It's foolish to say that green threads are strictly better and ignore async/await as something outdated

I’m not sure I said outdated, but I can see what you mean by how I called Javas approach “more modern”. What I should have called Javas approach was “correctly designed”.

C#’s async/await isn’t all terrible as you point out, but it’s designed wrong from the bottom up because computation should always be blocking by default. The fact that you can accidentally start running your code asynchronous is just… Aside from trapping developers with simple mistakes, it’s also part of what has lead to the ecosystem irrecoverably being split into two.

I was actually a little surprised to see Microsoft make their whole .Net to .Net core without addressing some of the glaring issues with it, when that massive disruption process uprooted everything anyway.

Post reply on HN