Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We switched to Java 21 virtual threads and got a deadlock in TPC-C for Postgres

blog.ydb.tech

Post reply on HN
Print view View original

101–110 of 253 posts

Re: We switched to Java 21 virtual threads and got a deadlock in TPC-C for Postgres

#101

Post by spintin »

NovaX wrote: post #10

It is a known caveat that virtual threads do not work well with long running synchronization by pinning the thread. That unfortunately means that for many applications it may be premature to adopt them, but it is mature enough for broader evaluation by the libraries and frameworks. The Java team provided a status of their efforts recently [1]. https://www.youtube.com/watch?v=WoQJnnMIlFY&t=421s

https://www.youtube.com/watch?v=WoQJnnMIlFY&t=260s

To get the whole context, so virtual threads are unusable?

What holds a monitor by default and is there a workaround?

Found more:

    A virtual thread cannot be unmounted during blocking operations when it is pinned to its carrier. A virtual thread is pinned in the following situations:

    The virtual thread runs code inside a synchronized block or method

    The virtual thread runs a native method or a foreign function (see Foreign Function and Memory API)
For those those that don't know what this means: Blocking network TCP IO needs a sychronized block to work = you can't use virtual threads for networking. I wish they formulated it like that from the start!

Atleast now we know what they meant with don't use virtual threads for anything but tasks So for now manual NIO is still the king of the hill.

We are reaching peak humanity levels of complexity!

Re: We switched to Java 21 virtual threads and got a deadlock in TPC-C for Postgres

#102

Post by samus »

altfredd wrote: post #85
dikei wrote: post #52

Earlier quoted context omitted.

No, synchronized is a very primitive lock implementation compared to what's available in java.util.concurrent.Locks. However, it's built directly into the JVM specification, so it's difficult to change while keeping compatibility, while j.u.c.Locks is just a library. In other words, they can't change synchronized schematic, so they created j.u.c.Locks as a replacement.

> it's difficult to change while keeping compatibility Actually, it is trivial to change. Just embed a ReentrantLock into every object and rewrite all calls to "synchronized"/"Object.wait" to use that lock. Unfortunately, this would result in a bit of performance regression (increasing per-object memory footprint). To solve that would require turning ReentrantLock into a magical intrinsic, fully integrated with lock…

... was taking years to land Project Loom. So long that people started calling it vaporware. Project Valhalla is still regarded as such by many. It had to be shipped as soon as it was usable, even though a few rough edges remain that really ought to be deburred.

As you indicate, the complexity lies in not burning too many bridges with existing users and use cases. This is something that Android regularly does and which Go never really had to do due to its shorter history and up-front design.

Re: We switched to Java 21 virtual threads and got a deadlock in TPC-C for Postgres

#103

Post by samus »

lmm wrote: post #75
adra wrote: post #12

Earlier quoted context omitted.

This one was a very large caveat pointed out loudly every time the feature was mentioned to the community. So, yes the limitation/flaw was very well known and is ideally going to be addressed in a future JDK release.

> This one was a very large caveat pointed out loudly every time the feature was mentioned to the community. It really wasn't. There were people on here, including Oracle employees, claiming that the virtual thread implementation was a drop-in replacement that would work (not necessarily perform better, but work) in all cases.

And it indeed does in common usage scenarios. And also in this case once the issue with `synchronized` is resolved. After all, this is a benchmark and it's not surprising that one of the limitations of the design was hit.

Re: We switched to Java 21 virtual threads and got a deadlock in TPC-C for Postgres

#104

Post by jillesvangurp »

samus wrote: post #99
dikei wrote: post #52

Earlier quoted context omitted.

No, synchronized is a very primitive lock implementation compared to what's available in java.util.concurrent.Locks. However, it's built directly into the JVM specification, so it's difficult to change while keeping compatibility, while j.u.c.Locks is just a library. In other words, they can't change synchronized schematic, so they created j.u.c.Locks as a replacement.

The j.u.c.*Locks exist for a very long time already.

Since Java 1.5 in 2004. Twenty years this year. Before that it existed as a separate library developed by Doug Lee. I remember using that library before Java 1.5 was released. There is quite a bit of Java code out there that predates that of course. Also, lots of people continued to not grasp the essentials of that library and stuck with the primitives they knew. So there's a lot of code there with synchronized blocks post Java 1.5 instead of the more robust concurrency primitives that came with the java.util.concurrent package.

Re: We switched to Java 21 virtual threads and got a deadlock in TPC-C for Postgres

#105

Post by NovaX »

spintin wrote: post #101
NovaX wrote: post #10

It is a known caveat that virtual threads do not work well with long running synchronization by pinning the thread. That unfortunately means that for many applications it may be premature to adopt them, but it is mature enough for broader evaluation by the libraries and frameworks. The Java team provided a status of their efforts recently [1]. https://www.youtube.com/watch?v=WoQJnnMIlFY&t=421s

https://www.youtube.com/watch?v=WoQJnnMIlFY&t=260s To get the whole context, so virtual threads are unusable? What holds a monitor by default and is there a workaround? Found more: A virtual thread cannot be unmounted during blocking operations when it is pinned to its carrier. A virtual thread is pinned in the following situations: The virtual thread runs code inside a synchronized block or method The virtual thread…

Perhaps it’s better to say that they are not yet general purpose. There are many caveats which need to be resolved and are being actively worked on. I would not use them broadly yet, but that could change rapidly.

A monitor will pin the VT to the carrier thread. That can have surprising incompatibility in the current jdk. Soon these footguns will be fixed and you can use them worry free.

https://www.reddit.com/r/java/comments/1512xuo/virtual_threa...

Re: We switched to Java 21 virtual threads and got a deadlock in TPC-C for Postgres

#106

Post by kaba0 »

lmm wrote: post #75
adra wrote: post #12

Earlier quoted context omitted.

This one was a very large caveat pointed out loudly every time the feature was mentioned to the community. So, yes the limitation/flaw was very well known and is ideally going to be addressed in a future JDK release.

> This one was a very large caveat pointed out loudly every time the feature was mentioned to the community. It really wasn't. There were people on here, including Oracle employees, claiming that the virtual thread implementation was a drop-in replacement that would work (not necessarily perform better, but work) in all cases.

If your program was prone to deadlock as is, and is just more easily happening with virt threads, it means that the problem is your code.

Re: We switched to Java 21 virtual threads and got a deadlock in TPC-C for Postgres

#107

Post by davidgerard »

mrintegrity wrote: post #56

Totally off topic but I am getting tired of the AI generated images used on nearly all blog posts nowadays. They are instantly recognisable, it just seems low effort and lowers the feeling of quality one might otherwise have

exactly. It says "I can't be bothered producing this" and I feel like, so why should I be bothered reading it?

Re: We switched to Java 21 virtual threads and got a deadlock in TPC-C for Postgres

#108

Post by kelnos »

mrintegrity wrote: post #56

Totally off topic but I am getting tired of the AI generated images used on nearly all blog posts nowadays. They are instantly recognisable, it just seems low effort and lowers the feeling of quality one might otherwise have

I dislike the style this particular author chose, but don't object in general. Assuming the images are actually somewhat relevant (or at least funny), I think I'd prefer an AI-generated image over a big wall of text.

To each their own, though, of course.

Re: We switched to Java 21 virtual threads and got a deadlock in TPC-C for Postgres

#109

Post by kaba0 »

Groxx wrote: post #91
lmm wrote: post #76

Earlier quoted context omitted.

Java's virtual threads are supposed to be a drop-in replacement for real threads. But using virtual threads means you get a far smaller number of real threads, and things that were safe back when you had an unlimited number of real threads available (or at least, a larger number than your database connection pool) are no longer safe.

Shouldn't you be able to use the same number of real threads though, plus some additional effectively-threads for the virtual threads that are not pinned? Doesn't seem like this should change semantics there, so the risk would be code that changes because of perceived advantages which are not true in edge cases - that's new behavior that wasn't possible before, there aren't really any existing semantics to break. If…

That sort of what happens, there is just a configurable hard limit on how much new thread may be created that was hit by this benchmark.

As mentioned in another comment: jdk.virtualThreadScheduler.maxPoolSize

Re: We switched to Java 21 virtual threads and got a deadlock in TPC-C for Postgres

#110

Post by samus »

extractionmech wrote: post #43
Groxx wrote: post #34

Earlier quoted context omitted.

From a relatively brief skim and past Go and Java experience: synchronized blocks the current normal thread, so that doesn't really seem any different to me. If you starve your threads, you starve your threads. It definitely leaves room to optimize by not pinning that thread, which would be great, but that shouldn't change semantics at all. Or is there something actually screwed up in the implementation of virtual th…

It’s a thread that supports n virtual threads. You want synchronized in virtual thread a and not the carrier thread which will block all the virtual threads. Been away from Java land for a while. How did something like that even get into release? That’s like a pretty big loaded shotgun to leave lying around with lots of kids playing, no?

It's an explicitly documented shortcoming of the existing implementation that will be fixed soon. I knew immediately from the title of TA what probably happened. The other similar limitations (CPU-bound tasks, native calls) seem much more severe, but are ultimately unsolvable. Meanwhile, the issue with synchronized is regarded as a scalability bottleneck since the JDK is supposed to temporarily spawn additional platform threads. This behavior can be controlled via the system property `jdk.virtualThreadScheduler.maxPoolSize`.

Also, this is a benchmark. It's not surprising that they managed to produce a situation where more than n_cores virtual threads would actually start waiting.

Post reply on HN