Java Virtual Threads: A Case Study

Post by kaba0 » Wed, Jul 17, 2024, 6:47 PM UTC

derriz wrote: post #41

bheadmaster wrote: post #24

Earlier quoted context omitted.

> Async runtimes are always complicated, filled with leaky abstractions, it’s like another language that one has to learn in addition, but with a less thought-out, ad-hoc design. Difficult to reason and debug, especially in edge cases Async runtimes themselves are simply attempts to bolt-on green threads on top of a language that doesn't support them on a language level. In JavaScript, async/await uses Promises to en…

Indeed. Async runtimes/sytles are attempts to provide a more readable/useable syntax for CPS[1]. CPS originally had nothing to do with blocking/non-blocking or multi-threading but arose as a technique to structure compiler code. Its attraction for non-blocking coding is that it allows hiding the multi-threaded event dispatching loop. But as the parent comment suggests, this abstraction is extremely leaky. And in addi…

Post by twic » Wed, Jul 17, 2024, 7:03 PM UTC

immibis wrote: post #8

gregopet wrote: post #7

Earlier quoted context omitted.

It's a brave attempt to release the programmer from worrying or even thinking about thread pools and blocking code. Java has gone all in - they even cancelled a non-blocking rewrite of their database driver architecture because why have that if you won't have to worry about blocking code? And the JVM really is a marvel of engineering, it's really really good at what it does, so what team to better pull this off? So f…

So... What is it seeking to optimize? Why did you need a thread pool before but not any more? What resource was exhausted to prevent you from putting every request on a thread?

Post by kllrnohj » Wed, Jul 17, 2024, 7:07 PM UTC

brabel wrote: post #88

immibis wrote: post #70

Earlier quoted context omitted.

It seems that the answer to the question was "memory". Stack allocations, presumably. You have answered by telling us that virtual threads are better than real threads because real threads suck, but you didn't say why they suck or why virtual threads don't suck in the same way.

OS Threads do not suck, they're great. But they are expensive to create as they require a syscall, and they're expensive to maintain as they consume quite a bit of memory just to exist, even if you don't need it (due to how they must pre-allocate a stack which apparently is around 2MB initially, and can't be made smaller as in most cases you will need even more, so it would make most cases worse). Virtual Threads are…

Post by he0001 » Wed, Jul 17, 2024, 7:13 PM UTC

jayd16 wrote: post #110

he0001 wrote: post #51

Earlier quoted context omitted.

> My rough understanding is that this is similar to async/await in .NET? The biggest difference is that C# async/await code is rewritten by the compiler to be able to be async. This means that you see artifacts in the stack that weren’t there when you wrote the code. There are no rewrites with virtual threads and the code is presented on the stack just as you write it. They solve the same problem but in very differen…

There is overlap but they really don't solve the same problem. Cooperative threading has its own advantages and patterns that won't be served by virtual threads.

Post by joshlemer » Wed, Jul 17, 2024, 7:37 PM UTC

LinXitoW wrote: post #57

From my very limited exposure to virtual threads and the older solution (thread pools), the biggest hurdle was the extensive use of ThreadLocals by most popular libraries. In one project I had to basically turn a reactive framework into a one thread per request framework, because passing around the MDC (a kv map of extra logging information) was a horrible pain. Getting it to actually jump ship from thread to thread…

    class MyExecutor implements Executor {
        private final Executor delegate;
        public MyExecutor(Executor delegate) {
            this.delegate = delegate;
        }
        @Override
        public void execute(@NotNull Runnable command) {
            var mdc = MDC.getCopyOfContextMap();
            delegate.execute(() -> {
                MDC.setContextMap(mdc);
                try {
                    command.run();
                } finally {
                    MDC.clear();
                }
            });
        }
    }