171–180 of 199 posts
Earlier quoted context omitted.
In large numbers the cost of switching between threads does consume CPU while they're waiting for the database. This is why green threads exist, to have large numbers of in flight work executing over a smaller number of OS threads.
When using OS threads, there's no switching when they are waiting for a socket (db connection). The OS knows to wake the thread up only when there's something new to see on the connection.
What is the virtual thread / event loop pattern seeking to optimize? Is it context switching? A number of years ago I remember trying to have a sane discussion about “non blocking” and I remember saying “something” will block eventually no matter what… anything from the buffer being full on the NIC to your cpu being at anything less than 100%. Does it shake out to any real advantage?
It's a brave attempt to release the programmer from worrying or even thinking about thread pools and blocking code. Java has gone all in - they even cancelled a non-blocking rewrite of their database driver architecture because why have that if you won't have to worry about blocking code? And the JVM really is a marvel of engineering, it's really really good at what it does, so what team to better pull this off? So f…
My question is though: Why even do alleged “non-blocking” _at all_? What are people trying to optimize against?
Earlier quoted context omitted.
> So... What is it seeking to optimize? The goal is to maximize the number of tasks you can run concurrently, while imposing on the developers a low cognitive load to write and maintain the code. > Why did you need a thread pool before but not any more? You still need a thread pool. Except with virtual threads you are no longer bound to run a single task per thread. This is specially desirable when workloads are IO-b…
> why should you block a thread if creating gazillion threads on modern hardware is super cheap why not? I have transparency and debuggability what threads are running, can check stacktrace of each and what are they blocked on. virtual threads adds lots of magic under the hood, and if there will be some bug or lib in your infra with no vthreads support it is absolutely not clear how to debug it.
Virtual threads are a performance improvement over threads, no matter how cheap to create threads are. Virtual threads run on threads. If threads become cheaper to create, so do virtual threads. They are not mutually exclusive.
Virtual threads are on top of that a developer experience improvement. Code is easier to write and maintain.
Virtual threads improve throughput because the moment a task is waiting for anything like IO, the thread is able to service any other task in the queue.
Earlier quoted context omitted.
In large numbers the cost of switching between threads does consume CPU while they're waiting for the database. This is why green threads exist, to have large numbers of in flight work executing over a smaller number of OS threads.
When using OS threads, there's no switching when they are waiting for a socket (db connection). The OS knows to wake the thread up only when there's something new to see on the connection.
Earlier quoted context omitted.
It's been a really long time since I dealt with anything this low level, but in my very limited and ancient experience when people talk about context switching they're talking specifically about the userland process yielding execution back to the kernel so that the processor can be reassigned to a different process/thread. Naively, if the JVM isn't actually yielding control back to the kernel, it has the freedom to d…
When a real thread is allocated from a virtual thread to another, the JVM needs to save into the heap the stack of the first virtual thread and restore from the heap the stack of the second virtual thread, see slide 13 of [1]. This is in fact called mounting/unmounting as already pointed out, and occurs via Java Continuation, but from the JVM perspective this is a context switch. It's called JVM and the V stands for…
Earlier quoted context omitted.
This is a type of context switch. You are saying dollars are cheaper than money.
It's been a really long time since I dealt with anything this low level, but in my very limited and ancient experience when people talk about context switching they're talking specifically about the userland process yielding execution back to the kernel so that the processor can be reassigned to a different process/thread. Naively, if the JVM isn't actually yielding control back to the kernel, it has the freedom to d…
Earlier quoted context omitted.
If you need to be explicit about thread contexts because you're using a thread that's bound to some other runtime (say, a GL Context) or you simply want to use a single thread for synchronization like is common in UI programming with a Main/UI Thread, async/await does quite well. The async/await sugar ends up being a better devx than thread locking and implicit threading just doesn't cut it. In Java they're working o…
What’s stopping you from using a single thread for synchronization?
Async/await is able to achieve good UX around explicitly defining what goes on your Main thread and what goes elsewhere. Its trivial to mix UI thread and background thread code by bouncing between synchronization contexts as needed.
When the threading model is implicit its impossible to have this control.
Earlier quoted context omitted.
> why should you block a thread if creating gazillion threads on modern hardware is super cheap why not? I have transparency and debuggability what threads are running, can check stacktrace of each and what are they blocked on. virtual threads adds lots of magic under the hood, and if there will be some bug or lib in your infra with no vthreads support it is absolutely not clear how to debug it.
Each thread adds overhead. Some usage types don’t care, some do. From what I gather virtual threads are an alternative to “callback-hell” (js) or async coloring (python).
I suspect if you care about threads overhead, you won't pick Java, because there will be overhead in other areas too
> From what I gather virtual threads are an alternative to “callback-hell” (js) or async coloring (python).
there is also existing ExecutorService and futures in Java
Earlier quoted context omitted.
It's foolish to say that green threads are strictly better and ignore async/await as something outdated. It can do a lot that green threads can't. For example, you can actually share a thread with another runtime. Cooperative threading allows for implicit critical sections that can be cumbersome in preemptive threading. Async/await and virtual threads are solving different problems. > What is perhaps worse is that C#…
> It's foolish to say that green threads are strictly better and ignore async/await as something outdated I’m not sure I said outdated, but I can see what you mean by how I called Javas approach “more modern”. What I should have called Javas approach was “correctly designed”. C#’s async/await isn’t all terrible as you point out, but it’s designed wrong from the bottom up because computation should always be blocking…