Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Achieving 5M persistent connections with Project Loom virtual threads

github.com

Post reply on HN
Print view View original

141–150 of 150 posts

Re: Achieving 5M persistent connections with Project Loom virtual threads

#141

Post by jonpez2 »

pron wrote: post #140
jonpez2 wrote: post #138

Earlier quoted context omitted.

Let me preface by saying I am a Johnny-come-lately loom fanboy. Amazing work and huge impact. Re structured concurrency: I wonder if there’s any way to combine with generic exceptions such that we can not force a wrapping exception class. So maybe have an executor class that’s generic on the thrown exception type, and then have the join or get apis explicitly throw that type? This thought process is inspired by the g…

Convenient polymorphism over exceptions is something I would very much like to see in Java, but it's a separate topic. Given that structured concurrency is normally used with things that can fail, and whose failures must be handled, I hope (and think) you'll find that the use of checked exceptions is not onerous at all. If we're mistaken, we can consider solutions during the incubation period.

Totally agree that we need explicit handling of the concurrency-specific exceptions like interruptedexception. It’s just that concurrent apis by their nature take callable/runnable apis which lose any formality over exceptions thrown by client code, and thus someone up the stack is always forced to write a catch( Throwable ) block. So the concurrency leaks up the stack, and forces unsafe default clauses. You’re clearly correct that the topic is separate, but it has great impact on the leakiness of these apis.

Thanks for the response and the amazing work!

Re: Achieving 5M persistent connections with Project Loom virtual threads

#142

Post by blibble »

bullen wrote: post #136
blibble wrote: post #56

Earlier quoted context omitted.

there's still a context switch with NIO, you're just doing it manually

The context switches in NIO is between fewer threads, you just need one per core. Memory contention is also playing into this. The benchmark they made is asking the question in a way that it leans into the answer they need, just like 99% of all human activity it's biased.

you're missing the point and taking "context switch" literally

with NIO you are still managing the stack, just yourself instead of letting the operating system do it for you

it is still a "context switch", just done in your code instead of the OS

and that's not free (and likely more expensive than saving and restoring a set of registers)

Re: Achieving 5M persistent connections with Project Loom virtual threads

#143

Post by wiseowise »

ferdowsi wrote: post #28
wiseowise wrote: post #26

And how is that any different from Kotlin coroutines if you still need to call Thread.startVirtualThread?

Kotlin coroutines are colored and infect your whole codebase. Virtual threads do not.

You can mark everything suspend and there's no difference.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#144

Post by kmelva »

midislack wrote: post #106

I see a lot of these making the FP of HN. But it's very difficult to be impressed, or unimpressed because it's all about hardware. How much hardware is everybody throwing at all of this? 5M persistent connections on a Pi with mere GigE? Pretty frickin' amazing. 5M persistent connections on a Threadripper with 128 cores and a dozen trunked 4 port 10GE NICs? Yaaaaawwwnnn snooze. We need a standardized computer for benc…

Could a 128c Threadripper even do 5M kernel threads?

Re: Achieving 5M persistent connections with Project Loom virtual threads

#145

Post by jpollock »

midislack wrote: post #106

I see a lot of these making the FP of HN. But it's very difficult to be impressed, or unimpressed because it's all about hardware. How much hardware is everybody throwing at all of this? 5M persistent connections on a Pi with mere GigE? Pretty frickin' amazing. 5M persistent connections on a Threadripper with 128 cores and a dozen trunked 4 port 10GE NICs? Yaaaaawwwnnn snooze. We need a standardized computer for benc…

This isn't about the hardware, it's about thread count.

There are limits in the linux kernel, and the 5m concurrent connections was chosen to exceed it.

From what I remember (my knowledge is ancient though), a Java thread consumes a pid_t in the linux kernel. By default this is limited to 64k. However, this can be increased by setting a flag in the kernel, to a maximum 2^22 or 4m.

In order to have more than 4m connections, the existing Java code either needs to be changed to be event driven, or it can't use kernel threads.

Event driven code is very different. It's very powerful, but it is very easy to get lost. Think writing Java code that looks like a Makefile with dependencies or "andThen" everywhere, and everyone having to make sure everything is threadsafe. Thread safety is hard for large teams with high qps services - deadlocks can bring down a service.

If a developer can write "regular" non-re-entrant Java code and still get the concurrent connections? Win all around.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#146

Post by [deleted] »

charcircuit wrote: post #72
mypalmike wrote: post #35

Earlier quoted context omitted.

Except your service is almost certainly listening on one non-ephemeral port. But having "only" tens of thousands of connections per client is rarely a problem in practice, apart from some load testing scenarios (such as the experiment here, where they opened a number of ports so they could test a large number of connections with a single client machine).

1 IP can correspond to multiple different clients.

[deleted]

Re: Achieving 5M persistent connections with Project Loom virtual threads

#147

Post by mypalmike »

charcircuit wrote: post #72
mypalmike wrote: post #35

Earlier quoted context omitted.

Except your service is almost certainly listening on one non-ephemeral port. But having "only" tens of thousands of connections per client is rarely a problem in practice, apart from some load testing scenarios (such as the experiment here, where they opened a number of ports so they could test a large number of connections with a single client machine).

1 IP can correspond to multiple different clients.

With NAT, right. I guess there may be situations where client ports for a single server IP dry up due to NAT, but I've not encountered that issue.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#148

Post by abollaert »

namdnay wrote: post #71
chrisseaton wrote: post #62

Earlier quoted context omitted.

> I'm not using one request per Java thread anyway The point is with Loom you can, and you can stop putting everything into a continuation and go back to straight-line code.

And hopefully we can bury Reactor Core in the garden and never talk about it again

... at which point we can also "undeprecate" RestTemplate and pretend that never happened either :-)

Re: Achieving 5M persistent connections with Project Loom virtual threads

#149

Post by richdougherty »

kaba0 wrote: post #134
richdougherty wrote: post #130

Earlier quoted context omitted.

> Coroutines are much less coloured I think another commenter pointed out that they are still coloured though. Still, they're very cool - and you can use them for more than just lightweight threading. > As for the tailrecursive keyword, it is not a constraint but a feature since it guarantee at the type level that this function cannot stack overflow I'd say tailrecursive is compiler feature (codegen the recursion int…

Doesn’t it possibly get inlined by current mechanisms as well?

Can you explain a bit more?

Re: Achieving 5M persistent connections with Project Loom virtual threads

#150

Post by kaba0 »

richdougherty wrote: post #149
kaba0 wrote: post #134

Earlier quoted context omitted.

Doesn’t it possibly get inlined by current mechanisms as well?

Can you explain a bit more?

I am absolutely a novice at this level of detail with the current OpenJDK implementation, so I’m only asking it, but wouldn’t a method call as a last operation be also a target of the basic inlining done by the JIT compilers? How is a non-recursive tail-call any different than an inline at any another location?
Post reply on HN