Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Achieving 5M persistent connections with Project Loom virtual threads

github.com

Post reply on HN
Print view View original

71–80 of 150 posts

Re: Achieving 5M persistent connections with Project Loom virtual threads

#71

Post by namdnay »

chrisseaton wrote: post #62
lostcolony wrote: post #60

Earlier quoted context omitted.

One additional - as noted, it's been 26 years since Java's founding. Project Loom has been around since at least 2018 and still has no release date. It'll be cool for Java projects whenever it comes out, but I just...have a hard time caring right now. I can't use it for old codebases currently, and new codebases I'm not using one request per Java thread anyway (tbh - when it's my choice I'm not choosing the JVM at al…

> I'm not using one request per Java thread anyway The point is with Loom you can, and you can stop putting everything into a continuation and go back to straight-line code.

And hopefully we can bury Reactor Core in the garden and never talk about it again

Re: Achieving 5M persistent connections with Project Loom virtual threads

#72

Post by charcircuit »

mypalmike wrote: post #35
akvadrako wrote: post #10

Earlier quoted context omitted.

Actually every client IP+port / server IP+port pair. Linux uses 60999 − 32768 for ephemeral ports so can support 28e3^2 = 784 million connections per IP pair.

Except your service is almost certainly listening on one non-ephemeral port. But having "only" tens of thousands of connections per client is rarely a problem in practice, apart from some load testing scenarios (such as the experiment here, where they opened a number of ports so they could test a large number of connections with a single client machine).

1 IP can correspond to multiple different clients.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#73

Post by simulate-me »

zinxq wrote: post #32

Loom sets out to give you a sane programming paradigm similar to what threads do (i.e. as opposed to programming asynchronous I/O in Java with some type of callback) without the overhead of Operating System threads. That's a very cool and a noble pursuit. But the title of this article might as well have been "5M persistent connections with Linux" because that's where the magic 5M connections happen. I could also atte…

As the GP said, what's cool about this is how simple the code is. You might be able to achieve 5M connections in Java using an event loop based solution (eg Netty), but if the connection handlers need to do any async work, then they also need to be written using an event loop, which is not how most people write Java. Simply put, 5M connections was not possible using Java in the way most people write Java.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#74

Post by toast0 »

Nullabillity wrote: post #43
rvcdbn wrote: post #41

Earlier quoted context omitted.

Maybe threads don’t work for your thinking style but your claim that this is generally true is baseless and pretty well refuted by languages like Go or Erlang that feature stackfull threads/processes as a critical part of their best-in-class concurrency stories.

Erlang sidesteps the problem by avoiding mutable shared state, in this context they're threads/processes in name only. Go is just yet another implementation of green threads that is slightly less broken than prior implementations, because it had the benefit of being implemented on day 1 (so the whole ecosystem is green thread-aware). It's certainly nowhere near "best-in-class".

Shared mutable state is hard to work with, but Java threads and Java promises both give you access to it. In either case, you'd need discipline to avoid patterns which reduce concurrency.

From the article, it seems that Loom (in preview) enables the threaded model for Java to scale. IMHO, this is great because you can write simple straightforward code in a threaded model. You can certainly write complex code in a threaded model too. Maybe there's an argument that promises can be simple and straightforward too, but my experience with them hasn't been very straightforward.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#75

Post by vlovich123 »

pron wrote: post #33
bullen wrote: post #30

Earlier quoted context omitted.

Agreed it's simpler, but using NIO with one OS thread per core also has it's benefits. The context switch (how ever small) will cause latency when this solution is at saturation. I think they should write four tests: fiber, NIO and each with userspace networking (no kernel copying network memory) and compare them. Why Oracle is stalling removing the kernel for Java networking is surprising to me, they allready have a…

https://github.com/ebarlas/project-loom-comparison

Shouldn’t you be able to send authorization and authentication requests in parallel in the async and virtual threads cases?

Re: Achieving 5M persistent connections with Project Loom virtual threads

#77

Post by TYMorningCoffee »

I was only able to get to 840,000 open connections with my experiment. My machine only has 8GB of memory. https://josephmate.github.io/2022-04-14-max-connections/

Is there anyway for the TCP connections share memory in kernel space? My experiment only uses two 8 byte buffers in userspace.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#78

Post by invalidname »

isbvhodnvemrwvn wrote: post #61
invalidname wrote: post #2

This is pretty fantastic! I'm very excited about the possibilities of Loom. Would love to have a more realistic sample with Spring Boot that would demonstrate the real world scale. I saw a few but nothing remotely as ambitious as that.

Spring Boot overhead would likely make that infeasible.

I'm not saying 5M. I just want to see to what scale it would get without threading issues. Spring Boot isn't THAT heavy.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#79

Post by pron »

lostcolony wrote: post #60
mike_hearn wrote: post #53

Earlier quoted context omitted.

A couple of points to consider. 1. Demanding scalability for inappropriate projects and at any cost is something I've seen too, and on investigation it was usually related to former battle scars. A software system that stops scaling at the wrong time can be horrific for the business. Some of them never recover, the canonical example being MySpace, but I've heard of other examples that were less public. In finance ent…

One additional - as noted, it's been 26 years since Java's founding. Project Loom has been around since at least 2018 and still has no release date. It'll be cool for Java projects whenever it comes out, but I just...have a hard time caring right now. I can't use it for old codebases currently, and new codebases I'm not using one request per Java thread anyway (tbh - when it's my choice I'm not choosing the JVM at al…

> and still has no release date

JEP 425 has been proposed to target JDK 19, out September 20. It will first be a "Preview" feature, which means supported but subject to change, and if all goes well would normally be out of Preview two releases, i.e. one year, after that.

> I'm not using one request per Java thread anyway

You don't have to, but not that only the thread-per-request model offers you world-class observability/debuggability.

> other than "ugh, this again".

Ok, although in 2022, the Java platform is still among the most technologically advanced, state-of-the art, software plarform out there. It stands shoulder to shoulder with clang and V8 on compilation, and beats everything else on GC and low-overhead observability (yes, even eBPF).

Post reply on HN