Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Achieving 5M persistent connections with Project Loom virtual threads

github.com

Post reply on HN
Print view View original

21–30 of 150 posts

Re: Achieving 5M persistent connections with Project Loom virtual threads

#21

Post by cheradenine_uk »

I think a lot of people are missing the point.

Go look at the sourcecode. Look at how simple it is - anyone who has created a thread with java knows what's happening. With only minor tweaks, this means your pre-existing code can take advantage of this with, basically, no effort. And it retains all the debuggability of traditional java thread (I.e: a stack trace that makes sense!)

If you've spent any time at all dealing with the horrors of c# async/await (Why am I here? Oh, no idea) and it's doubling of your APIs to support function colouring - or, you've fought with the complexities of reactive solutions in the Java space -- often, frankly, in the name of "scalability" that will never be practically required -- this is a big deal.

You no longer have to worry about any of that.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#22

Post by jeroenhd »

jiggawatts wrote: post #14
wiradikusuma wrote: post #13

The experiment is about Java app, but the tweaks are at the O/S level. Does it mean any app (Java/not, Loom/not) can achieve target given correct tweak? Also, why are these not default for the O/S? What are we compromising by setting those values?

There's always trade-offs. It would be very rare for any server to reach even 100K concurrent connections, let alone 5M. Optimising for that would be optimising for the 0.000001% case at the expense of the common case. Some back of the envelope maths: https://www.wolframalpha.com/input?i=100+Gbps+%2F+5+million If the server had a 100 Gbps Ethernet NIC, this would leave just 20 kbps for each TCP connection. I could im…

20kbps should be sufficient for things like chat apps if you have the CPU power to actually process chat messages like that. Modern apps also require attachments and those will require more bandwidth, but for the core messaging infrastructure without backfilling a message history I think 20kbps should be sufficient. Chat apps are bursty, after all, leaving you with more than just the average connection speed in practice.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#25

Post by henrydark »

jeroenhd wrote: post #22
jiggawatts wrote: post #14

Earlier quoted context omitted.

There's always trade-offs. It would be very rare for any server to reach even 100K concurrent connections, let alone 5M. Optimising for that would be optimising for the 0.000001% case at the expense of the common case. Some back of the envelope maths: https://www.wolframalpha.com/input?i=100+Gbps+%2F+5+million If the server had a 100 Gbps Ethernet NIC, this would leave just 20 kbps for each TCP connection. I could im…

20kbps should be sufficient for things like chat apps if you have the CPU power to actually process chat messages like that. Modern apps also require attachments and those will require more bandwidth, but for the core messaging infrastructure without backfilling a message history I think 20kbps should be sufficient. Chat apps are bursty, after all, leaving you with more than just the average connection speed in pract…

I have a memory of some chat site, maybe discord, sending attachments to a different server, thus exchanging the bandwidth problem with extra system complexity

Re: Achieving 5M persistent connections with Project Loom virtual threads

#27

Post by jeroenhd »

henrydark wrote: post #25
jeroenhd wrote: post #22

Earlier quoted context omitted.

20kbps should be sufficient for things like chat apps if you have the CPU power to actually process chat messages like that. Modern apps also require attachments and those will require more bandwidth, but for the core messaging infrastructure without backfilling a message history I think 20kbps should be sufficient. Chat apps are bursty, after all, leaving you with more than just the average connection speed in pract…

I have a memory of some chat site, maybe discord, sending attachments to a different server, thus exchanging the bandwidth problem with extra system complexity

That's how I'd solve the problem. The added complexity isn't even that high, give the application an endpoint to push an attachment into a distributed object store of your choice, submit a message with a reference to the object and persist it the moment the chat message was sent. This could be done with mere bytes for the message itself and some very dumb anycast-to-s3 services in different data centers.

I'm sure I'm skipping over tons of complexity here (HTTP keepalives binding clients to a single attachment host for example) because I'm no chat app developer, but the theoretical complexity is still relatively low.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#29

Post by pron »

wiseowise wrote: post #26

And how is that any different from Kotlin coroutines if you still need to call Thread.startVirtualThread?

1. These are actual threads from the Java runtime's perspective. You can step through them and profile them with existing debuggers and profilers. They maintain stacktraces and ThreadLocals just like platform threads.

2. There is no need for a split world of APIs, some designed for threads and others for coroutines (so-called "function colouring"). Existing APIs, third-party libraries, and programs — even those dating back to Java 1.0 (just as this experiment does with Java 1.0's java.net.ServerSocket) — just work on millions of virtual threads.

Normally, you wouldn't even call Thread.startVirtualThread(), but just replace your platform-thread-pool-based ExecutorService with an ExecutorService that spawns a new virtual thread for each task (Executors.newVirtualThreadPerTaskExecutor()). For more details, see the JEP: https://openjdk.java.net/jeps/425

Re: Achieving 5M persistent connections with Project Loom virtual threads

#30

Post by bullen »

cheradenine_uk wrote: post #21

I think a lot of people are missing the point. Go look at the sourcecode. Look at how simple it is - anyone who has created a thread with java knows what's happening. With only minor tweaks, this means your pre-existing code can take advantage of this with, basically, no effort. And it retains all the debuggability of traditional java thread (I.e: a stack trace that makes sense!) If you've spent any time at all deali…

Agreed it's simpler, but using NIO with one OS thread per core also has it's benefits.

The context switch (how ever small) will cause latency when this solution is at saturation.

I think they should write four tests: fiber, NIO and each with userspace networking (no kernel copying network memory) and compare them.

Why Oracle is stalling removing the kernel for Java networking is surprising to me, they allready have a VM.

Post reply on HN