Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Achieving 5M persistent connections with Project Loom virtual threads

github.com

Post reply on HN
Print view View original

121–130 of 150 posts

Re: Achieving 5M persistent connections with Project Loom virtual threads

#121

Post by kaba0 »

mike_hearn wrote: post #58
nelsonic wrote: post #3

Reminds of https://phoenixframework.org/blog/the-road-to-2-million-webs... Would love to see this extended to more Languages/Frameworks.

In theory once Graal adds support for it, any Graal/Truffle-compatible language can benefit. IMHO it's only JVM+Graal that can bring this to other languages. Loom relies very heavily on some fairly unique aspects of the Java ecosystem (Go has these things too though). One is that lots of important bits of code are implemented in pure Java, like the IO and SSL stacks. Most languages rely heavily on FFI to C libraries.…

With Truffle you have to map your language’s semantics to java ones. I am unfortunately out of my depth on the details, but my guess would be that LLVM operates here with this in mind in a completely safe way (I guess pointers to the stack are not safe) so presumably it should work for these as well.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#122

Post by SemanticStrengh »

ohgodplsno wrote: post #120
SemanticStrengh wrote: post #117

Earlier quoted context omitted.

Coroutines are much less coloured than async await programming though since functions returns resolved types directly instead of futures. But yes there is the notion of coroutine scope but I don't see how to supress it without making it less expressive. Very few people know it but Oracle is developping an alternative to Loom, in parallel. https://github.com/oracle/graal/pull/4114 BTW i expect Kotlin coroutines to lev…

> Coroutines are much less coloured than async await programming though since functions returns resolved types directly instead of futures Only because the compiler does its magic behind the scenes and transforms it into bytecode that takes a lambda with a continuation. Try calling a suspend function from java or starting a job and surprise, it's continuations all the way down

yes interfacing with java is generally made via RxJava and reactor. Interfacing is easy but yes nobody wants to use rxjava and reactor in the first place.. I wonder wether loom will enable easier interop and make the magic work from java side POV

Re: Achieving 5M persistent connections with Project Loom virtual threads

#123

Post by mike_hearn »

kaba0 wrote: post #121
mike_hearn wrote: post #58

Earlier quoted context omitted.

In theory once Graal adds support for it, any Graal/Truffle-compatible language can benefit. IMHO it's only JVM+Graal that can bring this to other languages. Loom relies very heavily on some fairly unique aspects of the Java ecosystem (Go has these things too though). One is that lots of important bits of code are implemented in pure Java, like the IO and SSL stacks. Most languages rely heavily on FFI to C libraries.…

With Truffle you have to map your language’s semantics to java ones. I am unfortunately out of my depth on the details, but my guess would be that LLVM operates here with this in mind in a completely safe way (I guess pointers to the stack are not safe) so presumably it should work for these as well.

Not exactly, no. That's the whole point of Truffle and why it's such a big leap forward. You do not map your language's semantics to Java semantics. You can implement them on top of the JVM but bypassing Java bytecode. Your language doesn't even have to be garbage collected, and LLVM bitcode isn't (unless you use the enterprise version which adds support for automatically converting C/C++ to memory safe GCd code!).

So - C code running on the JVM via Sulong keeps C/C++ semantics. That probably means you can build pointers into the stack, and then I don't know what Loom would do. Right now they aren't integrated so I guess that's a research question.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#124

Post by Matthias247 »

toast0 wrote: post #108
TYMorningCoffee wrote: post #77

I was only able to get to 840,000 open connections with my experiment. My machine only has 8GB of memory. https://josephmate.github.io/2022-04-14-max-connections/ Is there anyway for the TCP connections share memory in kernel space? My experiment only uses two 8 byte buffers in userspace.

Does Linux actually allocate buffers for each socket or does it just link to sk_buff's (which I understand are similar to FreeBSD's mbuf's) and then limit how much storage can be linked? FreeBSD has a limit on the total ram used for mbufs as well, not sure about Linux. Otoh, FreeBSD's maximum FD limit is set as a factor of total memory pages (edit: looked it up, it's in sys/kern/subr_param.c, the limit is one FD per…

I think the socket buffers (sk_buff) are actually shared. They are all packet sized, and whatever socket needs to transmit some data or receives it gets the buffers attached. So my assumption is that the amount of required socket buffers scales more with the amount of data transmission than with the number of sockets.

But independent of socket buffers, the kernel obviously needs to allocate other state per socket, which tracks the state of the TCP connection.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#125

Post by Matthias247 »

toast0 wrote: post #102
mike_hearn wrote: post #54

Earlier quoted context omitted.

Presumably at 100M simultaneous connections the machine CPU would be saturated with setting up and closing them, without getting much actual work done. TCP connections seem too fragile to make it worth trying to keep them open for really long periods. It's interesting to think about though, I agree. What are the next scaling bottlenecks now (for JVM compatible languages) threading is nearly solved? There are some obv…

You're totally spot on that connection establishment is much more challenging than steady state; with TLS or just TCP. I don't think QUIC helps with that at all. Afaik, QUIC is all userland, so you'd skip kernel processing, but that doesn't really make establishment cheaper. And TCP+TLS establishes the connection before doing crypto, so that saves effort on spoofing (otoh, it increases the round trips, so pick your t…

QUIC will help with some things, and make others worse. With QUIC you don't need a file descriptor per connection anymore. A single file descriptor for one UDP socket will be sufficient to handle an arbitrary amount of connections (although you might want more to actually exploit concurrency). That fact will help limiting resources that the kernel uses. However the state that needs to be tracked per established connection is likely way larger than for TCP, due to being a more complex and featureful protocol. E.g. QUIC needs state for tracking sub-streams on a connection, while TCP does not. And of course there's all the mandatory crypto state. I am fairly familiar with QUIC implementations, and made a multitude of changes in various libraries (e.g. Quinn and s2n-quic). I wouldn't be surprised if the baseline memory usage of a QUIC connection in most libraries is > 10x of what the Linux TCP stack requires for tracking a connection

Re: Achieving 5M persistent connections with Project Loom virtual threads

#126

Post by tadfisher »

midislack wrote: post #110
shadowpho wrote: post #109

Earlier quoted context omitted.

Raspberry pi 4 performance changes wildly based on cooling. Bare die vs heatsink vs heatsink + fan will give you wildly different results.

Same is true with any computer these days. So let's go no heat sink, Pi 4 4GB anyway.

At STP, of course, as you need to standardize the convection rate of the SoC package.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#128

Post by niederman »

midislack wrote: post #106

I see a lot of these making the FP of HN. But it's very difficult to be impressed, or unimpressed because it's all about hardware. How much hardware is everybody throwing at all of this? 5M persistent connections on a Pi with mere GigE? Pretty frickin' amazing. 5M persistent connections on a Threadripper with 128 cores and a dozen trunked 4 port 10GE NICs? Yaaaaawwwnnn snooze. We need a standardized computer for benc…

> Everybody can find one

LMAO I wish.

https://rpilocator.com/?cat=PI4

Re: Achieving 5M persistent connections with Project Loom virtual threads

#129

Post by pron »

lostcolony wrote: post #99
pron wrote: post #89

Earlier quoted context omitted.

> and there's no still no date when it's supposed to be September 20 (in Preview) > I'm saying I can get the benefits of Project Loom -right now-, in production ready languages/libraries, outside of the JVM Only sort-of. The only languages offering something similar in terms of programming model are Erlang (/Elixir) and Go — both inspired virtual threads. But Erlang doesn't offer similar performance, and Go doesn't o…

I'm not saying there aren't tradeoffs, just that if I need the benefits of virtual threads...I have other options. I'm all for this landing on the JVM, mainly so that non-Java languages there can take advantage of it rather than the hoops they currently have to jump through to offer a saner concurrency model, but that until it does...don't care. And last I saw this feature is proposed to land in preview in JDK19; not…

Erlang is very cool, and Go has certainly achieved a notable measure of popularity — both served as an inspiration here — but a super-popular language like Java plays in a different world and a completely different scale than languages with a smaller reach. Virtual threads will bring the benefits of lightweight user-mode threads to an audience that is many times that of Erlang and Go combined.

As to legacy code, Java programs have been using the thread-per-request model for over 25 years (there's been a lot of talk of reactive, but actual adoption is relatively low), and Java's threads were designed to be abstracted from day one (in fact, early versions of Java implemented them in user mode). So the right fit has been there all along. Migrating applications to use virtual threads requires relatively few changes because of those reasons, and because we designed them with easy adoption in mind. This particular experiment is about simple, "legacy" Java 1.0 code enjoying terrific scalability.

BTW, Java's observability has come a long way in recent years (largely thanks to JFR — Java Flight Recorder), and even Erlang's is no match for it, although Java still lags behind Erlang's hot-swapping capabilities.

[1]: BTW, I always find talk about the "average Java programmer" a bit out of touch. The top 1% of Java programmers, the experts, outnumber all Rust (or Haskell, or Erlang) programmers several times over, and there are many more reliable Java programs than reliable Erlang programs. The average Java (or Python, or JavaScript, the two other dominant languages these days) programmer, is just the average programmer, period.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#130

Post by richdougherty »

SemanticStrengh wrote: post #117
richdougherty wrote: post #115

Earlier quoted context omitted.

Kotlin coroutines are amazing. They're built on very clever tech that converts fairly normal source code into a state machine when compiled. This has huge benefits and allows the programmer to break their code up without the hassle of explicitly programming callbacks, etc. https://kotlinlang.org/spec/asynchronous-programming-with-co... However... an unavoidable fact is that converted code works differently to other c…

Coroutines are much less coloured than async await programming though since functions returns resolved types directly instead of futures. But yes there is the notion of coroutine scope but I don't see how to supress it without making it less expressive. Very few people know it but Oracle is developping an alternative to Loom, in parallel. https://github.com/oracle/graal/pull/4114 BTW i expect Kotlin coroutines to lev…

> Coroutines are much less coloured

I think another commenter pointed out that they are still coloured though. Still, they're very cool - and you can use them for more than just lightweight threading.

> As for the tailrecursive keyword, it is not a constraint but a feature since it guarantee at the type level that this function cannot stack overflow

I'd say tailrecursive is compiler feature (codegen the recursion into a loop) to work around a runtime contraint (no tail call optimisation).

The lack of tail call optimisation on the JRE means recursion is a lot less safe than in functional language runtimes which guarantee stacks don't overflow when you make tail calls.

> As for Java, there is universal support for tail recursion at the bytecode level.

Just a note here for other readers that there are several terms in play here.

I was talking about "tail calls" - when a function calls a function as its last operation - and I mentioned some annotations to do "tail recursion", which is a special case - when a function calls _itself_ as its last operation.

SemanticStrength is talking about "tail recursion" only here. The JVM bytecode can support tail recursion (tail calls on the same method), since we can use the same bytecode that is used for while loops, etc.

However, we cannot do safe tail recursion between different functions (yet), in the same way that we cannot have a loop spanning more than one function. Tail call optimisation is something that will hopefully come in Project Loom.

Post reply on HN