Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Achieving 5M persistent connections with Project Loom virtual threads

github.com

Post reply on HN
Print view View original

111–120 of 150 posts

Re: Achieving 5M persistent connections with Project Loom virtual threads

#111

Post by pjmlp »

gavinray wrote: post #107
pjmlp wrote: post #94

Earlier quoted context omitted.

Or inserting the occasional Task.Run() calls, as means to avoiding changing the whole call stack up to Main().

This hasn't been that much of a problem, IME If you decide somewhere deep in your program you want to use async operations, most languages allow you to keep the invoking function/closure synchronous and return some kind of Promise/Future-like value

Which is exactly the workaround with Task.Run(), being able to integrate a library written with async/await in codebases older than the feature, where no one is paying for a full rewrite.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#113

Post by SemanticStrengh »

cheradenine_uk wrote: post #21

I think a lot of people are missing the point. Go look at the sourcecode. Look at how simple it is - anyone who has created a thread with java knows what's happening. With only minor tweaks, this means your pre-existing code can take advantage of this with, basically, no effort. And it retains all the debuggability of traditional java thread (I.e: a stack trace that makes sense!) If you've spent any time at all deali…

Except Kotlin coroutines already works, can be very easily integrated in existing java codebases and are much superior than loom (structured concurrency, flow, etc)

Re: Achieving 5M persistent connections with Project Loom virtual threads

#114

Post by natdempk »

mike_hearn wrote: post #54
10000truths wrote: post #39

A bit of a digression, but I’d love to see how much further one could go with a memory-optimized userland TCP stack, and storing the send and receive buffers on disk. A TCP connection state machine consists of a few variables to keep track of sequence numbers and congestion control parameters (no more than 100-200 bytes total), plus the space for send/receive buffers. A 4 TB SSD would fit ~125 million 16-KB buffer pa…

Presumably at 100M simultaneous connections the machine CPU would be saturated with setting up and closing them, without getting much actual work done. TCP connections seem too fragile to make it worth trying to keep them open for really long periods. It's interesting to think about though, I agree. What are the next scaling bottlenecks now (for JVM compatible languages) threading is nearly solved? There are some obv…

It depends on what you do, but I think GC/memory pressure can become an issue rather quickly with the default programming models Java leads you towards. I end up seeing this a lot in somewhat high throughput services/workers I own where fetching a lot of data to handle requests and discarding it afterwards leads to a lot of GC time. Curious if anyone has any sage advice on this front.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#115

Post by richdougherty »

SemanticStrengh wrote: post #113
cheradenine_uk wrote: post #21

I think a lot of people are missing the point. Go look at the sourcecode. Look at how simple it is - anyone who has created a thread with java knows what's happening. With only minor tweaks, this means your pre-existing code can take advantage of this with, basically, no effort. And it retains all the debuggability of traditional java thread (I.e: a stack trace that makes sense!) If you've spent any time at all deali…

Except Kotlin coroutines already works, can be very easily integrated in existing java codebases and are much superior than loom (structured concurrency, flow, etc)

Kotlin coroutines are amazing. They're built on very clever tech that converts fairly normal source code into a state machine when compiled. This has huge benefits and allows the programmer to break their code up without the hassle of explicitly programming callbacks, etc.

https://kotlinlang.org/spec/asynchronous-programming-with-co...

However... an unavoidable fact is that converted code works differently to other code. The programmer needs to know the difference. Normal and converted code compose together differently. The Kotlin compiler and type system helps keep track, but it can't paper over everything.

Having lightweight thread and continuations support directly in the VM makes things very much simpler for programmers (and compiler writers!) since the VM can handle the details of suspending/resuming and code composes together effortlessly, even without compiler support, so it works across languages and codebases.

I don't want to be critical about Kotlin. It's amazing what it achieves and I'm a big fan of this stuff. Here are some notes I wrote on something similar, Scala's experiments with compile-time delimited continuations: https://rd.nz/2009/02/delimited-continuations-in-scala_24.ht...

I think this is a general principle about compiler features vs runtime features. Having things in the runtime makes life a lot easier for everyone, at the cost of runtime complexity, of course.

Another one I'd like to see is native support for tail calls in Java. Kotlin, Scala, etc have to do compile-time tricks to get basic tail call support, but it doesn't work across functions well.

Scala and Kotlin both ask the programmer to add annotations where tail calls are needed, since the code gen so often fails.

https://kotlinlang.org/docs/functions.html#tail-recursive-fu...

https://www.scala-lang.org/api/3.x/scala/annotation/tailrec....

https://rd.nz/2009/04/tail-calls-tailrec-and-trampolines.htm...

As a side note, I can see that tail calls are planned for Project Loom too, but I haven't heard if that's implemented yet. Does anyone know the status?

"Project Loom is to intended to explore, incubate and deliver Java VM features and APIs built on top of them for the purpose of supporting easy-to-use, high-throughput lightweight concurrency and new programming models on the Java platform. This is accomplished by the addition of the following constructs:

* Virtual threads

* Delimited continuations

* Tail-call elimination"

https://wiki.openjdk.java.net/display/loom/Main

Re: Achieving 5M persistent connections with Project Loom virtual threads

#116

Post by richdougherty »

deepsun wrote: post #5

How does that compare to Kotlin suspend functions?

I made a comment about this above: https://news.ycombinator.com/item?id=31218826

I won't repeat it all, but the main point is that having runtime support is much better than relying on compiler support, even if compiler support is pretty fantastic.

Note that the two aren't mutally exclusive, you should still be able to use coroutines after Project Loom ships, and it still might make sense in many places.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#117

Post by SemanticStrengh »

richdougherty wrote: post #115
SemanticStrengh wrote: post #113

Earlier quoted context omitted.

Except Kotlin coroutines already works, can be very easily integrated in existing java codebases and are much superior than loom (structured concurrency, flow, etc)

Kotlin coroutines are amazing. They're built on very clever tech that converts fairly normal source code into a state machine when compiled. This has huge benefits and allows the programmer to break their code up without the hassle of explicitly programming callbacks, etc. https://kotlinlang.org/spec/asynchronous-programming-with-co... However... an unavoidable fact is that converted code works differently to other c…

Coroutines are much less coloured than async await programming though since functions returns resolved types directly instead of futures. But yes there is the notion of coroutine scope but I don't see how to supress it without making it less expressive.

Very few people know it but Oracle is developping an alternative to Loom, in parallel. https://github.com/oracle/graal/pull/4114

BTW i expect Kotlin coroutines to leverage loom eventually.

As for the tailrecursive keyword, it is not a constraint but a feature since it guarantee at the type level that this function cannot stack overflow. Few people know there is an alternative to tailrecursive, that can make any function stackoverflow safe by leveraging the heap via continuations https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-deep-re...

As for Java, there is universal support for tail recursion at the bytecode level https://github.com/Sipkab/jvm-tail-recursion

Re: Achieving 5M persistent connections with Project Loom virtual threads

#118

Post by yosefk »

gpderetta wrote: post #103
yosefk wrote: post #82

Earlier quoted context omitted.

Sure, I wrote some myself. Q is what libraries you can use on top of the userspace thread package that are aware of the userspace threads rather than just using OS APIs and thus eg blocking the current OS thread.

There are .so interposition tricks that can be used for that. I think Pth used to do that for example.

Could you elaborate?

Re: Achieving 5M persistent connections with Project Loom virtual threads

#119

Post by gavinray »

SemanticStrengh wrote: post #117
richdougherty wrote: post #115

Earlier quoted context omitted.

Kotlin coroutines are amazing. They're built on very clever tech that converts fairly normal source code into a state machine when compiled. This has huge benefits and allows the programmer to break their code up without the hassle of explicitly programming callbacks, etc. https://kotlinlang.org/spec/asynchronous-programming-with-co... However... an unavoidable fact is that converted code works differently to other c…

Coroutines are much less coloured than async await programming though since functions returns resolved types directly instead of futures. But yes there is the notion of coroutine scope but I don't see how to supress it without making it less expressive. Very few people know it but Oracle is developping an alternative to Loom, in parallel. https://github.com/oracle/graal/pull/4114 BTW i expect Kotlin coroutines to lev…

Thanks for posting that link to Java tail recursion library, super handy + didn't know about it. You need tail recursion for writing expression evaluators/visitors frequently.

I've been using an IntelliJ extension that can do magic by rewriting recursive functions to stateful stack-based code for performance, but it spits out very ugly code:

https://github.com/andreisilviudragnea/remove-recursion-insp...

  > "This inspection detects methods containing recursive calls (not just tail recursive calls) and removes the recursion from the method body, while preserving the original semantics of the code. However, the resulting code becomes rather obfuscated if the control flow in the recursive method is complex."
It was this guy's whole Bachelor thesis I guess:

https://github.com/andreisilviudragnea/remove-recursion-insp...

Re: Achieving 5M persistent connections with Project Loom virtual threads

#120

Post by ohgodplsno »

SemanticStrengh wrote: post #117
richdougherty wrote: post #115

Earlier quoted context omitted.

Kotlin coroutines are amazing. They're built on very clever tech that converts fairly normal source code into a state machine when compiled. This has huge benefits and allows the programmer to break their code up without the hassle of explicitly programming callbacks, etc. https://kotlinlang.org/spec/asynchronous-programming-with-co... However... an unavoidable fact is that converted code works differently to other c…

Coroutines are much less coloured than async await programming though since functions returns resolved types directly instead of futures. But yes there is the notion of coroutine scope but I don't see how to supress it without making it less expressive. Very few people know it but Oracle is developping an alternative to Loom, in parallel. https://github.com/oracle/graal/pull/4114 BTW i expect Kotlin coroutines to lev…

> Coroutines are much less coloured than async await programming though since functions returns resolved types directly instead of futures

Only because the compiler does its magic behind the scenes and transforms it into bytecode that takes a lambda with a continuation. Try calling a suspend function from java or starting a job and surprise, it's continuations all the way down

Post reply on HN