Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Achieving 5M persistent connections with Project Loom virtual threads

github.com

Post reply on HN
Print view View original

131–140 of 150 posts

Re: Achieving 5M persistent connections with Project Loom virtual threads

#131

Post by vlovich123 »

threeseed wrote: post #104
vlovich123 wrote: post #75

Earlier quoted context omitted.

Shouldn’t you be able to send authorization and authentication requests in parallel in the async and virtual threads cases?

It is just an example so they could do anything. But in the real world it is common to need information from the authorization stage to use in the authentication stage. For example you may have a user login with an email address/password which you then pass to an LDAP server in order to get a userId. This userId is then used in a database to determine with objects/groups they have access to.

A lower latency design would be for the authorization service to be able to work with either. That way those requests could be done in parallel to reduce latency.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#132

Post by toast0 »

jeroenhd wrote: post #18
KingOfCoders wrote: post #12

Something to learn for everybody, the article is mainly about Linux tuning.

The Linux tuning part seems to have been inspired by these blog posts from 14 years ago: https://www.metabrew.com/article/a-million-user-comet-applic... It's almost a little disappointing that beefy modern servers only manage a x5 scale improvement, though that could be due to the differences in runtime behaviour between Erlang and the JVM.

I mean... is 5M very impressive? Not really. Does it show that Project Loom meets the goal of being able to do large client count thread per server workloads? I think so. Does the name remind me of a best selling point and click adventure game? Definitely yes.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#133

Post by kaba0 »

mike_hearn wrote: post #123
kaba0 wrote: post #121

Earlier quoted context omitted.

With Truffle you have to map your language’s semantics to java ones. I am unfortunately out of my depth on the details, but my guess would be that LLVM operates here with this in mind in a completely safe way (I guess pointers to the stack are not safe) so presumably it should work for these as well.

Not exactly, no. That's the whole point of Truffle and why it's such a big leap forward. You do not map your language's semantics to Java semantics. You can implement them on top of the JVM but bypassing Java bytecode. Your language doesn't even have to be garbage collected, and LLVM bitcode isn't (unless you use the enterprise version which adds support for automatically converting C/C++ to memory safe GCd code!). S…

Perhaps I wasn’t clear. I do know that Truffle works by writing an AST interpreter for another language, but to achieve the best performance you have to map/reuse existing java constructs. E.g. I have read that perhaps Ruby uses java exceptions in a not too idiomatic way, but this is what Graal can later optimize to very good code.

My way out of depth idea with Sulong is that it uses small heap-allocated regions for every manual memory usage (it even has a Managed mode in Enterprise).

Re: Achieving 5M persistent connections with Project Loom virtual threads

#134

Post by kaba0 »

richdougherty wrote: post #130
SemanticStrengh wrote: post #117

Earlier quoted context omitted.

Coroutines are much less coloured than async await programming though since functions returns resolved types directly instead of futures. But yes there is the notion of coroutine scope but I don't see how to supress it without making it less expressive. Very few people know it but Oracle is developping an alternative to Loom, in parallel. https://github.com/oracle/graal/pull/4114 BTW i expect Kotlin coroutines to lev…

> Coroutines are much less coloured I think another commenter pointed out that they are still coloured though. Still, they're very cool - and you can use them for more than just lightweight threading. > As for the tailrecursive keyword, it is not a constraint but a feature since it guarantee at the type level that this function cannot stack overflow I'd say tailrecursive is compiler feature (codegen the recursion int…

Doesn’t it possibly get inlined by current mechanisms as well?

Re: Achieving 5M persistent connections with Project Loom virtual threads

#135

Post by bullen »

pron wrote: post #33
bullen wrote: post #30

Earlier quoted context omitted.

Agreed it's simpler, but using NIO with one OS thread per core also has it's benefits. The context switch (how ever small) will cause latency when this solution is at saturation. I think they should write four tests: fiber, NIO and each with userspace networking (no kernel copying network memory) and compare them. Why Oracle is stalling removing the kernel for Java networking is surprising to me, they allready have a…

https://github.com/ebarlas/project-loom-comparison

Thx, I'm missing bandwidth and latency in those graphs.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#136

Post by bullen »

blibble wrote: post #56
bullen wrote: post #30

Earlier quoted context omitted.

Agreed it's simpler, but using NIO with one OS thread per core also has it's benefits. The context switch (how ever small) will cause latency when this solution is at saturation. I think they should write four tests: fiber, NIO and each with userspace networking (no kernel copying network memory) and compare them. Why Oracle is stalling removing the kernel for Java networking is surprising to me, they allready have a…

there's still a context switch with NIO, you're just doing it manually

The context switches in NIO is between fewer threads, you just need one per core.

Memory contention is also playing into this.

The benchmark they made is asking the question in a way that it leans into the answer they need, just like 99% of all human activity it's biased.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#137

Post by gpderetta »

yosefk wrote: post #118
gpderetta wrote: post #103

Earlier quoted context omitted.

There are .so interposition tricks that can be used for that. I think Pth used to do that for example.

Could you elaborate?

For example: https://www.gnu.org/software/pth/pth-manual.html#system_call...

See the hard system call wrapping. This is just one option.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#138

Post by jonpez2 »

pron wrote: post #93
amluto wrote: post #69

Earlier quoted context omitted.

Threads (whether lightweight or heavyweight) can’t fully replace reactive/proactive/async programming even ignoring performance and scalability. Sometimes network code simply needs to wait for more than one event as a matter of functionality. For example, a program might need to handle the availability of outgoing buffer space and also handle the availability of incoming data. And it might also need to handle complet…

> Sure, using extra threads might do it, but it’s awkward. It's simpler and nicer, actually — and definitely offers better tooling and observability — especially with structured concurrency: https://download.java.net/java/early_access/loom/docs/api/jd...

Let me preface by saying I am a Johnny-come-lately loom fanboy. Amazing work and huge impact. Re structured concurrency: I wonder if there’s any way to combine with generic exceptions such that we can not force a wrapping exception class. So maybe have an executor class that’s generic on the thrown exception type, and then have the join or get apis explicitly throw that type? This thought process is inspired by the goto-considered-harmful trail of logic: I think it would get us even closer to concurrency encapsulated in function blocks.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#139

Post by mike_hearn »

kaba0 wrote: post #133
mike_hearn wrote: post #123

Earlier quoted context omitted.

Not exactly, no. That's the whole point of Truffle and why it's such a big leap forward. You do not map your language's semantics to Java semantics. You can implement them on top of the JVM but bypassing Java bytecode. Your language doesn't even have to be garbage collected, and LLVM bitcode isn't (unless you use the enterprise version which adds support for automatically converting C/C++ to memory safe GCd code!). S…

Perhaps I wasn’t clear. I do know that Truffle works by writing an AST interpreter for another language, but to achieve the best performance you have to map/reuse existing java constructs. E.g. I have read that perhaps Ruby uses java exceptions in a not too idiomatic way, but this is what Graal can later optimize to very good code. My way out of depth idea with Sulong is that it uses small heap-allocated regions for…

You use Java constructs to implement the interpreter, but that doesn't mean the language itself has to be mapped to Java constructs, any more than writing an interpreter in C means your language has to be mapped to C semantics.

Sulong uses a standard C-style heap in the open source version. In EE they (can) trap malloc/free and re-point it towards the GCd heap. They also do bounds checking on pointer de-references. It's actually amazingly cool but unfortunately, EE is expensive enough in dollar terms that it gets ignored. I don't know of anything that uses it for real.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#140

Post by pron »

jonpez2 wrote: post #138
pron wrote: post #93

Earlier quoted context omitted.

> Sure, using extra threads might do it, but it’s awkward. It's simpler and nicer, actually — and definitely offers better tooling and observability — especially with structured concurrency: https://download.java.net/java/early_access/loom/docs/api/jd...

Let me preface by saying I am a Johnny-come-lately loom fanboy. Amazing work and huge impact. Re structured concurrency: I wonder if there’s any way to combine with generic exceptions such that we can not force a wrapping exception class. So maybe have an executor class that’s generic on the thrown exception type, and then have the join or get apis explicitly throw that type? This thought process is inspired by the g…

Convenient polymorphism over exceptions is something I would very much like to see in Java, but it's a separate topic. Given that structured concurrency is normally used with things that can fail, and whose failures must be handled, I hope (and think) you'll find that the use of checked exceptions is not onerous at all. If we're mistaken, we can consider solutions during the incubation period.
Post reply on HN