Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Achieving 5M persistent connections with Project Loom virtual threads

github.com

Post reply on HN
Print view View original

11–20 of 150 posts

Re: Achieving 5M persistent connections with Project Loom virtual threads

#11

Post by ivanr »

notorandit wrote: post #4

With a maximum of 64k TCP connections per single server IP, you need 77 different IP on the server side. This is a fact.

I imagine that's the limit per client IP address [for a single server port], no? The Linux kernel can use multiple pieces of information to track connections: client IP address, client port, server IP address, server port.

Cloudflare has some interesting blog posts on this topic:

- https://blog.cloudflare.com/how-we-built-spectrum/

- https://blog.cloudflare.com/how-to-stop-running-out-of-ephem...

Re: Achieving 5M persistent connections with Project Loom virtual threads

#13

Post by wiradikusuma »

The experiment is about Java app, but the tweaks are at the O/S level. Does it mean any app (Java/not, Loom/not) can achieve target given correct tweak?

Also, why are these not default for the O/S? What are we compromising by setting those values?

Re: Achieving 5M persistent connections with Project Loom virtual threads

#14

Post by jiggawatts »

wiradikusuma wrote: post #13

The experiment is about Java app, but the tweaks are at the O/S level. Does it mean any app (Java/not, Loom/not) can achieve target given correct tweak? Also, why are these not default for the O/S? What are we compromising by setting those values?

There's always trade-offs. It would be very rare for any server to reach even 100K concurrent connections, let alone 5M. Optimising for that would be optimising for the 0.000001% case at the expense of the common case.

Some back of the envelope maths: https://www.wolframalpha.com/input?i=100+Gbps+%2F+5+million

If the server had a 100 Gbps Ethernet NIC, this would leave just 20 kbps for each TCP connection.

I could imagine some IoT scenarios where this might be a useful thing, but outside of that? I doubt there's anyone that wants 20 kbps throughput in this day and age...

It's a good stress test however to squeeze out inefficiencies, super-linear scaling issues, etc...

Re: Achieving 5M persistent connections with Project Loom virtual threads

#15

Post by NovemberWhiskey »

notorandit wrote: post #4

With a maximum of 64k TCP connections per single server IP, you need 77 different IP on the server side. This is a fact.

What?

Having run production services that had over 250,000 sockets connecting to a single server port, I'm calling "nope" on that.

Are you thinking of the ephemeral port limit? That's on the client side; not the server side. Each TCP socket pair is a four-tuple of [server IP, server port, client IP, client port]; the uniqueness comes from the client IP/port part in the server case.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#16

Post by jeroenhd »

notorandit wrote: post #4

With a maximum of 64k TCP connections per single server IP, you need 77 different IP on the server side. This is a fact.

You don't really need 77 IP addresses (the 64k limit for TCP is per client IP, per source port, per server IP) but even if you did, your average IPv6 server will have a few billion available. Every client can connect to a server IP of their own if you ignore the practical limits of the network acceleration and driver stack. If you're somehow dealing with this scale, I doubt you'll be stuck with pure legacy IP addressing.

The real problem with such a setup is that you're not left with a whole lot of bandwidth per connection, even if you ignore things like packet loss and retransmits mucking up the connections. Most VPS servers have a 1gbps connection, with 5 million clients that leaves 200 bytes per second of concurrent bandwidth for TCP signaling and data to flow through. You'll need a ridiculous network card for a single server to deal with such a load, in the terabits per second range.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#17

Post by torginus »

deepsun wrote: post #5

How does that compare to Kotlin suspend functions?

While I can't answer the question directly there is an article about C#-s async/await vs Go's goroutines, which compare the two approaches, and while some of the stuff is probably stack-specific, a lot of it is probably intrinsic to the approach:

- Green threads scale somewhat better, but both scale ridiculously well, meaning probably you won't run into scaling issues.

- async/await generators use way less memory than a dedicated green thread, this affects both memory consumption and startup time, since the process has to run around asking the OS for more memory

- green threads are faster to execute

Here's the link:

https://alexyakunin.medium.com/go-vs-c-part-1-goroutines-vs-...

Re: Achieving 5M persistent connections with Project Loom virtual threads

#18

Post by jeroenhd »

KingOfCoders wrote: post #12

Something to learn for everybody, the article is mainly about Linux tuning.

The Linux tuning part seems to have been inspired by these blog posts from 14 years ago: https://www.metabrew.com/article/a-million-user-comet-applic...

It's almost a little disappointing that beefy modern servers only manage a x5 scale improvement, though that could be due to the differences in runtime behaviour between Erlang and the JVM.

Re: Achieving 5M persistent connections with Project Loom virtual threads

#19

Post by Koffiepoeder »

jiggawatts wrote: post #14
wiradikusuma wrote: post #13

The experiment is about Java app, but the tweaks are at the O/S level. Does it mean any app (Java/not, Loom/not) can achieve target given correct tweak? Also, why are these not default for the O/S? What are we compromising by setting those values?

There's always trade-offs. It would be very rare for any server to reach even 100K concurrent connections, let alone 5M. Optimising for that would be optimising for the 0.000001% case at the expense of the common case. Some back of the envelope maths: https://www.wolframalpha.com/input?i=100+Gbps+%2F+5+million If the server had a 100 Gbps Ethernet NIC, this would leave just 20 kbps for each TCP connection. I could im…

Open, idle websockets can be a use case for a large amount of tcp connections with a small data footprint.
Post reply on HN