I thought you can make only 65k connections per port because of TCP limitations.
This comment appears routinely on HN. I'd love to find out how something so basic about TCP came to be so widely misunderstood.
Handling 1M websocket connections in Go
11–20 of 68 posts
Re: Handling 1M websocket connections in Go
#12I thought you can make only 65k connections per port because of TCP limitations.
The limit is 65k connections per client IP address, limited by the number of available ports on the client. You can theoretically accept connections from all possible client IP addresses simultaneously. This means that in order to test 1M simultaneous connections, you would need to use at least 16 client IP addresses. Probably more.
And most linux distros limits the ephemeral range even further to about 30K ports
If you take a deeper look, the solution uses Docker to connect clients from different network namespaces to mitigate this
Re: Handling 1M websocket connections in Go
#13Can someone make a comparison between this and the Elixir Phoenix 2m websocket conmections example. I want to sleep.
I remember seeing a talk on the 2m websocket Elixir example and one of the keys to it was that the sockets were actually being used and processing messages intermittently during the test. Important thing to keep in mind vs simply opening. The other thing I'd be interested to see Elixir demonstrate would be doing a hot deployment to avoid triggering all 2m connections to try reconnect at the same time. The other impor…
If you look at the server implementation, the client messages are also read and parsed, but are not printed to stdout for demonstration purposes (otherwise it hard to keep track)
You can uncomment that printf instruction and see that it still performs nearly the same way
Re: Handling 1M websocket connections in Go
#14I thought you can make only 65k connections per port because of TCP limitations.
An open connection is identified by all 4 values of the unique tuple source IP, source port, destination IP, destination port.
Re: Handling 1M websocket connections in Go
#15These kinds of benchmarks are not very meaningful. I think that pretty much any modern framework/language can handle at least 1 million idle WebSockets. It's much more interesting to measure performance when you start sending messages through them at regular intervals.
Re: Handling 1M websocket connections in Go
#16These kinds of benchmarks are not very meaningful. I think that pretty much any modern framework/language can handle at least 1 million idle WebSockets. It's much more interesting to measure performance when you start sending messages through them at regular intervals.
They are not idling. Look at the code.
Re: Handling 1M websocket connections in Go
#17Is there a video of the talk?
Re: Handling 1M websocket connections in Go
#18Very cool. Noticed they had to go down to epoll: https://github.com/eranyanay/1m-go-websockets/blob/master/3_... that seems a bit too low level. Why not spawn 1m goroutines is that not feasible?
This also reminds me of Erlang VM handling 2M for Whatsapp on a single server in 2012:
Re: Handling 1M websocket connections in Go
#19I'd like to see this kind of story combined with the "how to build a business" threads to discuss what kinds of business models and sizes require a million websocket connections to one process.
1M simultaneous users of a React (or other) app, is that a decently simple case? What are some sites that have this level of activity? I found a 5yr old article that says Spotify had 20MM simultaneous users then, but spread over 12,000 machines, so suffice it to say I'm having trouble finding a use-case here besides (good!) research.
Re: Handling 1M websocket connections in Go
#20I thought you can make only 65k connections per port because of TCP limitations.
The limit is 65k connections per client IP address, limited by the number of available ports on the client. You can theoretically accept connections from all possible client IP addresses simultaneously. This means that in order to test 1M simultaneous connections, you would need to use at least 16 client IP addresses. Probably more.
As a fun aside, you can do this on localhost by defining additional addresses tied to the loopback interface:
127.0.0.2
127.0.0.3
etc.