Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Handling 1M websocket connections in Go

github.com

Post reply on HN
Print view View original

31–40 of 68 posts

Re: Handling 1M websocket connections in Go

#31

Post by kjksf »

windlep wrote: post #28
rdtsc wrote: post #18

Very cool. Noticed they had to go down to epoll: https://github.com/eranyanay/1m-go-websockets/blob/master/3_... that seems a bit too low level. Why not spawn 1m goroutines is that not feasible? This also reminds me of Erlang VM handling 2M for Whatsapp on a single server in 2012: https://blog.whatsapp.com/196/1-million-is-so-2011

> Noticed they had to down to epoll: https://github.com/eranyanay/1m-go-websockets/blob/master/3_... . that seems a bit too low level. Agreed, I'd be quite curious how many libraries in the Go ecosystem they can't use as a result. Either because the library spawns a goroutine, uses one under the hood, etc. Having to drop to this level makes me wonder if it'd be better to just use a language better suited to this type…

Very few Go libraries start goroutines on their own. Concurrency is mostly an application level concern.

HTTP library is unusual in that sense.

At the same time, the beauty of how http library is designed and the solution he describes is that there are hooks that allow being even more efficient with very little code.

Or to put it differently: Go gives you excellent (compared to everything else out there except C++/Rust) networking performance out of the box and you can go even faster with a minimal amount of effort.

What you call "dropping to this level" is 80 lines of code (https://github.com/eranyanay/1m-go-websockets/blob/master/3_...) and now you don't even have to write them yourself.

Re: Handling 1M websocket connections in Go

#32

Post by kjksf »

jondubois wrote: post #15

These kinds of benchmarks are not very meaningful. I think that pretty much any modern framework/language can handle at least 1 million idle WebSockets. It's much more interesting to measure performance when you start sending messages through them at regular intervals.

When people rewrite system from X where X in (Python, Ruby, Node, Clojure) to Go they usually see at least 10x improvement.

This is just one recent example: https://www.infoq.com/articles/api-gateway-clojure-golang

The money quote:

"The end result enabled us to reduce 25 instances (c4 xlarge) running Clojure code - able to process 60 concurrent requests, to two instances (c3.2xlarge) running Go code able to support ~5000 concurrent requests a minute"

If you google around you'll find more stories like that.

To do better than Go you would have to drop to C++ or Rust.

Re: Handling 1M websocket connections in Go

#33

Post by tomohawk »

rhizome wrote: post #19

I'd like to see this kind of story combined with the "how to build a business" threads to discuss what kinds of business models and sizes require a million websocket connections to one process. 1M simultaneous users of a React (or other) app, is that a decently simple case? What are some sites that have this level of activity? I found a 5yr old article that says Spotify had 20MM simultaneous users then, but spread ov…

Flip it around. If the stack you're using is memory or compute intensive for a given load, it will cost you more to provision that load. You may not need to service 1M connections, but you may need to service 10K connections with a single box instead of multiple boxes due to cost.

Re: Handling 1M websocket connections in Go

#34

Post by leonhandreke »

rhizome wrote: post #19

I'd like to see this kind of story combined with the "how to build a business" threads to discuss what kinds of business models and sizes require a million websocket connections to one process. 1M simultaneous users of a React (or other) app, is that a decently simple case? What are some sites that have this level of activity? I found a 5yr old article that says Spotify had 20MM simultaneous users then, but spread ov…

Apparently WhatsApp at some point handled 2M connections per server: http://highscalability.com/blog/2014/3/31/how-whatsapp-grew-...

Re: Handling 1M websocket connections in Go

#35

Post by Scarbutt »

kjksf wrote: post #32
jondubois wrote: post #15

These kinds of benchmarks are not very meaningful. I think that pretty much any modern framework/language can handle at least 1 million idle WebSockets. It's much more interesting to measure performance when you start sending messages through them at regular intervals.

When people rewrite system from X where X in (Python, Ruby, Node, Clojure) to Go they usually see at least 10x improvement. This is just one recent example: https://www.infoq.com/articles/api-gateway-clojure-golang The money quote: "The end result enabled us to reduce 25 instances (c4 xlarge) running Clojure code - able to process 60 concurrent requests, to two instances (c3.2xlarge) running Go code able to support ~…

"The end result enabled us to reduce 25 instances (c4 xlarge) running Clojure code - able to process 60 concurrent requests, to two instances (c3.2xlarge) running Go code able to support ~5000 concurrent requests a minute"

That doesn't make any sense though.

Re: Handling 1M websocket connections in Go

#36

Post by karim »

kjksf wrote: post #32
jondubois wrote: post #15

These kinds of benchmarks are not very meaningful. I think that pretty much any modern framework/language can handle at least 1 million idle WebSockets. It's much more interesting to measure performance when you start sending messages through them at regular intervals.

When people rewrite system from X where X in (Python, Ruby, Node, Clojure) to Go they usually see at least 10x improvement. This is just one recent example: https://www.infoq.com/articles/api-gateway-clojure-golang The money quote: "The end result enabled us to reduce 25 instances (c4 xlarge) running Clojure code - able to process 60 concurrent requests, to two instances (c3.2xlarge) running Go code able to support ~…

"When people rewrite system from X where X in (Python, Ruby, Node, Clojure) to Go they usually see at least 10x improvement."

Sure but how much of the performance gains come from Go and how much come from just having a better understanding of the problem the second time around?

Re: Handling 1M websocket connections in Go

#37

Post by cdoxsey »

e-r-a-n wrote: post #21
rdtsc wrote: post #18

Very cool. Noticed they had to go down to epoll: https://github.com/eranyanay/1m-go-websockets/blob/master/3_... that seems a bit too low level. Why not spawn 1m goroutines is that not feasible? This also reminds me of Erlang VM handling 2M for Whatsapp on a single server in 2012: https://blog.whatsapp.com/196/1-million-is-so-2011

The idea behind going all the way down to epoll is to reduce the memory footprint by around 30% otherwise, with every goroutine stack consuming 4-8KB, it's not feasible to go with 1goroutine-1websocket approach

That's what, 8GB of ram? A lot for a laptop... but not too much for a server.

Re: Handling 1M websocket connections in Go

#38

Post by windlep »

kjksf wrote: post #31
windlep wrote: post #28

Earlier quoted context omitted.

> Noticed they had to down to epoll: https://github.com/eranyanay/1m-go-websockets/blob/master/3_... . that seems a bit too low level. Agreed, I'd be quite curious how many libraries in the Go ecosystem they can't use as a result. Either because the library spawns a goroutine, uses one under the hood, etc. Having to drop to this level makes me wonder if it'd be better to just use a language better suited to this type…

Very few Go libraries start goroutines on their own. Concurrency is mostly an application level concern. HTTP library is unusual in that sense. At the same time, the beauty of how http library is designed and the solution he describes is that there are hooks that allow being even more efficient with very little code. Or to put it differently: Go gives you excellent (compared to everything else out there except C++/Ru…

To be more specific, lots of idiomatic Go patterns typically involve using channels. Working at this level means you can't do that per 'client' since that would imply at least one goroutine per client.

There's plenty of libraries that involve channels (and assume you have goroutines per connection/client), do those play well with the use of epoll in this manner? I would assume I can't use the stdlib time package to do a timeout for example, since that provides a channel to wait on, while in this setup I need objects that work with epoll. Obviously in that case I could use evio which abstracts over epoll/kqueue and provides a Tick...

So my comment was more that "dropping to this level" means dropping the use of the majority of standard Go concurrency idioms which revolve around goroutines and channels. It's not about the raw lines of code involved. Writing code in this style of async doesn't feel very Go-like and when you look at an example of some code using this style its exactly the thing Go was trying to avoid with goroutines (https://github.com/tidwall/evio/blob/master/examples/http-se...).

You end up with async event-based code, not the clean synchronous-appearing Go code that goroutines and channels provide.

Re: Handling 1M websocket connections in Go

#40

Post by spenrose »

An issue with Go channels and high-performance networking from 2016:

"There was one fundamental mistake made, however, which is that we shouldn't have used channels. ... First, they don't perform well enough. ... Second, they make it very hard to prevent message loss. ... Third, the buffered channels mean that Heka consumes much more RAM than would be otherwise needed"

https://mail.mozilla.org/pipermail/heka/2016-May/001059.html

Post reply on HN