Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: SocketCluster.io – flexible open-source real-time server with pub/sub

socketcluster.io

Post reply on HN
Print view View original

31–40 of 41 posts

Re: Show HN: SocketCluster.io – flexible open-source real-time server with pub/sub

#31

Post by vhiremath4 »

geekuillaume wrote: post #25

I tried SocketCluster for a highly scalable chat platform and it couldn't handle very big conversations. SocketCluster is good but the bottleneck here was NodeJS. I finnaly used Nginx PushStream to do the sub and the pub was handled by NodeJS (with the authentication). It has been used to host chat room with more than 70,000 concurrent clients, scaled on 7 m3.large EC2 instances and one relatively small Redis server.…

Would you happen to have any metrics of how far you could push SC and how far you're getting with NGINX PushStream for sub?

Re: Show HN: SocketCluster.io – flexible open-source real-time server with pub/sub

#32

Post by dang »

nkrisc wrote: post #13
0xADADA wrote: post #2

How is this different than this? https://news.ycombinator.com/item?id=11071916

I was wondering the same. What are the odds of two very similar projects with nearly identical headlines appearing on HN at nearly the same time?

Three: https://news.ycombinator.com/item?id=11073966. But detaro no doubt has it right.

Re: Show HN: SocketCluster.io – flexible open-source real-time server with pub/sub

#33

Post by zokier »

geekuillaume wrote: post #25

I tried SocketCluster for a highly scalable chat platform and it couldn't handle very big conversations. SocketCluster is good but the bottleneck here was NodeJS. I finnaly used Nginx PushStream to do the sub and the pub was handled by NodeJS (with the authentication). It has been used to host chat room with more than 70,000 concurrent clients, scaled on 7 m3.large EC2 instances and one relatively small Redis server.…

> It has been used to host chat room with more than 70,000 concurrent clients, scaled on 7 m3.large EC2 instances and one relatively small Redis server.

Knowing nothing about anything, 70k clients on 7 servers does not sound immediately very impressive. That's roughly 10k clients per server, and this is 15 years after "C10k problem" was coined. Could someone comment if I'm missing something major here?

Re: Show HN: SocketCluster.io – flexible open-source real-time server with pub/sub

#34

Post by sudovoodoo »

z3t4 wrote: post #12
sudovoodoo wrote: post #5

Earlier quoted context omitted.

Should say that we horizontally scale this thing pretty heavily using the sc-redis module. Elasticache + ELB + 4 EC2 Instances = support for 5000+ person conferences :D

This sounds interesting. Do you have a web-site? Open source?

www.talkfusion.com -- in specific, the live meetings & video chat portion use socket cluster as its backer. We generally use c4.8xlarge instances to back the socket infrastructure.

Re: Show HN: SocketCluster.io – flexible open-source real-time server with pub/sub

#35

Post by sudovoodoo »

geekuillaume wrote: post #25

I tried SocketCluster for a highly scalable chat platform and it couldn't handle very big conversations. SocketCluster is good but the bottleneck here was NodeJS. I finnaly used Nginx PushStream to do the sub and the pub was handled by NodeJS (with the authentication). It has been used to host chat room with more than 70,000 concurrent clients, scaled on 7 m3.large EC2 instances and one relatively small Redis server.…

So -- m3.large ec2 instances have a few problems with them. 1, your only running a 2 core machine (1 worker, and assuming 1 broker to distribute through redis). That is not nearly enough to handle really a real-time chat server where everyone is spamming it all the time. Another issue is the network performance of an m3.large instance -- its moderate (which is another word for not very good).

With 2 c4.8xlarge instances behind ELB, we consistently see between 20 to 50K active connections on live meetings, and the servers sit at I see a lot of people having problems with this, but the issue is usually the resources they give to it, or the actual handling of socket events. If the socket server is just a relay there should be no reason a single m4.4xlarge instance cant handle 600K+ connections.

www.jayway.com/2015/04/13/600k-concurrent-websocket-connections-on-aws-using-node-js/

Re: Show HN: SocketCluster.io – flexible open-source real-time server with pub/sub

#36

Post by jondubois »

jadbox wrote: post #17
cryptica wrote: post #10

Earlier quoted context omitted.

I'm the main author of SC. Your system does sound similar to SC - It seems to be a pretty standard realtime architecture - Those who started building their realtime systems with Socket.io (as you did) often end up with something similar to what you describe except it takes a lot of work to get there... A lot of people who use SC decided to make the switch because they started implementing their own pub/sub stack (as…

Does SC require the need for a DB?

No but there is an optional CRUD module for RethinkDB https://github.com/socketcluster/sc-crud-rethink

Re: Show HN: SocketCluster.io – flexible open-source real-time server with pub/sub

#37

Post by billmalarky »

jontammeh wrote: post #9

Interesting these are coming around now. Some services (i.e. Pubnub) have been doing this (for money) for a long time and what I noticed were at the end of the day, the use cases were actually rather limited. Apart from stock quotes, bitcoin quotes and multiplayer games - polling just isn't that bad. And it's actually a very good place to start when developing as it's simple.

Polling works for current use cases, but products of the future aren't built around current use cases ;-)

Re: Show HN: SocketCluster.io – flexible open-source real-time server with pub/sub

#38

Post by chrismccord »

dfischer wrote: post #30

Does phoenix's / elixir channels compare with this well?

Our realtime features seem to overlap based on a quick look on their website, except we get distribution for free from the Erlang VM so you can deploy Phoenix on a cluster and you don't an intermediate redis instance or similar for pubsub/IPC across nodes.

Re: Show HN: SocketCluster.io – flexible open-source real-time server with pub/sub

#39

Post by cryptica »

If anyone opened the console and noticed the socket timing out earlier - That was because I upgraded to a bigger 4-CPU core Amazon instance and in the process, I accidentally installed a newer version of the SC server (which caused a protocol mismatch with the client version). The load per CPU was around 3% on each core at its peak of 250 concurrent users.

There were occasional spikes to 6% CPU use per core when people started spamming :p

Re: Show HN: SocketCluster.io – flexible open-source real-time server with pub/sub

#40

Post by arrty88 »

sudovoodoo wrote: post #35
geekuillaume wrote: post #25

I tried SocketCluster for a highly scalable chat platform and it couldn't handle very big conversations. SocketCluster is good but the bottleneck here was NodeJS. I finnaly used Nginx PushStream to do the sub and the pub was handled by NodeJS (with the authentication). It has been used to host chat room with more than 70,000 concurrent clients, scaled on 7 m3.large EC2 instances and one relatively small Redis server.…

So -- m3.large ec2 instances have a few problems with them. 1, your only running a 2 core machine (1 worker, and assuming 1 broker to distribute through redis). That is not nearly enough to handle really a real-time chat server where everyone is spamming it all the time. Another issue is the network performance of an m3.large instance -- its moderate (which is another word for not very good). With 2 c4.8xlarge instan…

How does nodejs stand up against Java netty NIO for the web socket use case?
Post reply on HN