DIY Meteor-like Realtime Functionality Using Socket.io and RethinkDB Changefeeds • Hacker News Board

Re: DIY Meteor-like Realtime Functionality Using Socket.io and RethinkDB Changefeeds

#11

Post by jamescun » Mon, Mar 14, 2016, 10:59 AM UTC

phaedryx wrote: post #7

One thing that I've never understood is why websockets instead of server sent events?

Possibly Internet Explorer support. Even Edge doesn't support them yet[1].

With deployments of HTTP/2 increasing, I think we'll see greater adoption of server sent events. At the time of writing, HTTP/2 and WebSockets are two incompatible standards. For non-latency sensitive applications (i.e. not games) it is very easy to implement a WebSocket like interface over HTTP/2 using SSE (server -> client) and POST requests (client -> server) with very little overhead (thanks to HTTP/2's multiplexing).

[1] http://caniuse.com/eventsource

Re: DIY Meteor-like Realtime Functionality Using Socket.io and RethinkDB Changefeeds

#12

Post by djKianoosh » Mon, Mar 14, 2016, 12:33 PM UTC

phaedryx wrote: post #7

One thing that I've never understood is why websockets instead of server sent events?

I'd love to use a sane, simple SSE server impl in Java. Does one exist? I see there's very little about plain java event source servlet implementations. Looks like there's some promising things in Jersey and Atmosphere at least, but things tend to get complicated quick in the java world...

Re: DIY Meteor-like Realtime Functionality Using Socket.io and RethinkDB Changefeeds

#13

Post by bryanlarsen » Mon, Mar 14, 2016, 4:05 PM UTC

KaoruAoiShiho wrote: post #6

meatier is this but less DIY.

Perhaps downvoters thought you were making a joke? https://github.com/mattkrick/meatier is very appropriate to this discussion.

Re: DIY Meteor-like Realtime Functionality Using Socket.io and RethinkDB Changefeeds

#14

Post by jondubois » Tue, Mar 15, 2016, 6:41 AM UTC

kjstevo wrote: post #10

jondubois wrote: post #9

RethinkDB changefeeds are great for dealing with data streams which are common to all users, but changefeeds can get tricky when you need to deal with hundreds of thousands of data streams which are more or less unique to each user (or unique to many small groups of users) - This is particularly true when you need to scale your app beyond a single process/instance/machine. The approach described in this article may n…

What approach would you take to solve the problem you describe of individual fruit baskets that can scale?

The simplest way to implement this is using a client-side pub/sub engine and have a channel subscription for each resource on the frontend.

So for example, if you have a Fruit basket with id 123 and you want to get realtime updates when that basket changes (E.g. new fruits are added or whatever), you could subscribe to a channel 'fruitbasket/123'.

You would have a separate channel for every basket but because the backend treats all channels in a generic way, the complexity on the backend remains minimal (and is handled by your pub/sub engine anyway).

You need to make sure that your pub/sub engine is good at handling lots of distinct channels (and is designed to be user-facing - A lot of pubsub engines like Redis are only designed for backend use). There are user-facing pub/sub SaaS services which you can hook into your app but they are a bit expensive.

If you want a self-hosted user-facing pub/sub solution, there are only two popular options:

- SocketCluster https://github.com/socketcluster/socketcluster

- Faye https://github.com/faye/faye

Both Faye and SocketCluster are good at handling lots of unique channels - Based on benchmarks I ran, both can handle the same load of about 10k new subscriptions per second (per CPU core). SC makes scaling across multiple CPU cores trivial so if you have an 8-core machine, you can handle 80K new subscriptions per second with SC out of the box. You can also scale with Faye but it will require a lot more work to get working.

Disclosure: I'm the main author of SocketCluster.