I followed it the whole way through until I got to the diagram of the way Moot handles channels compared to static channels. What does the diagram illustrate? It seems to be that the Moot version doesn't have central input. But it would all need to be filtered through something central, otherwise it is completely possible to miss a location the payload was intended to be delivered to.
That central location is Redis pub/sub. Multiple publishers publish to Redis and many JSON-RPC instances listen and then process the notifications to distribute. The diagram is more meant to illustrate that the notifications don't just blanket deliver to all subscribers rather than be an exactly technically accurate.
The realtime magic behind Moot and how we deliver all your instant notifications
11–20 of 47 posts
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#12Earlier quoted context omitted.
Yes we do see this as becoming a bottleneck. We are already working on a solution for that. I mention in the "What's Next" section about a bit more advanced routing using ZeroMQ instead of Redis. The problem is a difficult one to say the least. We're nowhere near the bottleneck now, but we do want to resolve this long before it is close to becoming an issue.
Could you please elaborate on how using ZeroMQ would solve your problem? I have a surprisingly similar setup and I'm also worried that this is going to be a bottleneck.
The RPC servers register the tags and channels we distribute based on that data. Any ZeroMQ server can process and deliver any event to the end RPC server.
We'll probably do a detailed writeup when this is done. We plan on using this for all the infrastructure wiring when it's done.
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#13Are you going to open-source your json-rpc library? If so, is there a specific timeframe you have in mind? I'd really love to use this technology in my own projects.
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#14> tens of millions of realtime events daily Does it means hundreds per second ?
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#15Are you going to open-source your json-rpc library? If so, is there a specific timeframe you have in mind? I'd really love to use this technology in my own projects.
Yes we intent to open source all of this. No timeframe in mind. We need to get through our beta roadmap before packaging up the various pieces we plan on open sourcing.
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#16404 Not Found
Code: NoSuchWebsiteConfiguration
Message: The specified bucket does not have a website configuration
BucketName: www.moot.it
RequestId: xxxx
HostId: xxxxx
I didn't know what "Moot" was so I tried www.moot.it
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#17Earlier quoted context omitted.
That central location is Redis pub/sub. Multiple publishers publish to Redis and many JSON-RPC instances listen and then process the notifications to distribute. The diagram is more meant to illustrate that the notifications don't just blanket deliver to all subscribers rather than be an exactly technically accurate.
Realistically, it sounds like the 2nd diagram should have the middle section of the first diagram. As in, a single message/circle, gets "split" (or more accurately dup'd) and somewhat simultaneously delivered to the start of every 'channel'. The way it is now makes the appearance that a message isn't sent down a channel where the logic gate will not eventually deliver it. But that doesn't seem to be the case.
Btw, did you notice it has blue heads? I thought those were cool.
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#18Ehm.. tens of millions of realtime events but no "www." alias?: 404 Not Found Code: NoSuchWebsiteConfiguration Message: The specified bucket does not have a website configuration BucketName: www.moot.it RequestId: xxxx HostId: xxxxx I didn't know what "Moot" was so I tried www.moot.it
Will be updating the alias to point to the CDN. Thanks for catching that.
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#19Re: The realtime magic behind Moot and how we deliver all your instant notifications
#20Earlier quoted context omitted.
Could you please elaborate on how using ZeroMQ would solve your problem? I have a surprisingly similar setup and I'm also worried that this is going to be a bottleneck.
ZeroMQ doesn't solve the problem. ZeroMQ is flexible enough that we can build a solution. We're working on distributed routing and registration using ZeroMQ and Redis as it's data store. The RPC servers register the tags and channels we distribute based on that data. Any ZeroMQ server can process and deliver any event to the end RPC server. We'll probably do a detailed writeup when this is done. We plan on using this…