The realtime magic behind Moot and how we deliver all your instant notifications
1–10 of 47 posts
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#2"This is then picked up by every JSON-RPC server and they will intersect those two tags, sending the event {hello: 'world'} to any connection that matches."
Do you see this eventually becoming your bottleneck (every rpc server has to handle every event)? If so, what's the plan for reducing the event fan-out?
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#3I followed it the whole way through until I got to the diagram of the way Moot handles channels compared to static channels. What does the diagram illustrate? It seems to be that the Moot version doesn't have central input. But it would all need to be filtered through something central, otherwise it is completely possible to miss a location the payload was intended to be delivered to.
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#4"This is then picked up by every JSON-RPC server and they will intersect those two tags, sending the event {hello: 'world'} to any connection that matches." Do you see this eventually becoming your bottleneck (every rpc server has to handle every event)? If so, what's the plan for reducing the event fan-out?
Yes we do see this as becoming a bottleneck. We are already working on a solution for that. I mention in the "What's Next" section about a bit more advanced routing using ZeroMQ instead of Redis.
The problem is a difficult one to say the least. We're nowhere near the bottleneck now, but we do want to resolve this long before it is close to becoming an issue.
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#5"This is then picked up by every JSON-RPC server and they will intersect those two tags, sending the event {hello: 'world'} to any connection that matches." Do you see this eventually becoming your bottleneck (every rpc server has to handle every event)? If so, what's the plan for reducing the event fan-out?
[deleted]
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#6I followed it the whole way through until I got to the diagram of the way Moot handles channels compared to static channels. What does the diagram illustrate? It seems to be that the Moot version doesn't have central input. But it would all need to be filtered through something central, otherwise it is completely possible to miss a location the payload was intended to be delivered to.
That central location is Redis pub/sub. Multiple publishers publish to Redis and many JSON-RPC instances listen and then process the notifications to distribute.
The diagram is more meant to illustrate that the notifications don't just blanket deliver to all subscribers rather than be an exactly technically accurate.
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#7"This is then picked up by every JSON-RPC server and they will intersect those two tags, sending the event {hello: 'world'} to any connection that matches." Do you see this eventually becoming your bottleneck (every rpc server has to handle every event)? If so, what's the plan for reducing the event fan-out?
Yes we do see this as becoming a bottleneck. We are already working on a solution for that. I mention in the "What's Next" section about a bit more advanced routing using ZeroMQ instead of Redis. The problem is a difficult one to say the least. We're nowhere near the bottleneck now, but we do want to resolve this long before it is close to becoming an issue.
Could you please elaborate on how using ZeroMQ would solve your problem? I have a surprisingly similar setup and I'm also worried that this is going to be a bottleneck.
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#8 > tens of millions of realtime events daily
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#9If you want something similar to this and you're on the JVM, you can use open source Atmosphere[1] which handles this flow exactly (websocket with fallback, multiple 'tags', etc).
Re: The realtime magic behind Moot and how we deliver all your instant notifications
#10Are you going to open-source your json-rpc library? If so, is there a specific timeframe you have in mind? I'd really love to use this technology in my own projects.