Eventual Consistency in Real-time Web Apps
blog.johnryding.com
Eventual Consistency in Real-time Web Apps
1–10 of 27 posts
Re: Eventual Consistency in Real-time Web Apps
#2Are there any "standard" models for treating a realtime web app as just another distributed database node (with, of course, extra security precautions and having to do server-side data re-validation)? I'm aware of CouchDB/PouchDB and Meteor's use of mini-Mongo client-side. Are there others?
Re: Eventual Consistency in Real-time Web Apps
#3Are there any "standard" models for treating a realtime web app as just another distributed database node (with, of course, extra security precautions and having to do server-side data re-validation)? I'm aware of CouchDB/PouchDB and Meteor's use of mini-Mongo client-side. Are there others?
Actors.
Re: Eventual Consistency in Real-time Web Apps
#4Are there any "standard" models for treating a realtime web app as just another distributed database node (with, of course, extra security precautions and having to do server-side data re-validation)? I'm aware of CouchDB/PouchDB and Meteor's use of mini-Mongo client-side. Are there others?
Actors.
Actors says nothing about consistency; you have to do it all on your own. Basically a non solution in this area.
Re: Eventual Consistency in Real-time Web Apps
#5I hadn't seen the merge / fill idea before. Worth reading to get more ideas on this subject.
OT but seems like a good time to ask, does anyone have any experience with Operational Transformations for dealing with syncing in web apps? I've started throwing the idea around for a web app I'm building and it seems like a really interesting pattern. The only real implementation I can find is in sharejs [1]. I work with python on the backend and I can't really find an implementation of it (though for my constrained use-case I can generate good enough code to do it myself).
It seems like a good approach but there's very little in the way of libraries implementing it, so I was wondering if it's somehow problematic.
Re: Eventual Consistency in Real-time Web Apps
#6Are there any "standard" models for treating a realtime web app as just another distributed database node (with, of course, extra security precautions and having to do server-side data re-validation)? I'm aware of CouchDB/PouchDB and Meteor's use of mini-Mongo client-side. Are there others?
If you have a CRDT implementation you can use on both sides, you can perform operations on the client and merge with the server correctly.
Re: Eventual Consistency in Real-time Web Apps
#7It's a lot simpler to just have the socket, with a notification when your PUT has landed. This is what Meteor does.
I don't see anything I recognize as eventual consistency in the traditional sense. It sounds like the client is just trying to receive a stream of updates, which could have been made by a single writer, and the only consistency issues are caused by the different overlapping mechanisms for getting updates.
Re: Eventual Consistency in Real-time Web Apps
#8Are there any "standard" models for treating a realtime web app as just another distributed database node (with, of course, extra security precautions and having to do server-side data re-validation)? I'm aware of CouchDB/PouchDB and Meteor's use of mini-Mongo client-side. Are there others?
If you have a CRDT implementation you can use on both sides, you can perform operations on the client and merge with the server correctly.
That's exactly what I'm planning on doing, as that seems to be simultaneously correct and relatively easy to program. I'm surprised, though, that I haven't found a "nicely packaged" version of some CRDTs, a server DB, and a client-side DB. I suppose it is probably because everyone's use-cases are so different and that using CRDTs limit the data you can use, to some degree.
Re: Eventual Consistency in Real-time Web Apps
#9I hadn't seen the merge / fill idea before. Worth reading to get more ideas on this subject. OT but seems like a good time to ask, does anyone have any experience with Operational Transformations for dealing with syncing in web apps? I've started throwing the idea around for a web app I'm building and it seems like a really interesting pattern. The only real implementation I can find is in sharejs [1]. I work with py…
In general, I've read that OT tends to be very complicated to get right, which is why only a few libraries implement it. If you can use what's called a CRDT, your life will be easier, but there are fewer data structures that are supported.
Aral Balkan gave a talk on OT vs a CRDT called WOOT for text editing[1] that I found really helpful.
If you want more of the nitty-gritty on some of the different types of CRDT, there's fairly readable paper on the topic[2].
[1] https://www.youtube.com/watch?v=NSTZ4mIv_wk [2] http://hal.upmc.fr/docs/00/55/55/88/PDF/techreport.pdf
Re: Eventual Consistency in Real-time Web Apps
#10I hadn't seen the merge / fill idea before. Worth reading to get more ideas on this subject. OT but seems like a good time to ask, does anyone have any experience with Operational Transformations for dealing with syncing in web apps? I've started throwing the idea around for a web app I'm building and it seems like a really interesting pattern. The only real implementation I can find is in sharejs [1]. I work with py…
In general, I've read that OT tends to be very complicated to get right, which is why only a few libraries implement it. If you can use what's called a CRDT, your life will be easier, but there are fewer data structures that are supported. Aral Balkan gave a talk on OT vs a CRDT called WOOT for text editing[1] that I found really helpful. If you want more of the nitty-gritty on some of the different types of CRDT, th…
That's great information, thank you. The paper looks really interesting - going to take a little while to get through, might start with the video :)
I'd also read that it's hard to get right (I think that was in a comment on HN by one of the people who worked on wave/sharejs). In my case I've limited myself to two different flavours of small object with a limited set of fields (instead of a totally generalised system). Most of the algorithms I've seen focus on the text element of the syncing but I'm much more interested in the object / list / field part for my use case.
Sharejs actually have a generic system of handling the OT on arbitrary json structures, but I'm not sure how well it works.