This is amazing. Please share with the big projects which need it the most.. collabora and libreoffice. Also, a product which the world needs badly.. would be a software which would abstract git and present text to lawyers as regular word processor, but in the backend it's git for the win.
Loro: Reimagine state management with CRDTs
21–30 of 45 posts
Re: Loro: Reimagine state management with CRDTs
#22This is amazing. Please share with the big projects which need it the most.. collabora and libreoffice. Also, a product which the world needs badly.. would be a software which would abstract git and present text to lawyers as regular word processor, but in the backend it's git for the win.
> Also, a product which the world needs badly.. would be a software which would abstract git and present text to lawyers as regular word processor, but in the backend it's git for the win.
Ink & Switch Upwelling [0] goes into that direction. A must-watch is the StrangeLoop 2023 talk by Martin Kleppmann "New Algorithms for Collaborative Editing" [1] that excellently explains things.
Re: Loro: Reimagine state management with CRDTs
#23We've been using https://github.com/electric-sql/electric for real-time sync for the past month or so and it's been great. Rather than make you think about CRDTs explicitly, Electric syncs an in-browser sqlite db (WASM powered) with a central postgres instance. As a developer, you get local-first performance and real-time sync between users. And it's actually faster to ship an application without writing any APIs and…
How are conflicts resolved?
Re: Loro: Reimagine state management with CRDTs
#24We've been using https://github.com/electric-sql/electric for real-time sync for the past month or so and it's been great. Rather than make you think about CRDTs explicitly, Electric syncs an in-browser sqlite db (WASM powered) with a central postgres instance. As a developer, you get local-first performance and real-time sync between users. And it's actually faster to ship an application without writing any APIs and…
I've been wondering how well Electric's been working for people ever since I heard about it; good to hear that it's been useful for you.
Couple of questions:
- How big is the WASM blob that you need to ship for in-browser SQLite? Have you had any noticable issues from shipping a large payload to the browser?
- What are you using to persist the SQLite database on clients? Have you been using the Origin Private File System?
Re: Loro: Reimagine state management with CRDTs
#25We've been using https://github.com/electric-sql/electric for real-time sync for the past month or so and it's been great. Rather than make you think about CRDTs explicitly, Electric syncs an in-browser sqlite db (WASM powered) with a central postgres instance. As a developer, you get local-first performance and real-time sync between users. And it's actually faster to ship an application without writing any APIs and…
How are conflicts resolved?
With something called Rich-CRDTs - they were invented by Electric's CTO. They have a section in the docs and some blog posts dedicated to it: https://electric-sql.com/docs/reference/consistency#rich-crd...
Re: Loro: Reimagine state management with CRDTs
#26We've been using https://github.com/electric-sql/electric for real-time sync for the past month or so and it's been great. Rather than make you think about CRDTs explicitly, Electric syncs an in-browser sqlite db (WASM powered) with a central postgres instance. As a developer, you get local-first performance and real-time sync between users. And it's actually faster to ship an application without writing any APIs and…
I've been wondering how well Electric's been working for people ever since I heard about it; good to hear that it's been useful for you. Couple of questions: - How big is the WASM blob that you need to ship for in-browser SQLite? Have you had any noticable issues from shipping a large payload to the browser? - What are you using to persist the SQLite database on clients? Have you been using the Origin Private File Sy…
This is the WASM blob and it's 1.1 MB uncompressed. https://github.com/rhashimoto/wa-sqlite/blob/master/dist/wa-.... No issues - it's cached by cloudflare.
We're using IndexedDB. Here's a writeup on alternatives https://github.com/rhashimoto/wa-sqlite/issues/85 and a benchmark https://rhashimoto.github.io/wa-sqlite/demo/benchmarks.html
Re: Loro: Reimagine state management with CRDTs
#27Earlier quoted context omitted.
I've been wondering how well Electric's been working for people ever since I heard about it; good to hear that it's been useful for you. Couple of questions: - How big is the WASM blob that you need to ship for in-browser SQLite? Have you had any noticable issues from shipping a large payload to the browser? - What are you using to persist the SQLite database on clients? Have you been using the Origin Private File Sy…
This is the WASM blob and it's 1.1 MB uncompressed. https://github.com/rhashimoto/wa-sqlite/blob/master/dist/wa-... . No issues - it's cached by cloudflare. We're using IndexedDB. Here's a writeup on alternatives https://github.com/rhashimoto/wa-sqlite/issues/85 and a benchmark https://rhashimoto.github.io/wa-sqlite/demo/benchmarks.html
Gotcha, interesting. 1.1 MB isn't too bad, especially with Cloudflare providing a local PoP. And if this is for Hocus, I'm guessing your frontend isn't used much on mobile devices with iffy connections.
That writeup on different SQLite VFS's for in-browser use is helpful, thanks for linking that.
Re: Loro: Reimagine state management with CRDTs
#28Can someone explain to me what happens when there is a destructive update on one side while the other side is still relying on some old version?
Can this even be reconciliated?
Or is it append only?i.e. No delete operation.
UIs have delete operations in general.
Re: Loro: Reimagine state management with CRDTs
#29Earlier quoted context omitted.
What kinds of bugs have you run into? Any large-scale corruption of data at rest?
We have run into queries that corrupted the database client-side, but fortunately that doesn't propagate into postgres itself. In that case we had to drop the client-side db and resync from a clean state. The corruption was also caught by sqlite itself - it threw a "malformed disk image" error and stopped responding to any further queries. Also bugs around syncing some kinds of data - one bug that's already been fixe…
Any idea on what the root cause of the sqlite corruption was? There's some discussion on the SQLite forums about corruption with wasm (I've encountered it myself on a personal project), but from what I understand no one has identified a cause yet.
Re: Loro: Reimagine state management with CRDTs
#30The demo code for Loro looks very easy to use, I love how they infer a CRDT from an example plain JS object. I’ve played with a Zod schema Yjs CRDT translator and found it kinda annoying to maintain. However this looks so easy I worry about apps building with too little thought about long term data modeling. Migrations on CRDTs are challenging, so it’s important to “get it right” at the beginning. I’m curious how thi…
If you're looking at declaring schemas for CRDT docs in Yjs, I've found https://syncedstore.org/docs/ productive if you're open to using TypeScript.
Agreed on migrations!