Anyone have any info on who is behind this project, how reliable it is (will it be around in 2 years), etc? Considering using it for one of my projects.
[deleted]
Automerge-Repo: A "batteries-included" toolkit for local-first applications
21–30 of 45 posts
Re: Automerge-Repo: A "batteries-included" toolkit for local-first applications
#22Earlier quoted context omitted.
still not as fast/efficient as Yjs. there are some benchmarks here from late September’23: https://arxiv.org/abs/2212.02618 disclaimer: i’m a co-author and the paper is focused on a different CRDT framework, but point is that it measures Yjs and automerge side by side
The benchmarks Matt Weidner has been working on are great and outside scrutiny is always welcome, but I should note that I find there's an element of artificiality to them. In particular, testing the performance of the sync system while simulating many users typing into the same document doesn't really measure behaviour we have observed "in the wild". In our research, we've found that editing is usually serial or asy…
Ouch Peter. Massive offense taken.
> In our research, we've found that editing is usually serial or asynchronous.
Medium-to-large-size company with a town hall = many people editing a document at the same time. Workshop at a company or a university with a modest size classroom = many people editing a document at the same time. I can't tell you how many times our web-based collaborative code editors would fall over during talks with small audiences we would give back in the days when I led the Scala Center.
Just because one of the benchmarks you have seen (of a multitude of benchmarks) breaks automerge by stressing it in what we believe is the most stressful scenario possible– multiple concurrent users, which is sort of the point of concurrency/collaboration frameworks, does not make it artificial or worth so flippantly discarding.
> long-form essay with Automerge and that the end-to-end keypress-to-paint latency using Codemirror is under 10ms (next frame at 100hz)
Not at all what we measured.
I'd just like to register here that Yjs is the framework most widely used "in real usage" (your words) and not automerge (for many reasons, not just performance.)
Re: Automerge-Repo: A "batteries-included" toolkit for local-first applications
#23Earlier quoted context omitted.
The benchmarks Matt Weidner has been working on are great and outside scrutiny is always welcome, but I should note that I find there's an element of artificiality to them. In particular, testing the performance of the sync system while simulating many users typing into the same document doesn't really measure behaviour we have observed "in the wild". In our research, we've found that editing is usually serial or asy…
Ouch Peter. Massive offense taken. > In our research, we've found that editing is usually serial or asynchronous. Medium-to-large-size company with a town hall = many people editing a document at the same time. Workshop at a company or a university with a modest size classroom = many people editing a document at the same time. I can't tell you how many times our web-based collaborative code editors would fall over du…
Please accept my unreserved apologies, Heather! No offense is intended. I can speak for everyone working on Automerge when I say that we've very much appreciated Matthew's work and have indeed spent quite a lot of time studying and responding to it. We spoke about it in person last week, in fact.
As for the use-cases, I do not mean to exclude live collaboration from consideration, just to note that it hasn't been our focus or come up often in the use-cases we study. Live meeting notes are definitely a real use-case and I don't dispute the performance results you show.
As for Y-js, it's a wonderful piece of software with excellent performance and a vibrant community made by exceptional people like Kevin Jahns. We simply have slightly different goals in our work, which undoubtedly reflect where our engineering investments lie.
Indeed, your paper did not measure the same things we look at, and that's why it found new results. Hopefully in time we will join the other systems in performing well on your benchmarks as well.
Re: Automerge-Repo: A "batteries-included" toolkit for local-first applications
#24Cool stuff! What do you suggest is the sweet spot for document size and "hotness"? Your cookbook [0] says "We suspect that an Automerge document is best suited to being a unit of collaboration between two people or a small group." Does that mean tens of kilobytes? Hundreds? More? And how much concurrent contention is viable? And is the "atom of contention" the document as a whole, or do you have any plans for merging…
The way I think about it is that if the data should always travel together it should be in one document. For example -- if your TODO list always goes as a unit, then make it an array of objects in a single Automerge document. On the other hand, if you want to build an issue tracker and to be able to link to individual issues or share them individually then a document each is the way to go. Does that help? As for netw…
Cool, thanks for the details.
So if I smoosh everything in my sorta “collaboration context” together into one document, are there any provisions for delta updates on the wire? Your browser-side storage format sounds like it’s compatible with that approach, but what about clients that are far apart version-wise? Are you storing full relay history and also a snapshot?
I see in your format docs [0] that you store change chunks. Are these exposed in the API for atomicity at all? Are there any atomicity guaranties?
And you discuss backends, but I don’t see any pointers to an S3 or Postgres implementation. Is that something you’re keeping closed source for your business model, or am I just missing something?
I haven’t found anything about authorization. Have you done any work there? I quite like the Firebase model in which you can write simple validation rules that can evaluate against the document itself —- “only allow users who are listed in path `members` to write to this document” or whatever.
[0] https://automerge.org/automerge-binary-format-spec/#chunk-co...
Re: Automerge-Repo: A "batteries-included" toolkit for local-first applications
#25This is exciting in several ways including the fact that
Martin Kleppmann is involved with the project. Filed at the top of my reading list if nothing else to see an undoubtedly good example of a complex Rust project.
Re: Automerge-Repo: A "batteries-included" toolkit for local-first applications
#26Earlier quoted context omitted.
still not as fast/efficient as Yjs. there are some benchmarks here from late September’23: https://arxiv.org/abs/2212.02618 disclaimer: i’m a co-author and the paper is focused on a different CRDT framework, but point is that it measures Yjs and automerge side by side
The benchmarks Matt Weidner has been working on are great and outside scrutiny is always welcome, but I should note that I find there's an element of artificiality to them. In particular, testing the performance of the sync system while simulating many users typing into the same document doesn't really measure behaviour we have observed "in the wild". In our research, we've found that editing is usually serial or asy…
> there's an element of artificiality to them. In particular, testing the performance of the sync system while simulating many users typing into the same document doesn't really measure behaviour we have observed "in the wild".
I've seen Matt's work and I think it's quite reasonable to benchmark a concurrent datastructure under concurrent load. Placing systems under high load, even just as a limit study, is how we reveal scalability bottlenecks, optimize them, and avoid pathologies. It's part of good engineering.
If your work can produce more representative workloads from the real world, then they could add to the field's knowledge with new benchmarks.
Re: Automerge-Repo: A "batteries-included" toolkit for local-first applications
#27Curious how you think about this compared to Electric SQL [1] - I'm currently deciding what sync solution we're going to use for a product rebuild and have been looking at quite a few
Re: Automerge-Repo: A "batteries-included" toolkit for local-first applications
#28Anyone have any info on who is behind this project, how reliable it is (will it be around in 2 years), etc? Considering using it for one of my projects.
Quick note for the casual reader that this is one of the oldest and preeminent projects in the space.
Re: Automerge-Repo: A "batteries-included" toolkit for local-first applications
#29By calling it "repos", they're trying to capitalize on the popularity of VCS repositories, but these don't have their history implicitly tracked the way Automerge does, just explicitly tracked by committing and pushing.
I think it's cool, but I still see CRDTs as very niche.
I also want "local-first" but what I really want is something closer to how traditional desktop apps just open up, edit, and save files, not some real time collaboration that is already set up before I add my first collaborator.
Re: Automerge-Repo: A "batteries-included" toolkit for local-first applications
#30Earlier quoted context omitted.
The way I think about it is that if the data should always travel together it should be in one document. For example -- if your TODO list always goes as a unit, then make it an array of objects in a single Automerge document. On the other hand, if you want to build an issue tracker and to be able to link to individual issues or share them individually then a document each is the way to go. Does that help? As for netw…
Cool, thanks for the details. So if I smoosh everything in my sorta “collaboration context” together into one document, are there any provisions for delta updates on the wire? Your browser-side storage format sounds like it’s compatible with that approach, but what about clients that are far apart version-wise? Are you storing full relay history and also a snapshot? I see in your format docs [0] that you store change…
The sync protocol does indeed calculate the delta between peers and efficiently catches both sides up.
The backends you see are the ones I use, but the API is a binary blob key value store with range queries: supporting other stores should be straightforward.
Authentication isn’t exactly left as an exercise to the reader but is an area of active work. I would say securing access to a URL via whatever mechanism you’re used to should be fine for client server applications and peer to peer folk seem to mostly have their own ideas.