> Graft should be considered Alpha quality software. Thus, don't use it for production workloads yet. Beta ETA?
ASAP :)
Stop syncing everything
11–20 of 131 posts
Re: Stop syncing everything
#12How does this compare with Turso? I know it's mentioned in the article (mainly better support for partial replication and arbitrary schemas), but is there also a deeper architectural departure between the two projects? Looks really good, great work!
Thank you! Generally Turso has focused on operating more like a traditional network attached backend. Although that has changed recently with libsql and embedded replicas. I think at this point the main delta is they use traditional wal based physical replication while Graft is something new that permits trivial partial replication. Also, Graft is not exclusive to SQLite. It’s just transactional page addressed object storage with built in replication. I’m excited to see what people build on it.
Re: Stop syncing everything
#13Seems interesting. A very challenging problem to wrap your head around. Anyone working on this is exactly pushing the field forward.
I'm thinking to give it a try in one of my React Native apps that face very uncertain connectivity.
Re: Stop syncing everything
#14Re: Stop syncing everything
#15My ideal version of this is simple: just define the queries you want (no matter how complex) and the you'll get exactly the data you need to fulfill those queries, no more, no less. And the cherry on top would be to have your queries update automatically with changes both locally and remote in close to real-time. That's basically what we're doing with Triplit ( https://triplit.dev ), be it, not with SQL--which is a p…
I heavily disagree with the notion that most developers would rather query with something that isn't SQL.
Re: Stop syncing everything
#16My ideal version of this is simple: just define the queries you want (no matter how complex) and the you'll get exactly the data you need to fulfill those queries, no more, no less. And the cherry on top would be to have your queries update automatically with changes both locally and remote in close to real-time. That's basically what we're doing with Triplit ( https://triplit.dev ), be it, not with SQL--which is a p…
I heavily disagree with the notion that most developers would rather query with something that isn't SQL.
Funny, as Triplit front page shows a query much like SQL:
const deliveredMessagesQuery = client
.query("messages")
.Where("conversationId", "=", convoId)
.Order("created_at", "DESC")Re: Stop syncing everything
#17This is a really interesting project, and a great read. I learned a lot. I'm falling down the rabbit hole pretty hard reading about the "Leap" algorithm (https://www.usenix.org/system/files/atc20-maruf.pdf) it uses to predict remote memory prefetches.
It's easy to focus on libgraft's SQLite integration (comparing to turso, etc), but I appreciate that the author approached this as a more general and lower-level distributed storage problem. If it proves robust in practice, I could see this being used for a lot more than just sqlite.
At the same time, I think "low level general solutions" are often unhinged when they're not guided by concrete experience. The author's experience with sqlsync, and applying graft to sqlite on day one, feels like it gives them standing to take a stab at a general solution. I like the approach they came up with, particularly shifting responsibility for reconciliation to the application/client layer. Because reconciliation lives heavily in tradeoff space, it feels right to require the application to think closely about how they want to do it.
A lot of the questions here are requesting comparison's to existing SQLite replication systems, the article actually has a great section on this topic at the bottom: https://sqlsync.dev/posts/stop-syncing-everything/#compariso...
Re: Stop syncing everything
#18Seems interesting. A very challenging problem to wrap your head around. Anyone working on this is exactly pushing the field forward. I'm thinking to give it a try in one of my React Native apps that face very uncertain connectivity.
> I'm thinking to give it a try in one of my React Native apps that face very uncertain connectivity.
Some similar stuff you may want to investigate (no real opinion, just sharing since I've investigated this space a bit):
https://localfirstweb.dev is a good link too.
Re: Stop syncing everything
#19> After a client pulls a graft, it knows exactly what’s changed. It can use that information to determine precisely which pages are still valid and which pages need to be fetched Curious how this compares to Cloud-Backed SQLite’s manifest: https://sqlite.org/cloudsqlite/doc/trunk/www/index.wiki It’s similar to your design (sending changed pages), but doesn’t need any compute on the server, which I think is a huge win…
Woah, hadn't seen this before but this is really cool!
I was recently looking for a way to do low scale serverless db in gcloud, this might be better than any of their actual offerings.
Cloud firestore seems like the obvious choice, but I couldn't figure out a way to make it work with existing gcloud credentials that are ubiquitous in our dev and CI environments. Maybe a skill issue.
Re: Stop syncing everything
#20This is a really interesting project, and a great read. I learned a lot. I'm falling down the rabbit hole pretty hard reading about the "Leap" algorithm ( https://www.usenix.org/system/files/atc20-maruf.pdf ) it uses to predict remote memory prefetches. It's easy to focus on libgraft's SQLite integration (comparing to turso, etc), but I appreciate that the author approached this as a more general and lower-level dist…
Thank you! I'm extremely excited and interested to explore applying Graft to solutions outside of SQLlite/SQLSync. That was a driving factor behind why I decided to make it more general. But you're absolutely right, I'm glad I spent time developing use cases first and then worked backwards to a general solution. I made a lot of mistakes in the process that I wouldn't have seen if I had gone the other way.
And yea, I fell pretty far down the "Leap" rabbit hole. It's a fun one :)