Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: SyncKit – Offline-first sync engine (Rust/WASM and TypeScript)

github.com

Post reply on HN
Print view View original

21–30 of 41 posts

Re: Show HN: SyncKit – Offline-first sync engine (Rust/WASM and TypeScript)

#21

Post by danbitengo »

skybrian wrote: post #15

The conflict resolution guide seems quite clear: https://github.com/Dancode-188/synckit/blob/main/docs/guides... But I’m a little confused about why it says “Zero data loss with automatic conflict resolution” on the top level page and “data loss possible” under the disadvantages of last-write-wins conflict resolution. It makes sense that you have to start somewhere, but to my mind, last-write-wins isn’t really confli…

You're right that this is confusing.

"Zero data loss" means system-level guarantees (no corruption from network failures, crashes, etc), not that concurrent edits to the same field are preserved. LWW definitely clobbers one of the edits.

What v0.1.0 does give you is field-level granularity (edits to different fields don't conflict) and guaranteed convergence (all devices reach the same state). But yeah, same field = last write wins.

The Rust core already has better CRDTs (Text, Counter, Set) that handle concurrent edits properly. They just need to exposed in the TypeScript SDK for v0.2.0.

I should clarify this in the docs. Thanks for pointing it out.

Re: Show HN: SyncKit – Offline-first sync engine (Rust/WASM and TypeScript)

#22

Post by rancar2 »

satvikpendem wrote: post #14
danbitengo wrote: post #10

Earlier quoted context omitted.

Thanks! Yeah, Realm's shift in direction left a gap in the offline-first space. I didn't build Dart bindings yet (Rust → WASM → TypeScript for v0.1.0), but since the core is in Rust, Dart bindings via dart_rust_bridge are definitely feasible. Would actually be a great addition. Are you actively looking for a Realm replacement for a Dart project? Curious what your offline sync requirements are. It might help me priori…

I use Loro CRDT, which is written in Rust as well, on my Dart apps with flutter_rust_bridge and it works pretty well. For syncing I'm using their Loro protocol product although I used iroh before. However, the annoying thing about how the CRDT libraries work is that they're essentially key value NoSQL type storage, not relational. I would love something like SQLite or Postgres on both the server and client that can s…

This is a write up that may be helpful for you: https://sqlsync.dev/posts/stop-syncing-everything/

Re: Show HN: SyncKit – Offline-first sync engine (Rust/WASM and TypeScript)

#23

Post by danbitengo »

p2hari wrote: post #13

Always nice to see something in here. Would love to hear what is your take on https://www.instantdb.com/ ?

I haven't tried InstantDB yet, but it looks really interesting. Thanks for the pointer! Will definitely check it out.

Re: Show HN: SyncKit – Offline-first sync engine (Rust/WASM and TypeScript)

#24

Post by danbitengo »

rancar2 wrote: post #20
danbitengo wrote: post #11

Earlier quoted context omitted.

Thanks! The TLA+ modeling actually caught 3 bugs even before I wrote any code. Worth the upfront investment. It's way easier to debug a state machine model than distributed sync logic. Happy to share more about the verification approach if you're interested!

Please do in the repo, and thank you for the wonderful contribution on multiple fronts. This is very well done work and documentation. A few tips to help others discover the value on the Readme: Add the key memory benchmark (currently just above the acknowledgments) to the top of the Readme and link to the rest of the benchmarks on the page that you already have (maybe rename it benchmarks.md in the process): https:/…

That's a great suggestion. The memory benchmark being buried is a fair point. I'll move it up top and create a proper benchmarks.md page. Thanks for the detailed feedback.

Re: Show HN: SyncKit – Offline-first sync engine (Rust/WASM and TypeScript)

#25

Post by danbitengo »

mkl wrote: post #19
danbitengo wrote: post #3

Hey HN, I built SyncKit after shipping two local-first projects (RestBolt and Graft) and realizing there's no simple way to add cross-device sync. The problem: Existing solutions are either too complex (Automerge/Yjs require learning CRDTs) or too restrictive (Firebase isn't truly local-first, Supabase has no offline support - issue #357 has been open 4+ years with 350+ upvotes). SyncKit is the middle ground: simple…

The ReadMe says "Zero data loss with automatic conflict resolution (Last-Write-Wins)", bur doesn't LWW guarantee data loss?

See my response to skybrian above. You're both asking the same good question about the LWW messaging. Short version: system-level guarantees, not user-level concurrent edit preservation.

Re: Show HN: SyncKit – Offline-first sync engine (Rust/WASM and TypeScript)

#26

Post by danbitengo »

JackYoustra wrote: post #18
danbitengo wrote: post #11

Earlier quoted context omitted.

Thanks! The TLA+ modeling actually caught 3 bugs even before I wrote any code. Worth the upfront investment. It's way easier to debug a state machine model than distributed sync logic. Happy to share more about the verification approach if you're interested!

VERY interested, are you planning on writing a blog? If so I would definitely read that post.

I appreciate the enthusiasm! And yeah, I'm planning to start blogging about the build process and some of the technical decisions probably after launch week settles down. I'll post about it on LinkedIn and in the GitHub Discussions when it's live. If you're following either, you'll see it!

Thanks for the interest!

Re: Show HN: SyncKit – Offline-first sync engine (Rust/WASM and TypeScript)

#27

Post by danbitengo »

satvikpendem wrote: post #14
danbitengo wrote: post #10

Earlier quoted context omitted.

Thanks! Yeah, Realm's shift in direction left a gap in the offline-first space. I didn't build Dart bindings yet (Rust → WASM → TypeScript for v0.1.0), but since the core is in Rust, Dart bindings via dart_rust_bridge are definitely feasible. Would actually be a great addition. Are you actively looking for a Realm replacement for a Dart project? Curious what your offline sync requirements are. It might help me priori…

I use Loro CRDT, which is written in Rust as well, on my Dart apps with flutter_rust_bridge and it works pretty well. For syncing I'm using their Loro protocol product although I used iroh before. However, the annoying thing about how the CRDT libraries work is that they're essentially key value NoSQL type storage, not relational. I would love something like SQLite or Postgres on both the server and client that can s…

You've hit on the core limitation because SyncKit has the same issue. Document/key-value like Loro and Yjs. The relational model problem is fundamental to CRDTs. They guarantee conflict-free convergence because documents are independent, but that breaks with foreign keys and joins. Can't have arbitrary SQL queries and guaranteed convergence without coordination.

ElectricSQL and PowerSync solve it differently (full SQL but weaker offline guarantees). For SyncKit, document-based state is an architectural choice, not something v0.2.0 will change.

I'm curious to know how you're working around this with Loro. Are you building your own query layer or are you keeping everything denormalized?

Re: Show HN: SyncKit – Offline-first sync engine (Rust/WASM and TypeScript)

#28

Post by pm »

When you mentioned offline sync and graft, I mistook you for the author of this project:

https://github.com/orbitinghail/graft

However, they're clearly two different projects.

I don't want to take away from the work you've done, as you're clearly knowledgeable, but as someone else observed, heavy use of AI assistance can be observed in all your public projects. It's worth explicitly addressing, especially considering the foundational nature of your project: it's not easily replaced if it turns out to have to have subtle bugs.

Though I rarely use it myself, I'd like to know, simply because I'm curious as to how other engineers have incorporated such assistance it into their process.

Re: Show HN: SyncKit – Offline-first sync engine (Rust/WASM and TypeScript)

#29

Post by kbenson »

I think I know what this is, but the description is so much in its own context I'm not sure. It's for web-apps that also want an offline local version that works and deals with syncing the data when online again and either local or remote is updated?

It probably markets and explains itself perfectly find for someone in that space and/or looking for this solution, so I'm not sure that's actually a problem, but if you also want to stick in the mind of someone that sees this and doesn't have any current interest, but may stumble into needing a solution like this in the future, a few extra words in your initial description might help it be understood more quickly and be something remembered even if they don't dive into it. Or maybe it's fine and I'm just a bit slow today.

Re: Show HN: SyncKit – Offline-first sync engine (Rust/WASM and TypeScript)

#30

Post by Tooster »

I want to create a local first, offline/p2p realtime multiplayer prototype app soon with reactive/signal data model and frontend agnostic design (considering solidjs/svelte). I'm on a tech research stage. How does it compare to rxdb, tinybase and zero sync? For reference right now I'm considering tinydb/rxdb.
Post reply on HN