It took me a while to understand the usecase but this is a very good solution for apps that are read intensive, wonder why this pattern isn't very common.
Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps
81–90 of 173 posts
Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps
#82It took me a while to understand the usecase but this is a very good solution for apps that are read intensive, wonder why this pattern isn't very common.
Because synchronizing data with multiple users updating data in a distributed manner then reconciling it is not trivial.
Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps
#83I'm curious, CRDTs are good for having data converge in the same state but not necessarily ensuring that the state it converges to is "correct". How does this solve that?
Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps
#84I saw this recently and I am really excited for, hopefully, a renaissance in local-first apps; it's been quite a long time coming. That said, it seems like there's still a lot of problems to work on in this space and it'll take a while for different approaches and implementations to approach their local maxima. I am curious about encryption. Assuming that ElectricSQL is handling essentially all of the syncing, is it…
You can always encrypt local data even if the metadata is not encrypted. But that restricts it to the device itself by default, not even your other devices on the same account can read it.
If you want encrypted and shared with other devices or users you have to surface key management to users which is very difficult to do. Signal and a few others do this but they’re very bespoke to the specific business logic. To provide a general purpose abstraction layer is still an open problem, afaik.
Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps
#85Is the SQLite embedded in the browser?
It is in Firefox.
The SQLite team has been working on making WASM support first class, so should work anywhere WASM does.
Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps
#86I saw this recently and I am really excited for, hopefully, a renaissance in local-first apps; it's been quite a long time coming. That said, it seems like there's still a lot of problems to work on in this space and it'll take a while for different approaches and implementations to approach their local maxima. I am curious about encryption. Assuming that ElectricSQL is handling essentially all of the syncing, is it…
Absolutely, we are really keen to support an encryption pattern. We are at the design phase, the current thinking is to have a hook enabling developers to provided their own encryption callback so that you can have unencrypted data in the local database. Essentially encrypting and decrypting, on and off the sync layer.
Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps
#87I saw this recently and I am really excited for, hopefully, a renaissance in local-first apps; it's been quite a long time coming. That said, it seems like there's still a lot of problems to work on in this space and it'll take a while for different approaches and implementations to approach their local maxima. I am curious about encryption. Assuming that ElectricSQL is handling essentially all of the syncing, is it…
You can always encrypt local data even if the metadata is not encrypted. But that restricts it to the device itself by default, not even your other devices on the same account can read it. If you want encrypted and shared with other devices or users you have to surface key management to users which is very difficult to do. Signal and a few others do this but they’re very bespoke to the specific business logic. To pro…
I'm not asking for the problem of key derivation to be solved by ElectricSQL per se, just wondering if it can handle E2E encrypted data in any way. That said, I disagree with how difficult it is: there are a lot of approaches that are not too ridiculous. For example, a simple approach to E2EE is to use a PAKE like SRP to do password authentication, then since the password is kept secret, you can use a normal KDF to derive a symmetric key from the password. From here, you can e.g. store symmetric and public key matter on a server or synced across your clients, encrypted using the password/KDF. If this sounds familiar, it's exactly how password managers work and the main downsides are that it requires password authentication (can't use any auth mechanism that doesn't somehow discretely convey a secret to the client) and the forgot password situation is more complicated (if you have no backups of the key, you can't recover any data. However, that's not an unreasonable compromise in exchange for E2EE.)
Just like TLS, it'd probably be bad if most people implemented SRP from scratch. That said, I did write my own implementation of SRP-6a (a variant of SRP based on the RSA cryptosystem) in TypeScript and I found it fairly simple to do. There are also PAKEs that provide even better security properties than SRP-6a, but good implementations of them are still lacking for now.
What Matrix does seems pretty simple, it just has a lot of moving parts and nuance; the underlying keys are essentially sent directly between clients after a key exchange is performed and verified out of band to ensure there is no eavesdropping and that the two clients are connected to who they think they are.
As far as I know, Signal sidesteps key management almost entirely and only allows one logged in device, and everything else must proxy through it. That's pretty lame, though I understand the stakes are very high to get it 'right' and keeping the moving parts to a minimum was likely high priority.
Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps
#88I want to be able to have a Postgres database as the central source of truth for all data and user accounts, but then have each users private content to be siloed and synced to their own SQLite database which they alone have access to (maybe even they’re have one SQLite file on server and one SQLite file on phone or etc.). Is this possible with electricSQL? I remember looking at it a year ago or so and was excited bu…
Hey, This is the exact pattern we target :) Drop Electric onto an existing Postgres data model and use the Shape-based sync to manage dynamic partial replication of subsets of data between central Postgres and local SQLites: https://electric-sql.com/docs/usage/data-access/shapes James
This is awesome. I love the pattern, and the way you’ve built for this as a first-class concern is great.
Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps
#89Nice. When I looked at this last some months ago it only supported multi-tentant setups (where a whole db/table is synced) but seems like it now suports row based users, correct?
Hey, yup, a big part of the new release is a proper system for dynamic partial replication, as per https://electric-sql.com/docs/usage/data-access/shapes
Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps
#90I remember pitching this idea to a team of colleagues about 11 years ago. People weren’t ready for it and I never took it anywhere. I really hope this works like my dreams!