Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps

electric-sql.com

Post reply on HN
Print view View original

101–110 of 173 posts

Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps

#101

Post by meiraleal »

ochiba wrote: post #50
koolba wrote: post #9

Is there any validation or authorization for changes being merged back into the root database? In a traditional client / server model, the server has an opportunity to validate each request and optionally reject it. The lower level you go with the sync protocol (data changes vs high level requests) the more difficult that becomes. Have you addressed that and, if so, how? What prevents a malicious client from send arb…

The route that we went with for PowerSync (disclosure: co-founder) is to allow to define your own function for handling writes, where you would use your (presumably existing) backend application API to persist the writes to Postgres. Therefore you can handle any validation/authZ/business logic for writes in your backend layer. The PowerSync client SDK still handles the queueing and retrying of writes so that they can…

You are really taking the expression `shameless plug` to another level in this thread, aren't you?

Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps

#102

Post by ochiba »

meiraleal wrote: post #101
ochiba wrote: post #50

Earlier quoted context omitted.

The route that we went with for PowerSync (disclosure: co-founder) is to allow to define your own function for handling writes, where you would use your (presumably existing) backend application API to persist the writes to Postgres. Therefore you can handle any validation/authZ/business logic for writes in your backend layer. The PowerSync client SDK still handles the queueing and retrying of writes so that they can…

You are really taking the expression `shameless plug` to another level in this thread, aren't you?

Genuinely excited about this space and it's what I'm focused on full-time so definitely have thoughts to share. I am wary of self-promotion. I do want to contribute things that I feel are relevant to the discussion, since I assume folks would be interested to see different patterns/approaches around local-first/offline-first architecture.

Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps

#105

Post by thruflo »

aidos wrote: post #100
thruflo wrote: post #34

Earlier quoted context omitted.

Hey, This is the exact pattern we target :) Drop Electric onto an existing Postgres data model and use the Shape-based sync to manage dynamic partial replication of subsets of data between central Postgres and local SQLites: https://electric-sql.com/docs/usage/data-access/shapes James

Congrats on the launch. I’ve been keeping an eye on these sorts of tools for a while and had a look at this after it was mentioned on HN the other day. Looks great. We currently use Hasura subscriptions to pull data to the frontend (effectively select * from table where account_id = X and updated > recently). We then funnel changed rows into mobx objects so we have a lovely object graph to work with. I’m imagining do…

Interesting, I don’t know enough about MobX to be sure but yes, you can use the notifier to subscribe to data change notifications and query to get the changed rows.

We’d definitely be interested in chatting through the reactivity model you use if you’d be up for it.

Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps

#106

Post by ochiba »

afavour wrote: post #63
ochiba wrote: post #43

Earlier quoted context omitted.

This is also the pattern targeted by PowerSync (disclosure: co-founder) — selectively syncing scoped data for the user from Postgres to their own client SQLite database. Sync Rules are used to define which data is synced to which users: https://docs.powersync.co/usage/sync-rules

This is the fifth comment I’ve counted plugging Powersync in this thread. I guess it’s the nature of the startup hustle game but this level of promotion in someone else’s Show HN thread feels a little distasteful. Maybe you could post your own Show HN?

I hear you, thanks for sharing your thoughts and for the suggestion. Noted. I didn’t mean to just plug — I thought I was contributing to the discussion and that some people may find another implementation relevant/interesting.

Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps

#107

Post by irq-1 »

thruflo wrote: post #33
addisonj wrote: post #25

Congrats on the launch and interesting project! I have a lot of questions :) * I am by no means a crdt expert, but from my tinkering I have come to the opinion that the upside of the consistency does come at the cost of ease of understanding how state ends up how it does, which can bleed into user facing issues if how a crdt converges conflicts with intuition. A lot of that seems like an education and design problem…

Hey, thanks :) We have a Discord if you fancy chatting at more length! https://discord.electric-sql.com Re: CRDTs and intuition, yup, there's trade offs. We are working hard to deliver a solid model to code against. So we have finality of local writes and standard relational integrity guarantees. You can read a bit more about some of techniques here https://electric-sql.com/blog/2022/05/03/introducing-rich-cr... Re:…

Wouldn't making a backend service be better all around? All CRDT data could be stored elsewhere. The service could be easily updated and changed, and without changes the database. Any database could be used. Could have LDAP integration and other services.

I'm sure I don't understand CRDTs. Can they be used with database changes that are not part of the CRDT history? That is, can they be added to an existing database with other clients?

Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps

#108

Post by vbalegas »

mnahkies wrote: post #74
vbalegas wrote: post #70

Earlier quoted context omitted.

Valter from Electric here, Handling encryption/decryption of user data is one of our priorities. We're thinking along the lines of adding hooks for transforming data as you're seeing it. We're getting there on permissions implementations. If you want to do row-based filtering based on user_id today, you can add a special column 'electric_user_id' to your table: https://electric-sql.com/docs/reference/roadmap#shapes W…

Are you / have you considered leveraging postgres' row level security for this (row filtering)? Feels like a natural fit for ensuring the right subset of data gets to the right users https://www.postgresql.org/docs/current/ddl-rowsecurity.html E2E encryption would also be nice to have, it could be worth having a look at https://www.etesync.com/ for inspiration

You definitely can do row-level filtering (and a lot more) using DDLX[1], an extension we made to Postgres language that allows expressing flexible permission rules in a data-centric way.

[1] https://electric-sql.com/docs/api/ddlx

Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps

#109

Post by obeavs »

Huge congrats to the team. I reached out to them four or five months ago as we explored offline-first frameworks, and its awesome to see how far they've come in a short time.

We didnt find anyone else who was tackling the tie between client-side SQLite, an open source CRDT/sync layer, and Postgres. We found a few who were attempting to manage this in a closed source way, but it didn't make sense to give up control of the server/auth, and every other fully open source solution was SQLite -> SQLite, not Postgres.

Great stuff from some killer engineers.

Re: Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps

#110

Post by declan_roberts »

I love this project. It's 2023 and I'm still surprised how many projects require an actual database running someplace to work locally.

If the solution is to fire up a docker image with your app, then stop! Why not just use sqlite first?

Post reply on HN