Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accidental database programming

sqlsync.dev

Post reply on HN
Print view View original

51–60 of 310 posts

Re: Accidental database programming

#51

Post by wouldbecouldbe »

giancarlostoro wrote: post #35
neilk wrote: post #21

I’m familiar with this project - the creator is a friend. I’ll try to get him on here to answer questions. He’s a seasoned database architect. With SQLsync he’s made a way for frontend developers to query and update a remote database as if it was completely located right in the browser. Because it basically is. The power of WASM makes it possible to ship a whole SQLite database to the browser. The magic is in how it…

Genuinely curious why not just cache the relevant bits in LocalStorage / SessionStorage? I seem to remember Chrome trying to add a literal SQL database to the browser, but it never panned out, localStorage became king. I don't mean to downplay the usefulness, just I usually opt for what the browser gives me. I'm huge on WASM and what it will do for the browser as it matures more (or grows in features).

Because if this works it's amazing. Realtime sync with offline support out of the box, while not having to develop state management on client and api, but in one place. Those are very hard problems, done with less development. Will definitely give it a shot.

Re: Accidental database programming

#52

Post by commonenemy »

neilk wrote: post #21

I’m familiar with this project - the creator is a friend. I’ll try to get him on here to answer questions. He’s a seasoned database architect. With SQLsync he’s made a way for frontend developers to query and update a remote database as if it was completely located right in the browser. Because it basically is. The power of WASM makes it possible to ship a whole SQLite database to the browser. The magic is in how it…

That sounds awfully like Couchbase, which allows you to query/update databases that will sync to remote and the back to peers. And you can control the process (auth/business logic) with sever side JavaScript plugin with ease.

Re: Accidental database programming

#54

Post by carlsverre »

nikita wrote: post #38

My understanding is that we can sync any sqlite state to any other sqlite state using custom built replication. Is this how it works and how does it update all the web components? Would it work with all the frameworks or a custom framework is needed?

That's the dream! Currently SQLSync wraps SQLite in a fairly heavy weight way as it needs to both intercept the storage tier (to replicate page changes) as well as the mutation layer (to provide the reducer api). I'm interested in making it lighter weight and perhaps a regular SQLite extension you could install into any instance of SQLite.

As for the web integration, SQLSync works with any framework but currently only ships with a React library. Most of it's logic is framework agnostic though.

SQLSync also provides a query subscription layer that is table-level reactive. What this means it that the client API can subscribe to a query which will automatically re-run when any table dependencies change. I'm exploring more granular reactivity, however for many datasets re-running on table change is sufficient when coupled with OLTP query patterns and small-medium sized data.

Re: Accidental database programming

#55

Post by hughesjj »

cdchn wrote: post #45

Give someone state and they'll have a bug one day, but teach them how to represent state in two separate locations that have to be kept in sync and they'll have bugs for a lifetime -ryg

Collollary: if you don't represent state in more than one place, you'll eventually run into loss of availability (accessibility), integrity, of existence of data

Thus, bugs forever is a given.

Re: Accidental database programming

#57

Post by PH95VuimJjqBqy »

bradley13 wrote: post #46
0xbadcafebee wrote: post #20

Anyone remember when "frontend applications" were actual applications and not web pages? I'm willing to bet we have reached that point where new devs literally do not remember that time. There comes a time [..] where we [..] need to cache data from an API. It might start off benign – storing a previous page of data for that instant back button experience, implementing a bit of undo logic, or merging some state from d…

History doesn't repeat itself, but it does rhyme. One upon a dark age, we had mainframes and dumb terminals. Then came the first age of the PC - let's pull everything to the client. Then came servers and "thin clients". With faster processors and cheaper storage came the second age of the PC, with only the permanent data storage left on the server. As the Internet grew, centralization came back: web services and the…

years and years ago on a C++ forum someone made an observation that was eerily similar to yours. I still remember it to this day as it stuck in my head.

They made an observation that our industry goes in cyclical centralize/de-centralize cycles and that we we were (at the time) entering into a centralization cycle.

Now here I am reading a comment that we're going back into a de-centralization cycle and I wouldn't be surprised if you're the same poster.

probably 15-20 years ago (maybe more?) I made a prediction that I still think will come true.

The OS will become the "browser" and applications will run directly on the OS and will access local resources through standardized interfaces. WebAssembly and things like WebGL are already moving us in that direction. Honestly HTML5 was the first time I recognized standard updates as moving us towards that reality with things like localStorage, etc.

I honestly think if someone more imaginative had the helm at MS when the cloud started getting big they would have eaten google's lunch by leveraging their desktop dominance into the web. Instead they dd inane things like display websites on the desktop (win98 IIRC).

Re: Accidental database programming

#58

Post by PH95VuimJjqBqy »

0xbadcafebee wrote: post #20

Anyone remember when "frontend applications" were actual applications and not web pages? I'm willing to bet we have reached that point where new devs literally do not remember that time. There comes a time [..] where we [..] need to cache data from an API. It might start off benign – storing a previous page of data for that instant back button experience, implementing a bit of undo logic, or merging some state from d…

I completely agree with you, I'm going to copy part of another comment I made

-----

probably 15-20 years ago (maybe more?) I made a prediction that I still think will come true.

The OS will become the "browser" and applications will run directly on the OS and will access local resources through standardized interfaces. WebAssembly and things like WebGL are already moving us in that direction. Honestly HTML5 was the first time I recognized standard updates as moving us towards that reality with things like localStorage, etc.

I honestly think if someone more imaginative had the helm at MS when the cloud started getting big they would have eaten google's lunch by leveraging their desktop dominance into the web. Instead they did inane things like display websites on the desktop (win98 IIRC).

-----

Re: Accidental database programming

#59

Post by carlsverre »

robocat wrote: post #41
HatchedLake721 wrote: post #4

Unless I misunderstood, feels like I’ve been doing this with Ember Data since ~2013. https://guides.emberjs.com/release/models/ There’s also https://orbitjs.com/

I think you have misunderstood? The article is responding to the pattern of yet another custom data model and custom data API (à la Ember). Instead provide an SQL database (the well proven SQLite) within the front end and use SQL to interact. And sync data from-to the backend DB. Which one could then slap on a model or ORM layer on top of - should that be one's bent. It isn't clear how they manage the subscription to…

First, thanks!

> It isn't clear how they manage the subscription to data updates/inserts/deletions - it mentions supporting triggers, but that feels icky to me.

Architecture post coming soon. In the meantime, I want to clarify that SQLSync does not use triggers for sync. Instead, I hijack SQLites page storage and added page replication to it. Writes are consolidated through an API I call the "reducer" which allows SQLSync to keep track of which logical writes correspond to which sets of page changes. The actual sync is pretty dumb: we run the reducer on both the client and the server. The client replicates down server pages, and then periodically throws out local changes, resets to the server state, and then replays any mutations that haven't yet been acked on the server.

Post reply on HN