Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accidental database programming

sqlsync.dev

Post reply on HN
Print view View original

281–290 of 310 posts

Re: Accidental database programming

#281

Post by lmm »

holoduke wrote: post #254
lmm wrote: post #241

Earlier quoted context omitted.

> What is exactly hard about sql? - No non-nullable types (at the expression level). No way to express e.g. normal boolean logic - No real data structures (at the expression level), unless you count rows, which are not first-class values. Even collections aren't first-class - Very awkward control flow constructs. E.g. look at how you write recursive queries. Even if/else is weird. It's a classical Turing Tarpit: ever…

If you run into issues with non nullable types and missing of real data structures then i guess you dont get the purpose of SQL and its tech. Its a query language to retreive data. thats the only purpose. With under the hood a crafty machine that enables atomic principles and depending on the sql tech other different functionalities like scalability and more.. No Libraries? There are a zillion battle tested libs out…

> If you run into issues with non nullable types and missing of real data structures then i guess you dont get the purpose of SQL and its tech. Its a query language to retreive data. thats the only purpose.

That's no excuse for making it bad.

> No Libraries? There are a zillion battle tested libs out there.

Not in SQL.

> And why you would unit test datasets in a database?

For the same reason as anywhere else.

> Most if not all software stacks have mock libs available.

Exactly. They're a basic piece of functionality you'd expect most languages to provide, but not SQL, because it's awful.

Re: Accidental database programming

#282

Post by pdonis »

lmm wrote: post #237
pdonis wrote: post #144

Earlier quoted context omitted.

What many people either can't or don't want to acknowledge is that ultimately whether or not you support live updates in parallel by multiple users, instead of locking so only one update can proceed at a time, is not a technical decision, it's a business decision: do the business rules that are appropriate for your application enable you to deal with concurrent live updates or not? Ultimately that comes down to wheth…

Nah. I've seen plenty of systems where the business rules would handle concurrent updates fine, but since they're using a traditional Web/ORM/RDBMS setup they build a last-write-wins system without thinking about it. It's one of those rare problems where the technical part is actually harder than the business part.

Database systems have been able to deal with concurrent updates for quite some time now, so I don't think doing this is technically difficult with the current state of the art. Individual dev teams might not be well versed in the current state of the art, but the correct business response to that is not to restrict your business rules but to get developers who are well versed in the current state of the art.

Re: Accidental database programming

#283

Post by 6gvONxR4sf7o »

carlsverre wrote: post #280
leafmeal wrote: post #228

Earlier quoted context omitted.

I feel like you might be miss the point of the parent comment. Synchronizing between databases is a notoriously difficult problem. It's really hard to do while avoiding race conditions. To be fair, I don't understand the "git rebase" technique your project uses, but I'm doubtful it solves these problems. The underlying issue is that users of SQLsync are going to assume that consistency works, but in suble and unsuspe…

I've thought a lot about this, but sure there are still some rough edges to figure out. See this comment for a bit more of a deep dive into how SQLSync works and why it might be possible to make it easier for devs to write correct code that converges. https://news.ycombinator.com/item?id=38502987 Also, I'm considering adding a CRDT column type to SQLite to enable embedded collaborative use cases while using the relat…

A CRDT column type in a relational DB would be amazing for some things I’ve built. I’m really happy to see what you’re doing here!

Re: Accidental database programming

#284

Post by shepherdjerred »

I've found that TanStack Query [0] solved all of my problems around data fetching for the front end. It supports many different frameworks and supports plenty of advanced features.

SQLSync also looks very interesting. I'll try it out in my next project!

[0]: https://tanstack.com/query/latest/

Re: Accidental database programming

#285

Post by lmm »

pdonis wrote: post #282
lmm wrote: post #237

Earlier quoted context omitted.

Nah. I've seen plenty of systems where the business rules would handle concurrent updates fine, but since they're using a traditional Web/ORM/RDBMS setup they build a last-write-wins system without thinking about it. It's one of those rare problems where the technical part is actually harder than the business part.

Database systems have been able to deal with concurrent updates for quite some time now, so I don't think doing this is technically difficult with the current state of the art. Individual dev teams might not be well versed in the current state of the art, but the correct business response to that is not to restrict your business rules but to get developers who are well versed in the current state of the art.

> Database systems have been able to deal with concurrent updates for quite some time now, so I don't think doing this is technically difficult with the current state of the art.

Traditional ACID systems can't really handle them nicely - your only choice with an update is to commit it or discard it - so you have to do a lot of handwritten logic on top, and even if the database itself handles that well, the layers above it generally don't. Event-sourcing style systems work well but they're still not really mainstream yet.

Re: Accidental database programming

#286

Post by hinkley »

Terr_ wrote: post #172
pdonis wrote: post #144

Earlier quoted context omitted.

What many people either can't or don't want to acknowledge is that ultimately whether or not you support live updates in parallel by multiple users, instead of locking so only one update can proceed at a time, is not a technical decision, it's a business decision: do the business rules that are appropriate for your application enable you to deal with concurrent live updates or not? Ultimately that comes down to wheth…

Indeed, many of the most painful technical problems are actually three business problems in a trenchcoat.

Sometimes known as, "two people with firing authority fighting a proxy war through the dev team".

Re: Accidental database programming

#287

Post by willsmith72 »

naasking wrote: post #257
willsmith72 wrote: post #155

Earlier quoted context omitted.

so for a typical spa, you can: 1. always refetch data. always in sync, but needs a server request, so it's slow. 2. cache some data on the client. faster, but you're "building your own database". can get out of sync (redux) 3. NEW: use SQLSync. fast, client & server stay in sync, don't have to "build your own database" what you're describing just seems like number 1, right?

> 1. always refetch data. always in sync, but needs a server request, so it's slow Except you're not fetching data in htmx, you're fetching hypertext . This is an important distinction. With a JS app you fetch data, integrate it into some local data structure that is then rendered into some form that generates HTML that the browser then renders. With htmx, all of those steps happen in the (faster) server, and the cli…

i get that, and it's a good distinction, but it's not the same as removing a server/network request

Re: Accidental database programming

#290

Post by asaddhamani »

egeozcan wrote: post #275
BackBlast wrote: post #265

Earlier quoted context omitted.

I've used Meteor. I thought it was a good system. It didn't have offline capability, at least not back when I used it. It really needed to be connected to work. But conceptually, yes, it had a very similar system.

If Meteor could scale, we'd probably hear about it way more these days. I remember having problems with 200 users serving from my above average dev pc for testing internal tools. It's a DX dream though.

Yeah, faced the scaling challenges. What was recommended was using the "methods" which was basically RPC to fetch the data, rather than using the PubSub data syncing. Meteor maintained a full copy of what data each client had in memory, so this made it use a lot of RAM, and it did some kind of diffing for these on each update to only send the required updates over the wire, so this made it have high CPU utilisation. I had to move the chat portion of my app over to SocketIO which scaled wonderfully, but the DX was not quite as nice.

Rails and Phoenix are doing something similar (kind of) these days, and those seem to scale better, you just move all the data heavy lifting to the server and do almost nothing on the client.

Post reply on HN