Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accidental database programming

sqlsync.dev

Post reply on HN
Print view View original

101–110 of 310 posts

Re: Accidental database programming

#101

Post by carlsverre »

tegmarks wrote: post #96
carlsverre wrote: post #93

Author here. Finally got through most of the questions, will keep checking periodically for ones I've missed (has someone built a better way to track HN discussions?). I'm very happy to see the discussion thus far. This first post focused on the parts of frontend engineering that motivated me to create SQLSync rather than specifically how SQLSync works. I'll be addressing that in my next post. Thanks for all the grea…

Thankyou for the very informative article, I appreciate the irony using a post titled "Stop building databases" to announce a new database :-)

You're welcome! And I'm glad you enjoyed it. Once I thought of that title I had to use it.

My only saving grace is that technically I didn't make a new DB - just using good ol SQLite. Mostly. :)

Re: Accidental database programming

#102

Post by roenxi »

There is an interaction here between the "what gets measured gets managed" principle and the sunk cost fallacy.

The problem with databases is actually complexity. Any individual feature is more or less safe, but around the time reliability, caching and indexes get matched together there is a complexity explosion and it doesn't (normally, anyhow) make sense to implement a domain-specific DB (call is a DSD?).

But, around the time a company has invested in implementing those 3 features and discovered that it has sunk a lot of resources into the DSD, is politically averse to recommending it be stripped out and there is a high real cost to taking out the tech debt in one go.

Really the problem here is SQL's syntax. If using a basic relational database was a pleasant experience that involved some familiar C-like syntax instead of broken English people would be more tempted to go with a DB instead of rolling their own. The NoSQL databases were a good step in that direction, but then they by and large overfocused on big data instead of everyday usefulness. Things like Redis took hold which is nice.

Making it easy to run SQL is a reasonable approach, but the problem is that the good databases - I like postgres - are SQL native and it is hard to get efficiency without speaking the DB's language. We really need a PostgresPostSQL database that is a perfect postgres clone but primary parser supports a language with good syntax.

Re: Accidental database programming

#103

Post by hnlmorg »

fjcp wrote: post #70
nusmella wrote: post #65

An old company I worked for used project management software with a check-in/out mechanism for making changes. When you "check out" a project it downloads a copy that you change locally, then "check in" uploads it back to the server. A project is "locked" while in the "checked out" state. We all felt it was an archaic mechanism in a word of live updating apps. After 10 years of building SPA "web apps", that data sync…

Looks very similar to JEDI [0], an early Delphi VCS system that worked that way. It gave us the tranquility to know that no conflict would appear, as only one developer could work with a locked/checked out file at a time. There was no merge those days. In contrast, files that were frequently changed in every task would always cause a blocking between developers. [0] https://jedivcs.sourceforge.net/

There were loads of VCSs that operated this way. And I don’t miss them one bit.

Re: Accidental database programming

#104

Post by throwaheyy »

ethbr1 wrote: post #88
RHSeeger wrote: post #86

Earlier quoted context omitted.

Sounds like RCS [1]. I remember, back when a company I worked for switched from RCS to CVS, one of my coworkers was annoyed that CVS didn't support locking checkouts. [1] https://en.wikipedia.org/wiki/Revision_Control_System [2] https://en.wikipedia.org/wiki/Concurrent_Versions_System

And, of course, the default mode of Microsoft Team Foundation Server [0], decades after there were better patterns. So many forgotten locks from lazy devs... [0] https://en.m.wikipedia.org/wiki/Azure_DevOps_Server#TFVC

Are you sure? My experience of using TFVC was that it would warn you if someone else had opened the file for editing but would not actually lock it. Multiple people could edit the same file concurrently with standard automerging/conflict resolution afterwards.

Re: Accidental database programming

#105

Post by msie »

0xbadcafebee wrote: post #20

Anyone remember when "frontend applications" were actual applications and not web pages? I'm willing to bet we have reached that point where new devs literally do not remember that time. There comes a time [..] where we [..] need to cache data from an API. It might start off benign – storing a previous page of data for that instant back button experience, implementing a bit of undo logic, or merging some state from d…

What I find annoying is the still-existing problem that sometimes apps don't load properly and you have to refresh the browser. You don't get this with desktop apps. There are some caching capabilities in browsers but they are not being used by anyone to cache app code and resources. If I'm using an app for the first time it should properly load all code and resources or else report an error.

Re: Accidental database programming

#106

Post by vemv »

threatofrain wrote: post #85
vemv wrote: post #62

This seems to be one of those problems that entirely disappears by ditching SPAs. Using solutions from the Hotwire or htmx family would mean that a query is just a server query - making those fast is a better-understood problem.

This isn't a problem of only websites. Should mobile and desktop ecosystems start making a big move for thin-client like the browser? Should a simple app like Apple Reminders or Google Tasks have the GUI pause if there are delays or connection issues?

Read-only access for coarse-grained pages (as opposed to building a fine-grained ad-hoc DB) seems something reasonable (and easy) to cache for any kind of frontend.

That would allow offline viewing to a variety of apps, regardless of approach.

Last time I checked Google Docs doesn't primarily allow editing offline files, which hints how hard it is to support substantial features beyond mere reading.

Re: Accidental database programming

#107

Post by nemo44x »

ethbr1 wrote: post #88
RHSeeger wrote: post #86

Earlier quoted context omitted.

Sounds like RCS [1]. I remember, back when a company I worked for switched from RCS to CVS, one of my coworkers was annoyed that CVS didn't support locking checkouts. [1] https://en.wikipedia.org/wiki/Revision_Control_System [2] https://en.wikipedia.org/wiki/Concurrent_Versions_System

And, of course, the default mode of Microsoft Team Foundation Server [0], decades after there were better patterns. So many forgotten locks from lazy devs... [0] https://en.m.wikipedia.org/wiki/Azure_DevOps_Server#TFVC

Back in the early days of TFS I was briefly at a company that went all in on MS tools. TFS was used and to avoid the lock each developer had a clone made and after checking their clone in the “TFS Guy” in the office would merge it. He also had to merge things when later checking had conflicting changes.

Now, the best part of this shit show was they had ~30 different customers and each of these customers had a clone of the main thing that would be customized. So the “TFS Guy” had to determine if to keep in the customer clone only or to propagate to the main and then to all the other clones!

Needless to say the “TFS Guy” made a lot of money.

Re: Accidental database programming

#108

Post by carlsverre »

goleary wrote: post #92
carlsverre wrote: post #68

Earlier quoted context omitted.

Indeed it's a very hot space! So exciting to see all the different approaches. ElectricSQL and PowerSync are both tackling the very hard problem of partial replication. The idea is to build a general solution which allows a traditional centralized db to bidirectionally sync only what's needed on the client side - while still supporting optimistic mutations (and all the consistency/conflict stuff that goes along with…

>But consider a personal finance app. The main goal is cross device sync, cloud backup, offline capable, etc. In this case having the entire db stored on every device is probably what you want. A bit confused by this. If I'm a developer of a PFM, I don't want anything but a single user's financial data synced to their device. This sounds like partial replication to me.

Precisely. In the SQLSync model - every user would have a private database just for their data. For example, this is how the todo list demo works: https://sqlsync-todo.pages.dev

(Note: currently SQLSync's server tier doesn't support auth, just random 128bit ids. Auth will come as it matures - but don't use this for anything super secure at the moment).

Re: Accidental database programming

#109

Post by msie »

0xbadcafebee wrote: post #20

Anyone remember when "frontend applications" were actual applications and not web pages? I'm willing to bet we have reached that point where new devs literally do not remember that time. There comes a time [..] where we [..] need to cache data from an API. It might start off benign – storing a previous page of data for that instant back button experience, implementing a bit of undo logic, or merging some state from d…

I won't forget that someone at Google didn't have the courage to enable Dart in Chrome as a successor to Javascript. And someone killed SQLLite as a in-browser db.

Re: Accidental database programming

#110

Post by ozim »

Isn’t this idea something like couch-db? Then there is pouch-db which is browser implementation using local storage.

So nothing new but it is not bad not to be first. Maybe it is bad not knowing prior work and writing up your idea like it is something no one ever thought earlier about ;)

Post reply on HN