Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A future for SQL on the web

jlongster.com

Post reply on HN
Print view View original

91–100 of 227 posts

Re: A future for SQL on the web

#91

Post by regularfry »

411111111111111 wrote: post #62
kall wrote: post #22

Earlier quoted context omitted.

Yeah I think so too. It‘s a cliche by now to say these things go in cycles but they do. If we like this part of the cycle, let‘s ride it and let SQL get hyped, even though its been around forever and never went away.

What gives you the confidence that it's going on circles? I cannot see our industry repeating the mantra that NoSQL solves all our persistence problems ever again. There are use cases for them, but not everything is a good fit and I'm pretty sure we found that out as an industry. I'm sure there are always going to be individuals that will claim that they're better at everything, but that doesn't really mean anything.…

The cycles are longer than you're imagining. Relational database folk have been railing against the incursions of one generation of non-relational database type or another since at least the 1980's.

Re: A future for SQL on the web

#92

Post by mst »

tarsinge wrote: post #81
derefr wrote: post #49

Earlier quoted context omitted.

I would point out that the article you linked mentions that webapps "added to the home screen" on iOS Mobile Safari are exempt from this garbage-collection process. In such a case, the "add to the home screen" step is being taken as the user expressing the explicit desire to have that data around indefinitely. Which seems sensible to me; webapps should be treated as ephemeral-by-default, and only be allowed persisten…

PWA are not a standard, they are a Google concept pushed through Chrome (OS). I’m fine with Apple never supporting it.

So long as I can also use them in firefox they're 'unstandardised but not single vendor' and at that point I'm not as troubled about it.

Re: A future for SQL on the web

#93

Post by aseipp »

coliveira wrote: post #71
SahAssar wrote: post #27

Earlier quoted context omitted.

The justification was more that the standardization process requires at least 2 independent implementations, and nobody was working on rewriting SQLite from scratch. I would have loved WebSQL, but it is reasonable to require multiple implementations for full standardization.

I don't buy this reasoning because they could have standardized only the interface, letting it open for developers to choose SQLite or some other future implementation. There is no need to standardize SQLite behavior, including bugs.

What actually happens in that scenario is that one group now writes an adapter on top of SQLite to make it "standardized" to whatever interface you design, increasing the complexity and scope for bugs. To be clear: you're literally reimplementing something that SQLite already does, except probably much worse, in the name of the "standard."

Then that adapter is carbon copied by everyone, because again, nobody is going to reimplement an 80kLOC SQL database as well as whatever 10k LOC parser/lexer/typechecking adapter someone wrote, if they can avoid it. Then everyone just uses that forever, and you're back to square 1, using one implementation everywhere, which is the exact situation standards are supposed to avoid anyway.

The working group was correct to reject a "compromise" like that because that's never how it works out in practice; it's a submarine suggestion from the start. And a big part of this is all because, as evidenced by numerous responses in this thread, modern computer programmers seem to value their own immediate satisfaction and time over literally every other potential concern, no matter how significant.

Re: A future for SQL on the web

#94

Post by mst »

jlongster wrote: post #46
lioeters wrote: post #32

> browsers may delete your IndexedDB database under certain conditions Safari will happily delete your IndexedDB database after 7 days of inactivity. It deletes "all of a website’s script-writable storage after seven days of Safari use without user interaction on the site". That includes: - Indexed DB - LocalStorage - Media keys - SessionStorage - Service Worker registrations and cache Source: https://webkit.org/blog…

Yep, this is the biggest problem (although I haven't seen it happen after 7 days, at least on desktop). We will provide a new backend for the Storage Foundation API when it's available.

Clearly the solution to this is to keep a query log in an extra table and periodically stream that to the server as a form of logical replication (plus perhaps being able to load the initial database state from the server side as well, maybe even on-demand using the GH pages trickery until a write forces materialisation into IndexedDB).

As a bonus point this effectively adds yet another level of "Yo, Dawg" which I can't not love just as a matter of principle.

Re: A future for SQL on the web

#95

Post by jlongster »

eatonphil wrote: post #14

sql.js is pretty hard to use as is otherwise you run out of memory really quickly. I was trying to use it as the in-memory SQL flavor for an open source data ide [0] but my naive approach of `SELECT * FROM VALUES (...), ...` would run out of memory after only a few hundred rows. I ended up switching to https://github.com/agershun/alasql which could handle up to 80MB of data or so. (I haven't yet tested on larger data…

Something sounds wrong with your setup. I've had no problems!

Re: A future for SQL on the web

#96

Post by bob1029 »

justsomeuser wrote: post #72
bob1029 wrote: post #58

> SQLite, even though it’s implemented on top of IndexedDB, easily beats out IndexedDB in every single performance metric. The absurdity! This really is quite incredible. Same idea extends to your filesystem too. Tracking millions of 1KB objects on disk? You could load the whole set into memory substantially faster from SQLite using the same disk. If WAL is enabled with reasonable sync flags, the same applies going b…

Nice. What kind of business logic are you using SQL queries for?

Any sort of decision point that tends to vary between our customers. We are getting really tired of maintaining custom code piles.

Re: A future for SQL on the web

#97

Post by vlunkr »

lioeters wrote: post #32

> browsers may delete your IndexedDB database under certain conditions Safari will happily delete your IndexedDB database after 7 days of inactivity. It deletes "all of a website’s script-writable storage after seven days of Safari use without user interaction on the site". That includes: - Indexed DB - LocalStorage - Media keys - SessionStorage - Service Worker registrations and cache Source: https://webkit.org/blog…

This is why I haven't used LocalStorage or any other browser storage solution for a long time. They seem really cool, but if they will just get wiped by some browsers, then what is the point? As of a couple of years ago, localStorage didn't work at all in a private windows in safari. Unless you have some very specific needs, I don't know how you would every use IndexedDB

Re: A future for SQL on the web

#98

Post by ricardobeat »

arthurcolle wrote: post #84
derefr wrote: post #49

Earlier quoted context omitted.

I would point out that the article you linked mentions that webapps "added to the home screen" on iOS Mobile Safari are exempt from this garbage-collection process. In such a case, the "add to the home screen" step is being taken as the user expressing the explicit desire to have that data around indefinitely. Which seems sensible to me; webapps should be treated as ephemeral-by-default, and only be allowed persisten…

I wish these bookmarks extended to other browsers on iOS. It's already crazy they can't ship custom rendering engines, but basically this in and of itself could be considered anticompetitive (only Safari bookmarks can be exempted from this GC process) lol!

I think all home screen apps run on the system webview, there are no “safari bookmarks” in this context.

Re: A future for SQL on the web

#99

Post by nolanl »

hitekker wrote: post #85
daleharvey wrote: post #40

Earlier quoted context omitted.

I am familiar with Nolans article, I created PouchDB (the project he is discussing), you seem to have misred the post as it discusses the technical nuance and tradeoffs involved in the decision at many points entirely agreeing with the position against WebSQL. While Nolan came to the a different conclusion than I did (a point he made in the post) he laid out challenges very well and made it very clear there was no ob…

Edit: Nolan replied before and corrected me. I've removed my misinterpretation and kept my main point below. Back in the day, there were people who strongly suggested MongoDB and IndexedDB were the future, and that PostgreSQL, MySQL, SQLite were trash. I've noticed the folks who rode that hype-train moved into other kinds of occupations that aren't exactly engineering-focused anymore.

I wrote that article 7 years ago, and FWIW I would side more with Dale these days. It's probably a good thing we didn't just slap a half-baked API on top of SQLite and call it a web standard.

The biggest problem is that yeah, WebSQL tends to be faster than IndexedDB. Or at least it was back when I was working on PouchDB. Biggest issue IIRC was that joins were faster in SQLite than implementing the same thing in userland on top of IndexedDB. Browsers eventually shipped getAll/getAllKeys which also helped with cursor slowness.

I haven't looked much at the Storage Foundation API [1], but it seems like a more reasonable approach moving forward. Just give developers the low-level tools and let them build SQLite on top of it. Also the Chromium devs have been working on relaxed durability, which apparently improves IDB perf in some scenarios [2] (although still not as fast as Firefox it seems [3]).

[1]: https://github.com/WICG/storage-foundation-api-explainer

[2]: https://www.chromestatus.com/feature/5730701489995776

[3]: https://bugs.chromium.org/p/chromium/issues/detail?id=102545...

Re: A future for SQL on the web

#100

Post by matharmin »

EvanAnderson wrote: post #4

This is funny and sad to me. We had SQLite in the browser[0]. I only did a little bit of work with it but it seemed actually pretty nice. It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale"). There was the whole excuse that the specification was "whatever SQLite does" and, therefore, not suitable for being a standard. There would be worse things than SQLite upon which to base a stand…

SQLite as a library is amazing, SQLite as a spec for a standard web API is not.

The issue is not just that there wasn't a competing implementation at the time, it is that you could not feasibly create a competing implementation. The set of features supported by SQLite is _massive_. And even you stick with SQLite, which version? And which extensions? Which features do you remove because they're not safe or not applicable for web?

Alternatively you could start from the ground up and create a new database spec that is similar to SQLite, but more limited in scope. But then it will not be directly compatible with SQLite, and need a translation layer. The browser would likely have to re-implement large parts of SQLite for this to work. And you'd miss out on so much of the functionality that makes SQLite attractive to use.

Arguably that would still have been better than IndexedDB. All that IndexedDB has going for it, is that it is simple to implement. But even then Safari manages to get the implementation horribly broken so often.

What we need instead is to have a low-level storage api that can be used to implement other databases as libraries, including SQLite. The Storage Foundation API, mentioned in the post, might just give us that.

Post reply on HN