A future for SQL on the web • Hacker News Board

Re: A future for SQL on the web

#81

Post by tarsinge » Thu, Aug 12, 2021, 6:18 PM UTC

derefr wrote: post #49

lioeters wrote: post #32

> browsers may delete your IndexedDB database under certain conditions Safari will happily delete your IndexedDB database after 7 days of inactivity. It deletes "all of a website’s script-writable storage after seven days of Safari use without user interaction on the site". That includes: - Indexed DB - LocalStorage - Media keys - SessionStorage - Service Worker registrations and cache Source: https://webkit.org/blog…

I would point out that the article you linked mentions that webapps "added to the home screen" on iOS Mobile Safari are exempt from this garbage-collection process. In such a case, the "add to the home screen" step is being taken as the user expressing the explicit desire to have that data around indefinitely. Which seems sensible to me; webapps should be treated as ephemeral-by-default, and only be allowed persisten…

PWA are not a standard, they are a Google concept pushed through Chrome (OS). I’m fine with Apple never supporting it.

Re: A future for SQL on the web

#82

Post by wffurr » Thu, Aug 12, 2021, 6:20 PM UTC

eatonphil wrote: post #14

sql.js is pretty hard to use as is otherwise you run out of memory really quickly. I was trying to use it as the in-memory SQL flavor for an open source data ide [0] but my naive approach of `SELECT * FROM VALUES (...), ...` would run out of memory after only a few hundred rows. I ended up switching to https://github.com/agershun/alasql which could handle up to 80MB of data or so. (I haven't yet tested on larger data…

Emscripten wasm binaries with memory growth enabled can use up to 2 GB heap. That's very surprising that you're hitting a memory limit.

Re: A future for SQL on the web

#83

Post by SahAssar » Thu, Aug 12, 2021, 6:21 PM UTC

mschuster91 wrote: post #79

SahAssar wrote: post #27

Earlier quoted context omitted.

The justification was more that the standardization process requires at least 2 independent implementations, and nobody was working on rewriting SQLite from scratch. I would have loved WebSQL, but it is reasonable to require multiple implementations for full standardization.

> and nobody was working on rewriting SQLite from scratch. One could also have embedded a trimmed-down PostgreSQL or MariaDB into browsers.

I don't know the reasoning being not doing it, but I'd guess it's related to those DB engines not being geared or interested in embedded use-cases and that no SQL engine seems to entirely agree with any other how to handle certain parts of SQL.

Re: A future for SQL on the web

#84

Post by arthurcolle » Thu, Aug 12, 2021, 6:21 PM UTC

derefr wrote: post #49

lioeters wrote: post #32

> browsers may delete your IndexedDB database under certain conditions Safari will happily delete your IndexedDB database after 7 days of inactivity. It deletes "all of a website’s script-writable storage after seven days of Safari use without user interaction on the site". That includes: - Indexed DB - LocalStorage - Media keys - SessionStorage - Service Worker registrations and cache Source: https://webkit.org/blog…

I would point out that the article you linked mentions that webapps "added to the home screen" on iOS Mobile Safari are exempt from this garbage-collection process. In such a case, the "add to the home screen" step is being taken as the user expressing the explicit desire to have that data around indefinitely. Which seems sensible to me; webapps should be treated as ephemeral-by-default, and only be allowed persisten…

I wish these bookmarks extended to other browsers on iOS. It's already crazy they can't ship custom rendering engines, but basically this in and of itself could be considered anticompetitive (only Safari bookmarks can be exempted from this GC process) lol!

Re: A future for SQL on the web

#85

Post by hitekker » Thu, Aug 12, 2021, 6:23 PM UTC

daleharvey wrote: post #40

hitekker wrote: post #31

Earlier quoted context omitted.

I view the "standards" argument as a red herring for building a NoSQL db in the browser. Which, to this day, is slow, buggy and requires third party libraries to be usable [1] For those who are able to stomach an uncomfortable political history instead of an easy, technical answer, you can take a look at [2]. It's interesting that 7 years later, many the folks who pushed hard to get rid of SQL in favor of NoSQL seem…

I am familiar with Nolans article, I created PouchDB (the project he is discussing), you seem to have misred the post as it discusses the technical nuance and tradeoffs involved in the decision at many points entirely agreeing with the position against WebSQL. While Nolan came to the a different conclusion than I did (a point he made in the post) he laid out challenges very well and made it very clear there was no ob…

Edit: Nolan replied before and corrected me. I've removed my misinterpretation and kept my main point below.

Back in the day, there were people who strongly suggested MongoDB and IndexedDB were the future, and that PostgreSQL, MySQL, SQLite were trash. I've noticed the folks who rode that hype-train moved into other kinds of occupations that aren't exactly engineering-focused anymore.

Re: A future for SQL on the web

#86

Post by tarsinge » Thu, Aug 12, 2021, 6:24 PM UTC

lioeters wrote: post #32

> browsers may delete your IndexedDB database under certain conditions Safari will happily delete your IndexedDB database after 7 days of inactivity. It deletes "all of a website’s script-writable storage after seven days of Safari use without user interaction on the site". That includes: - Indexed DB - LocalStorage - Media keys - SessionStorage - Service Worker registrations and cache Source: https://webkit.org/blog…

As a Safari user I’m really happy with that behavior, it avoids the hassle of manually purging it regularly.

Also in my opinion websites have no business storing data on my device through a browser. If I want that behavior I would use a native app.

Re: A future for SQL on the web

#87

Post by mschuster91 » Thu, Aug 12, 2021, 6:28 PM UTC

SahAssar wrote: post #83

mschuster91 wrote: post #79

Earlier quoted context omitted.

> and nobody was working on rewriting SQLite from scratch. One could also have embedded a trimmed-down PostgreSQL or MariaDB into browsers.

I don't know the reasoning being not doing it, but I'd guess it's related to those DB engines not being geared or interested in embedded use-cases and that no SQL engine seems to entirely agree with any other how to handle certain parts of SQL.

Indeed, it would have been a lot of work trimming down any of the major FOSS RDBMS towards embedding them in a browser.

Regarding the SQL dialect - that could have been handled by specifying a standard SQL dialect that's then dynamically translated to the target engine's dialetc.

Re: A future for SQL on the web

#88

Post by apavlo » Thu, Aug 12, 2021, 6:29 PM UTC

zzzeek wrote: post #43

EvanAnderson wrote: post #4

This is funny and sad to me. We had SQLite in the browser[0]. I only did a little bit of work with it but it seemed actually pretty nice. It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale"). There was the whole excuse that the specification was "whatever SQLite does" and, therefore, not suitable for being a standard. There would be worse things than SQLite upon which to base a stand…

first thought, was there a "standard" for key/value stores? mongodb the reference implementation? overall, if you looked at HN like five years ago, every DB headline was key/value, mongodb, maybe some cassandra / couchdb, links to the "web scale" cartoon. these days, it's SQL SQL SQL, with a heavy dose of SQLite and PostgreSQL. SQL survived the key/value fad despite the nebulousness of a workable "standard" (yes ther…

> SQL survived the key/value fad

SQL has survived every fad since the 1970s:

Stonebraker "What Goes Around Comes Around"

https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/pape...

Re: A future for SQL on the web

#89

Post by tehbeard » Thu, Aug 12, 2021, 6:29 PM UTC

hitekker wrote: post #31

daleharvey wrote: post #10

Earlier quoted context omitted.

The excuse was that a standard needs to have multiple implementations otherwise we are standardising implementation details and bugs. Hindsight shows that was entirely correct, as SQLite bugs were then found that could be exploited directly via WebSQL, Firefox of course was not vunerable. ( https://hub.packtpub.com/an-sqlite-magellan-rce-vulnerabilit... ) As a sidenote, I worked a lot with the WebSQL API and it was n…

I view the "standards" argument as a red herring for building a NoSQL db in the browser. Which, to this day, is slow, buggy and requires third party libraries to be usable [1] For those who are able to stomach an uncomfortable political history instead of an easy, technical answer, you can take a look at [2]. It's interesting that 7 years later, many the folks who pushed hard to get rid of SQL in favor of NoSQL seem…

Having worked with IndexedDB quite a bit at my job, I can level three criticisms at IndexedDB.

1. The API is the dogshit hot mess you'd expect for a pre promise/async API.

2. The lack of partial/computed secondary indexes.

3. Apple/Safari does EVERYTHING in their power to break it. I refuse to believe it's incompetence at this point, it's actively malicious.

Re: A future for SQL on the web

#90

Post by busymom0 » Thu, Aug 12, 2021, 6:29 PM UTC

cosmie wrote: post #59

adam12 wrote: post #47

Earlier quoted context omitted.

Tim Cook basically lied to Congress when he stated that developers can create web apps as an alternative to using the app store. Edit: In order for this to be true, Apple (at the very least) needs to enable push notifications and an install prompt for progressive web apps.

PWA are explicitly exempt from the data storage wipe behavior[1]. So there's that, at least. For now. Although I agree that it's a fairly minor win in the grand scheme of how handicapped PWA's are on iOS. And like you said, the lack of install prompts and tucking away the PWA "installation" option in the share menu makes it less than intuitive and requires manual awareness efforts by devs[2]. [1] Last section of http…

That’s only half true. PWA are exempt from data storage wipe behaviour only if they have been added to the Home Screen. And as your second article points out:

> It’s important to ensure that our iOS visitor is using Safari because iOS doesn’t permit other browsers to install our awesome PWA’s!

So if the user is using Firefox or Chrome etc browser, they can’t do it. And since all browsers have to use WKWebView, the limitation applies to all browsers. Apple has basically crippled other browsers with this anti competitive behaviour.