A future for SQL on the web • Hacker News Board

Re: A future for SQL on the web

#161

Post by SilverRed » Fri, Aug 13, 2021, 1:20 AM UTC

daleharvey wrote: post #10

EvanAnderson wrote: post #4

This is funny and sad to me. We had SQLite in the browser[0]. I only did a little bit of work with it but it seemed actually pretty nice. It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale"). There was the whole excuse that the specification was "whatever SQLite does" and, therefore, not suitable for being a standard. There would be worse things than SQLite upon which to base a stand…

The excuse was that a standard needs to have multiple implementations otherwise we are standardising implementation details and bugs. Hindsight shows that was entirely correct, as SQLite bugs were then found that could be exploited directly via WebSQL, Firefox of course was not vunerable. ( https://hub.packtpub.com/an-sqlite-magellan-rce-vulnerabilit... ) As a sidenote, I worked a lot with the WebSQL API and it was n…

My feeling is that the web apis try to do too much. Why do we need a webSQL api. Why do we not just let websites create a file and then they can provide whatever kind of library they want. They could package a WASM version of sqlite and just work like they would as a desktop app.

That way you never have to deal with browser incompatibility or unchangeable specifications.

Re: A future for SQL on the web

#162

Post by DangitBobby » Fri, Aug 13, 2021, 1:21 AM UTC

ofrzeta wrote: post #109

"In all browsers except Chrome, IndexedDB is implemented using SQLite". That's a strange way to phrase the status quo. That is Firefox and ... Opera? While Chrome includes Edge.

It would include Safari, Firefox, IE11, and Edge (before it became Chrome Skin) as well right?

Re: A future for SQL on the web

#163

Post by SilverRed » Fri, Aug 13, 2021, 1:22 AM UTC

devwastaken wrote: post #12

EvanAnderson wrote: post #4

This is funny and sad to me. We had SQLite in the browser[0]. I only did a little bit of work with it but it seemed actually pretty nice. It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale"). There was the whole excuse that the specification was "whatever SQLite does" and, therefore, not suitable for being a standard. There would be worse things than SQLite upon which to base a stand…

No, SQLite was not safe to be used with arbitrary queries. There were multiple memory vulnerabilities that allowed escape of the browser sandbox. https://www.sqlite.org/cves.html

Would this not be solvable by just making it all arbitary WASM code? As long as the simple WASM sandbox is secure, it doesn't matter how buggy your internal SQLite implementation is.

Re: A future for SQL on the web

#164

Post by SilverRed » Fri, Aug 13, 2021, 1:24 AM UTC

jbergens wrote: post #139

zzzeek wrote: post #43

Earlier quoted context omitted.

first thought, was there a "standard" for key/value stores? mongodb the reference implementation? overall, if you looked at HN like five years ago, every DB headline was key/value, mongodb, maybe some cassandra / couchdb, links to the "web scale" cartoon. these days, it's SQL SQL SQL, with a heavy dose of SQLite and PostgreSQL. SQL survived the key/value fad despite the nebulousness of a workable "standard" (yes ther…

I think sql is a bad fit for front end code in js. We may have ended up with another fight over ORMs. The Vietnam war of computer science as I think it was called.

How else would you suggest full applications run in the browser. Desktop apps make use of SQLite all the time even when they are mostly server backed.

An example is that Telegram maintains a local db which stores the most recently accessed content so you can scroll through past messages and media without having to request them every time you view them.

Re: A future for SQL on the web

#165

Post by outworlder » Fri, Aug 13, 2021, 1:24 AM UTC

bob1029 wrote: post #58

> SQLite, even though it’s implemented on top of IndexedDB, easily beats out IndexedDB in every single performance metric. The absurdity! This really is quite incredible. Same idea extends to your filesystem too. Tracking millions of 1KB objects on disk? You could load the whole set into memory substantially faster from SQLite using the same disk. If WAL is enabled with reasonable sync flags, the same applies going b…

There are two pieces of tech that punch way above their weight: Lua and SQLite

So many apps could have just saved to a .db file instead of coming up with their own binary format. So many others could have just embedded a tiny, easily sandboxed, Lua library and added scripting languages, rather than embedding larger ones (or worse yet, implementing their own scripts).

Re: A future for SQL on the web

#166

Post by outworlder » Fri, Aug 13, 2021, 1:25 AM UTC

rektide wrote: post #11

James is one of the world's great techno-adventurers, & getting to para-socially share in wild adventures like this makes living on Spaceship Earth more lovely & lively! James has also done cool projects like sweet.js macros, helped kick off Firefox devtool's transition to react (iirc), oh and lead the basically industry standard JS formatter Priettier project. I'm forgetting a dozen other things over the years but i…

I remember him from the Gambit Scheme days.

https://archive.jlongster.com/Open-Sourcing-My-Gambit-Scheme...

Re: A future for SQL on the web

#167

Post by warvstar » Fri, Aug 13, 2021, 1:26 AM UTC

lioeters wrote: post #32

> browsers may delete your IndexedDB database under certain conditions Safari will happily delete your IndexedDB database after 7 days of inactivity. It deletes "all of a website’s script-writable storage after seven days of Safari use without user interaction on the site". That includes: - Indexed DB - LocalStorage - Media keys - SessionStorage - Service Worker registrations and cache Source: https://webkit.org/blog…

I assume that also includes the Cache API's Storage?

Re: A future for SQL on the web

#168

Post by SilverRed » Fri, Aug 13, 2021, 1:27 AM UTC

enkrs wrote: post #53

adam12 wrote: post #47

Earlier quoted context omitted.

Tim Cook basically lied to Congress when he stated that developers can create web apps as an alternative to using the app store. Edit: In order for this to be true, Apple (at the very least) needs to enable push notifications and an install prompt for progressive web apps.

If you add the website to homescreen (make it an “app”) then the 7 day storage limit does not apply. This makes total sense to me, and I’m happy Facebook can’t store tracking data in my Safari for longer than 7 days.

Clearing site data makes total sense. Otherwise after a year of web usage you have multiple GB of stored website data that the user does not know what it is being used for or how to clear it.

And the majority of uses for local data is to cache fetched data or store temporary preferences not worthy of being in the server side database.

Re: A future for SQL on the web

#169

Post by warvstar » Fri, Aug 13, 2021, 1:28 AM UTC

stevage wrote: post #132

Fascinating. I'm really curious what the use case is that so many people seem to have. Why do you need so much data in the browser, and to be doing queries and data manipulation there? Where does the data come from? Don't you need to sync it back to a server somewhere?

We have a gaming platform where users can download and play full games in the browser. We don't use indexedDB though, we use the Cache API.

Re: A future for SQL on the web

#170

Post by mr_toad » Fri, Aug 13, 2021, 1:30 AM UTC

lioeters wrote: post #32

> browsers may delete your IndexedDB database under certain conditions Safari will happily delete your IndexedDB database after 7 days of inactivity. It deletes "all of a website’s script-writable storage after seven days of Safari use without user interaction on the site". That includes: - Indexed DB - LocalStorage - Media keys - SessionStorage - Service Worker registrations and cache Source: https://webkit.org/blog…

If webSQL were a thing, that list would include webSQL data too.