A future for SQL on the web • Hacker News Board

Re: A future for SQL on the web

#211

Post by dmitriid » Fri, Aug 13, 2021, 12:55 PM UTC

jeswin wrote: post #205

dmitriid wrote: post #201

Earlier quoted context omitted.

> I'm saying nobody uses those Chrome only APIs, until they become standards Ah yes. Once Chrome releases something in stable, literally no one ever uses those APIs, no one. And even Google's propaganda machine doesn't tell you to use them (example: https://web.dev/usb/ ) > Or are you against experimentation? I'm not. What Google does isn't experimentation.

My assumption is that a very small percentage of developers participate in it. Feel free to prove me wrong if you have any numbers. > I'm not. What Google does isn't experimentation. So the way this is done now (with origin trials to collect feedback from developers) is not good enough? What exactly is your definition of experimentation?

> My assumption is that a very small percentage of developers participate in it.

It doesn't really matter how many. It means that people are already using this functionality, and depend on it. I didn't link to an article on removing alert just for the fun of it.

> So the way this is done now (with origin trials to collect feedback from developers) is not good enough? What exactly is your definition of experimentation?

Running this as origin trial is a very good way of doing it. What's not a good way is:

- having an unrealistic timeline.

Example: Mozilla was asked for position on WebHID three months before Chrome released in stable. The "standard" was so badly written that Mozilla engineers couldn't understand it. Chrome still released it, and updated the "standard" two months after it was already in the wild.

- ignoring any and all objections, and favoring own needs only

Example: Constructible Stylesheets. The spec has a trivially reproducible problem. Several developers (not only from Mozilla and Safari) pointed this out, and proposed several changes to the spec that would get rid of the problem. However, Google's own lit-html was interested in the spec. So they released it in stable, said that "0.8% of pageviews now use this" (reality: only lit-html used it at the time), and refused to hide it back under a flag. Safari simply said they are not going to implement the spec in this shape.

- completely ignoring the realities and needs of the web

See https://dev.to/richharris/stay-alert-d and my comment here: https://dev.to/dmitriid/comment/1h5bh

- presenting these "standards" as fait accompli even if other browser vendors will never ever implement them

They have co-opted web.dev as a propaganda machine. The site presents itself as a neutral resource for web develoeprs. It's not, it's a Chrome-only vehicle. They present various specs as already available and never specify issues with those specs. For example, WebUSB: https://web.dev/usb/

See, e.g., Mozilla's position on various specs. Scroll down to "harmful": https://mozilla.github.io/standards-positions/ I'll let you guess how many of those have already been shipped in Chrome. Then you can find what web.dev says about it

- gaslighting other browser vendors

I won't link to specific tweets because I don't need to be angry. But almost every single of Google's high visibility "standards people" and "developer advocates" and "community managers" will always, and I mean absolutely always paint all other browsers in as negative light as possible (as lagging behind, as hurting the web, as damaging the web, as being a detriment to the web etc.)

Re: A future for SQL on the web

#212

Post by scns » Fri, Aug 13, 2021, 1:51 PM UTC

adam12 wrote: post #47

lioeters wrote: post #32

> browsers may delete your IndexedDB database under certain conditions Safari will happily delete your IndexedDB database after 7 days of inactivity. It deletes "all of a website’s script-writable storage after seven days of Safari use without user interaction on the site". That includes: - Indexed DB - LocalStorage - Media keys - SessionStorage - Service Worker registrations and cache Source: https://webkit.org/blog…

Tim Cook basically lied to Congress when he stated that developers can create web apps as an alternative to using the app store. Edit: In order for this to be true, Apple (at the very least) needs to enable push notifications and an install prompt for progressive web apps.

I love the truth and ain't a Apple fanboi by any means, but this is not a lie to me.

Can you run a PWA on an iPhone? Yes, you can even pin them to the start screen to keep data long term, as others have pointed out.

Are you able to use all features other browsers enable? No, only the ones Apple allows.

Re: A future for SQL on the web

#213

Post by scns » Fri, Aug 13, 2021, 2:08 PM UTC

Opening this, on desktop, left me speechless with an open mouth: https://archive.jlongster.com/

You can explore it with the mouse cursor.

Re: A future for SQL on the web

#214

Post by scns » Fri, Aug 13, 2021, 2:23 PM UTC

dang wrote: post #123

jlongster wrote: post #55

Earlier quoted context omitted.

Read harder. https://jlongster.com/future-sql-web#more-than-just-another-...

Hey, can you please not do this ("Did you read the post?", "Read harder", etc.), even when someone else hasn't read an article? I understand how frustrating it can be when people don't read what you write very closely (believe me, I understand), but it's one of the tropes that degrade discussion and we're trying to avoid sinking to that level here. " Please don't comment on whether someone read an article. "Did you e…

Thank you dang, for your tireless work, maintaining a space where you can respectfully with each other.

Re: A future for SQL on the web

#215

Post by bayesian_horse » Fri, Aug 13, 2021, 2:51 PM UTC

EvanAnderson wrote: post #4

This is funny and sad to me. We had SQLite in the browser[0]. I only did a little bit of work with it but it seemed actually pretty nice. It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale"). There was the whole excuse that the specification was "whatever SQLite does" and, therefore, not suitable for being a standard. There would be worse things than SQLite upon which to base a stand…

I think security had more to do with it...

Re: A future for SQL on the web

#216

Post by dangerface » Fri, Aug 13, 2021, 3:13 PM UTC

daleharvey wrote: post #10

EvanAnderson wrote: post #4

This is funny and sad to me. We had SQLite in the browser[0]. I only did a little bit of work with it but it seemed actually pretty nice. It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale"). There was the whole excuse that the specification was "whatever SQLite does" and, therefore, not suitable for being a standard. There would be worse things than SQLite upon which to base a stand…

The excuse was that a standard needs to have multiple implementations otherwise we are standardising implementation details and bugs. Hindsight shows that was entirely correct, as SQLite bugs were then found that could be exploited directly via WebSQL, Firefox of course was not vunerable. ( https://hub.packtpub.com/an-sqlite-magellan-rce-vulnerabilit... ) As a sidenote, I worked a lot with the WebSQL API and it was n…

Better to have something thats already considered standard than to invent a new one because you think you can do better than sqlite.

Re: A future for SQL on the web

#217

Post by dmitriid » Fri, Aug 13, 2021, 6:39 PM UTC

scns wrote: post #206

dmitriid wrote: post #189

Earlier quoted context omitted.

> Very few people are using Chrome-only APIs which are not in the standards yet. So it's not really a concern. Ah yes. But SQlite not having competing independent implementations somehow is? Also, "not many people using something" is not as great an argument as you think it is. See, for example, the latest problem with browsers deciding to remove alert/prompt/confirm: https://dev.to/richharris/stay-alert-d > The web…

From the first comment below the article: > The Safari team did quite a lot of work re-styling their alerts and dialogs to be within the context of the webpage, yet phishing and scams that utilize this still run rampant on iOS. Repeated alerts are used to lock up the browser and make it unusable, forcing non-technical users to call a scam telephone number because they think their device was hacked. A few bad actors r…

> A few bad actors ruining it even for totally different usecases.

Same thing happened to Do Not Track... that was used for fingerprinting

Re: A future for SQL on the web

#218

Post by jlongster » Fri, Aug 13, 2021, 6:55 PM UTC

jitl wrote: post #207

This is great to see, and a project I considered attempting myself for a bit. I’m excited to test it out. @jlongster I have a question about this: > The backend calls it [Atomics.wait] to wait on the result from the worker and blocks until it’s done. Does this mean the main (UI) thread is blocked during queries? Or are there more threads, like UI SQLite main SQLite FS backend? ———— At Notion, we’ve used IndexedDB for…

> Does this mean the main (UI) thread is blocked during queries? Or are there more threads, like UI SQLite main SQLite FS backend?

The latter! Your app running queries must be on a worker, and then the IDB backend will spawn another worker. `Atomics.wait` is not even available on the main thread.

Ideally in the future, there will be a better storage API that we don't even need all the Atomics silly-ness (hopefully it provides Sync methods)

That's really cool re: Notion! That's exactly the kind of thing I want too: a way to just build apps the same way everywhere, on mobile/desktop/web.

You are right about various issues, and I personally don't have to worry much about it on my app because I have native mobile apps and I don't support the web version on mobile. I intentionally do that -- the mobile web is just too broken in too many ways. My impression is the IDB is more stable on desktop, but because mobile is more memory sensitive there are more issues there.

However, you should try it out! I definitely discovered a lot of weird things; I definitely was able to get Safari into a weird state the required a complete app restart. Here's the thing though: I found ways around them. If you do a lot of read requests in a certain way, Safari will lock up permanently. However, if you make sure to wait until the `readonly` transaction is finished before starting a new one, the problem goes away. I was able to reliably reproduce that problem and it went away with that fix.

I think absurd-sql is so promising because it normalizes the patterns of how IDB is accessed, and it already includes fixes for a bunch of edge cases. There are probably more, but try it out! If you run into an edge case, we can tweak the IDB backend until it works. We can paper over these issues in the underlying backend and you don't have to worry about it because you aren't directly managing IDB read/writes.

Re: A future for SQL on the web

#219

Post by EvanAnderson » Fri, Aug 13, 2021, 7:56 PM UTC

roca wrote: post #187

EvanAnderson wrote: post #154

Earlier quoted context omitted.

My "web scale" bit is snark. The "because it was SQL based" comes from the feeling I got at the time that Mozilla absolutely wouldn't accept anything was SQL-based. I strongly believe anti-SQL attitudes at Mozilla, and little else, are what killed WebSQL. This is also how I justify my opinion that the "no independent implementations" was more of an afterthought excuse, and less of a primary motivation on Mozilla's pa…

I have no doubt that Jonas told the truth that that's the feedback he got. I was never aware of Jonas or anyone else involved at Mozilla having some kind of "anti-SQL" prejudice. WebDatabase, with or without a spec, boiled down to shipping SQLite. We didn't want the Web to depend on that. It might even have required shipping a specific version of SQLite to make sure that query planning matches other browsers. It seem…

I don't think Jonas was dishonest in any way. I think Mozilla talked to front-end Javascript devs who came from a "NoSQL" background (after all, Javascript devs are what begat "web scale" MongoDB) and who had disdain for SQL. I think that disdain sealed the fate of any SQL-based API in HTML5, irrespective of the involvement of SQLlite.

The "no independent implementations" was certainly a convenient way to dismiss away the already-implemented SQLlite-based solution. It's a valid argument for rejecting SQLlite as the spec, for sure. Implementations notwithstanding it seems like Mozilla was unwilling to even consider anything SQL-based.

The same tone comes out in the "Beyond HTML5: Database APIs and the Road to IndexedDB" Mozilla blog post[0]:

> In order to really get Web SQL Database right, we’d have to first start with defining a meaningful subset of SQL for web applications. Why define a whole other language, when more elegant solutions exist within JavaScript itself?

...

> We were resolved that using strings representing SQL commands lacked the elegance of a "web native" JavaScript API ...

That sounds like the same rhetoric used in the early MongoDB / NoSQL craze days to dismiss RDMS, ACID, JOINs, etc. Never mind that SQL is a battle-hardened DSL that has proven itself to be suitable to the task-- it wasn't "elegant" enough for the front-end devs Mozilla surveyed.

As an aside, I also find the quote from that post very amusing:

> In another article, we compare IndexedDB with Web SQL Database, and note that the former provides much syntactic simplicity over the latter.

The example for a JOIN using IndexedDB in the referenced article[1] looks strained. It really doesn't look like it has "syntactic simplicity" compared to the WebSQL version above it.

[0] https://hacks.mozilla.org/2010/06/beyond-html5-database-apis...

[1] https://hacks.mozilla.org/2010/06/comparing-indexeddb-and-we...

Re: A future for SQL on the web

#220

Post by roca » Sat, Aug 14, 2021, 12:24 AM UTC

TeMPOraL wrote: post #198

roca wrote: post #187

Earlier quoted context omitted.

I have no doubt that Jonas told the truth that that's the feedback he got. I was never aware of Jonas or anyone else involved at Mozilla having some kind of "anti-SQL" prejudice. WebDatabase, with or without a spec, boiled down to shipping SQLite. We didn't want the Web to depend on that. It might even have required shipping a specific version of SQLite to make sure that query planning matches other browsers. It seem…

> WebDatabase, with or without a spec, boiled down to shipping SQLite. We didn't want the Web to depend on that. Ironically, all the popular browsers already ship SQLite - they just not expose it directly. So, in a way, the Web already depends on SQLite. So does a lot of other technology - we're talking about one of the, if not the , most widely distributed pieces of software on the planet.

That's true, but "in a way" does a lot of work there. "In a way" the Web already depends on the C++ standard library, but we aren't exposing that as a JS API.