Live data from Hacker News

A future for SQL on the web

jlongster.com

191–200 of 227 posts

Re: A future for SQL on the web

#191
post #51

This is funny and sad to me. We had SQLite in the browser[0]. I only did a little bit of work with it but it seemed actually pretty nice. It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale"). There was the whole excuse that the specification was "whatever SQLite does" and, therefore, not suitable for being a standard. There would be worse things than SQLite upon which to base a stand…

I think the current state is fine. You ship your WASM-blob of SQLite, which has the exact bug-compatible version of SQLite that you've tested your app against. The browsers are not burdened with maintaining a huge API surface that can "break the web". Otherwise you'd have to deal with different versions of SQLite in different browsers, most likely outdated, with many options turned off. SQLite is full of quirks and g…

[deleted]

Re: A future for SQL on the web

#192
post #99

Earlier quoted context omitted.

Edit: Nolan replied before and corrected me. I've removed my misinterpretation and kept my main point below. Back in the day, there were people who strongly suggested MongoDB and IndexedDB were the future, and that PostgreSQL, MySQL, SQLite were trash. I've noticed the folks who rode that hype-train moved into other kinds of occupations that aren't exactly engineering-focused anymore.

I wrote that article 7 years ago, and FWIW I would side more with Dale these days. It's probably a good thing we didn't just slap a half-baked API on top of SQLite and call it a web standard. The biggest problem is that yeah, WebSQL tends to be faster than IndexedDB. Or at least it was back when I was working on PouchDB. Biggest issue IIRC was that joins were faster in SQLite than implementing the same thing in userl…

> Consensus & Standardization

> Firefox: Negative [1]

> Safari: Negative [2]

However, I fully expect Chrome to ship it in stable sometime soon, like they do with dozens of other APIs.

There are now four different half-baked storage/file api proposals, at least one of them is already in stable Chrome... it's a mess.

[1] https://github.com/mozilla/standards-positions/issues/481

[2] https://lists.webkit.org/pipermail/webkit-dev/2021-February/...

Re: A future for SQL on the web

#193
post #11

James is one of the world's great techno-adventurers, & getting to para-socially share in wild adventures like this makes living on Spaceship Earth more lovely & lively! James has also done cool projects like sweet.js macros, helped kick off Firefox devtool's transition to react (iirc), oh and lead the basically industry standard JS formatter Priettier project. I'm forgetting a dozen other things over the years but i…

> Just a heads-up, the File System Access API is underway in Chrome, which potentially removes nearly all of the absurdity here.

It doesn't. Because there are now 4 different file access api proposals. At least one of them is already in Chrome (File System Access). [1]

Both Safari and Mozilla are unwilling to implement any of the four until the mess is cleaned up, and there's a single coherent proposal.

Chrome, of course, doesn't care. Storage Foundation API in addition to File System Access they already implemented.

[1] https://github.com/WICG/storage-foundation-api-explainer/iss...

Re: A future for SQL on the web

#194

Earlier quoted context omitted.

I think its fine for a web app to be multiple MB. Websites like HN/wikipedia/blogs have no need for an SQL database but if you are loading something like a full IM client, it makes sense to download a few MB to make it stable over all browsers.

Disagreed, this is a dangerous line of thinking that leads to using technologies like gwt, Vaadin, Blazor, Flutter or others that try to turn the browser into something that it's not suited to. Sure, there could be cases where you have absolutely no alternatives to do something really specific, but in those cases I'd first invite you to reconsider whether what you're attempting to do actually needs to be a web app. O…

I subscribe to an alternative premise - "The browser is effectively an OS and thus a standardized distribution target". There are cases where the people I want to distribute to don't suffer from the constraints you mentioned. e.g. will tolerate a 20 second page load in exchange for the experience my product will provide.

Being able to reach all devices and have a super fast experience without consuming bandwidth is always good, but not always a high priority .

Re: A future for SQL on the web

#195

Earlier quoted context omitted.

By that you mean several piles in different languages? Or one pile for each customer?

One for each. Our solution compiles to a single executable.

Would you not have unique code per customer regardless of if the code is SQL queries or C#?

Re: A future for SQL on the web

#196

Earlier quoted context omitted.

The excuse was that a standard needs to have multiple implementations otherwise we are standardising implementation details and bugs. Hindsight shows that was entirely correct, as SQLite bugs were then found that could be exploited directly via WebSQL, Firefox of course was not vunerable. ( https://hub.packtpub.com/an-sqlite-magellan-rce-vulnerabilit... ) As a sidenote, I worked a lot with the WebSQL API and it was n…

My feeling is that the web apis try to do too much. Why do we need a webSQL api. Why do we not just let websites create a file and then they can provide whatever kind of library they want. They could package a WASM version of sqlite and just work like they would as a desktop app. That way you never have to deal with browser incompatibility or unchangeable specifications.

I'd agree with some of the existing web APIs. But I think some web SQL API should be built-in.

Why? Because it's a very common pattern. Almost fundamental. Take any project more complex than a toy calculator, and you'll quickly find places where the authors are hand-rolling relational operations.

Does your app have an array of records in memory, which it then searches for values, filters by conditions, and/or sorts? I'd give it a 50/50 chance that if you replaced that array with an in-memory SQLite table, and all operations on it with SQL queries on it, the result would be less code, more readable code and better performance[0].

It's not just the web - I'd argue that programming languages in general should all embed first-class in-memory relational database engines. It should be a part of the language standard - if it's easy to write this:

  struct Foo {
    int bar;
    date baz;
    string quux;
  };

  //somewhere in something
  Array foos;
it should be easy to write this:

  table foos;
(or some other, better way of describing intended data access patterns)

and get a thing that can be efficiently queried, filtered and transformed along dimensions specified, with the compiler turning your request into efficient bytecode/nativecode. Doesn't mean you have to write queries in vanilla SQL - there are more composable syntaxes out there. But arbitrary loops and map-reduces aren't that good either.

When you start looking at your data processing code as database operations, you'll see it everywhere. That map/reduce/zip blob? That's a JOIN query. That struct you're keeping in your event loop, that looks like:

  struct Schedule {
    Array tasks;
    PQueue toRun;
  };
that priority queue is an index on the tasks array, and looking at its front is just SELECT id FROM tasks ORDER BY priority LIMIT 1;. That Entity-Component-System pattern you're using in your videogame? That's literally a relational database. It was conceived of as such (and for massively multiplayer games, is often implemented as such). Wouldn't it be nice if the ability to express this came built into the language? With a column-oriented option for improved performance, too?

--

[0] - It's trivial to add indexes to SQL table. The engine will maintain them for you. Nobody habitually adds indexes to their own regular variables. They, along with the code to maintain them, would manifest as extreme code bloat. Meanwhile, in-memory SQLite is freakishly fast. If you're measuring performance in Python/Ruby units, you won't even notice the FFI overhead.

Re: A future for SQL on the web

#197

Earlier quoted context omitted.

Seems like a great way to give every site a multiple MB dependency.

I think its fine for a web app to be multiple MB. Websites like HN/wikipedia/blogs have no need for an SQL database but if you are loading something like a full IM client, it makes sense to download a few MB to make it stable over all browsers.

Funny, how this is the static/shared library problem again :). And where the hivemind seems to prefer static libraries for regular software, it's reverse on the web.

I suppose the problem is, on the web, you're redownloading the whole bundle each time you run. There are layers of caching in the browser that are supposed to help, but they're almost always defeated by a combination of app misconfiguration and developers' reluctance to use the cache given the ridiculous rate of redeployment of web software.

I'd still argue that SQL API is a prime candidate for a shared system-level library - AKA browser built-in. The concepts behind it have been worked out in the 1970s and honed through decades since. It's not experimental tech. The API won't be changing every month.

Re: A future for SQL on the web

#198
post #187

Earlier quoted context omitted.

My "web scale" bit is snark. The "because it was SQL based" comes from the feeling I got at the time that Mozilla absolutely wouldn't accept anything was SQL-based. I strongly believe anti-SQL attitudes at Mozilla, and little else, are what killed WebSQL. This is also how I justify my opinion that the "no independent implementations" was more of an afterthought excuse, and less of a primary motivation on Mozilla's pa…

I have no doubt that Jonas told the truth that that's the feedback he got. I was never aware of Jonas or anyone else involved at Mozilla having some kind of "anti-SQL" prejudice. WebDatabase, with or without a spec, boiled down to shipping SQLite. We didn't want the Web to depend on that. It might even have required shipping a specific version of SQLite to make sure that query planning matches other browsers. It seem…

> WebDatabase, with or without a spec, boiled down to shipping SQLite. We didn't want the Web to depend on that.

Ironically, all the popular browsers already ship SQLite - they just not expose it directly. So, in a way, the Web already depends on SQLite. So does a lot of other technology - we're talking about one of the, if not the, most widely distributed pieces of software on the planet.

Re: A future for SQL on the web

#199
post #183

Earlier quoted context omitted.

> And yet we're are now at a point where Chrome rams its own APIs through standards bodies, and there are no (and often won't be) any independent competing implementations. Very few people are using Chrome-only APIs which are not in the standards yet. So it's not really a concern. But otherwise, Chrome really has pushed the web forward more than any other browser. If it hadn't, native (and walled garden style) app st…

> Very few people are using Chrome-only APIs which are not in the standards yet. So it's not really a concern. Ah yes. But SQlite not having competing independent implementations somehow is? Also, "not many people using something" is not as great an argument as you think it is. See, for example, the latest problem with browsers deciding to remove alert/prompt/confirm: https://dev.to/richharris/stay-alert-d > The web…

> Ah yes. But SQlite not having competing independent implementations somehow is?

Except that's not why it was culled. The reasons are discussed in great detail on this thread and elsewhere.

> non-standards that Chrome pushes (many of which will never get a different implementation because both Safari and Mozilla consider them harmful) are good

I'm saying nobody uses those Chrome only APIs, until they become standards (which requires acceptance by other vendors). Or are you against experimentation?

Re: A future for SQL on the web

#200

Earlier quoted context omitted.

Disagreed, this is a dangerous line of thinking that leads to using technologies like gwt, Vaadin, Blazor, Flutter or others that try to turn the browser into something that it's not suited to. Sure, there could be cases where you have absolutely no alternatives to do something really specific, but in those cases I'd first invite you to reconsider whether what you're attempting to do actually needs to be a web app. O…

I subscribe to an alternative premise - "The browser is effectively an OS and thus a standardized distribution target". There are cases where the people I want to distribute to don't suffer from the constraints you mentioned. e.g. will tolerate a 20 second page load in exchange for the experience my product will provide. Being able to reach all devices and have a super fast experience without consuming bandwidth is a…

That is a valid premise indeed, but one that i cannot agree with for a plethora of reasons. There are articles that go into more detail about what this approach leads to: https://idlewords.com/talks/website_obesity.htm

There are also aspects which are probably entirely overlooked because of that approach, for example, consider:

  - https://whatdoesmysitecost.com/test/210813_AiDcEB_e62852d10cb795009ccd61f1dd2d8623#gniCost
  - https://whatdoesmysitecost.com/test/210813_AiDcPZ_dd4b52c10c37e4c7483ab391015a00cb#gniCost
Sometimes, there are also accessibility concerns (especially in the case of Flutter), all of which will work against letting your software be used by as many people as possible, in as many hardware and software configurations as possible.

If the target audience of your products and sites are wealthy and able bodied individuals in 1st world countries, then by all means go ahead. However, in certain other pursuits inclusivity is definitely an important goal as well and one that i personally value a bit higher than fancy looking sites with rich functionality.

That said, isn't 20 seconds a bit much? Sure, i get that it's just an example, but i think even heavier pages like GMail load in approximately half of that. I think that bundle splitting and some optimization are probably a good idea even in heavier pieces of software!

Post reply on HN