Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hosting SQLite Databases on GitHub Pages

phiresky.netlify.app

Post reply on HN
Print view View original

71–80 of 82 posts

Re: Hosting SQLite Databases on GitHub Pages

#71

Post by devwastaken »

earthboundkid wrote: post #59
devwastaken wrote: post #49

Huh, this is a funny one. I had this idea a long time ago when doing some napkin design of a "static wiki". Problem was the querying didn't fit how software optimizes content delivery, so millions of people requesting from a single database would most likely be difficult to accomplish in a performant manner. Secondarily writing to said database would of course be impossible because locking, and you'd need a server an…

> People with older phones (which is many) wouldn't be able to use it at all. Caniuse.com does not agree with this assertion.

web statistics aren't a good reflection of use cases. It assumes that the percentage of people using browsers are using browsers, when often people whom are using an important service, namely a government service, are not often using other services.

Re: Hosting SQLite Databases on GitHub Pages

#72

Post by Hackbraten »

schwartzworld wrote: post #62
Hackbraten wrote: post #55

Earlier quoted context omitted.

MagSafe has had only two variants while it existed (2006–2017). That’s more than a decade. Both MacBook Pros I’ve owned at that time each came with its power supply included, and lasted me seven years each. I consider myself frugal, not a sucker.

If you're buying apple products still, you're telling on yourself.

An alternative explanation would be: people have individual preferences, and value is a subjective concept.

Re: Hosting SQLite Databases on GitHub Pages

#73

Post by zinekeller »

pwdisswordfish8 wrote: post #65
zinekeller wrote: post #61

Earlier quoted context omitted.

Browsers actually implement range-requests correctly, what Netlify has done is advertise support for it but send data incorrectly.

That doesn't change the thrust of the comment. If you're trying to work around spotty support and detecting the feature is "tricky", then change the program so it performs a small power-on self-test against a known dataset. (And what's the point of crafting a comment in this tone? Is it supposed to be a retort? Whether or not Netlify is doing the wrong thing, if it works in Chrome, but not in Firefox, then that's a m…

Because both Chromium and Gecko follow the IETF (RFC)/W3C specs about this, what Netlify is doing is plain out-of-spec, so what Chromiun and Gecko are doing are implementation details that is explicitly marked as "okay, if you encounter a stupid server that is somehow explicitly advertising range-request support but does it incorrectly, you can do anything and you're still compliant". Drop the request (like Firefox)? Compliant. Silently trim (like Chrome)? Still compliant. Just give zeroes matching the announced length? Yes, still compliant even if you think that's stupid. If you didn't get this simple fact (that is easily verifiable by opening your favourite browser's devtools or even in Fiddler), I don't know how you're not getting this. Yes, it's using heuristics, but Netlify announces support for range-requests so there's no heuristics here to do, either Netlify must remove the header announcing support or Netlify fixes this problem before we talk about heuristics.

Re: Hosting SQLite Databases on GitHub Pages

#74

Post by pwdisswordfish8 »

zinekeller wrote: post #73
pwdisswordfish8 wrote: post #65

Earlier quoted context omitted.

That doesn't change the thrust of the comment. If you're trying to work around spotty support and detecting the feature is "tricky", then change the program so it performs a small power-on self-test against a known dataset. (And what's the point of crafting a comment in this tone? Is it supposed to be a retort? Whether or not Netlify is doing the wrong thing, if it works in Chrome, but not in Firefox, then that's a m…

Because both Chromium and Gecko follow the IETF (RFC)/W3C specs about this, what Netlify is doing is plain out-of-spec, so what Chromiun and Gecko are doing are implementation details that is explicitly marked as "okay, if you encounter a stupid server that is somehow explicitly advertising range-request support but does it incorrectly, you can do anything and you're still compliant". Drop the request (like Firefox)?…

What do you think any of that has to do with anything? Whether Blink and Gecko are compliant or not was not an issue that was raised, outside of your attempt to raise it.

Whether Netlify is doing the wrong thing or not is irrelevant. The original comment was motivated only by the claim that it works in Chrome but not Firefox, and it was an attempt to address that issue. Identifying Netlify as the culprit doesn't make the issue go away any more than gesturing at the criminal statutes solves the problem of someone who is the victim of domestic abuse. Imagine they're being offered guidance on how to enter their home, collect their belongings, and then extract themselves from the situation as cleanly as they can without suffering further violence. Your entire line of reasoning here is tantamount interrupting conversation so you can shout the other person down for how stupid they are while announcing that the plan is unnecessary on the basis that it would be illegal for the victim's partner to give them a thrashing. Yeah, no shit, but once again, totally irrelevant. You're wielding a quiver full of oughts in a conversation about ises.

> If you didn't get this simple fact (that is easily verifiable by opening your favourite browser's devtools or even in Fiddler), I don't know how you're not getting this.

Reality check: it's you who completely failed to track the salience of the message that you chose to respond to, jerk. Get a fucking clue.

Re: Hosting SQLite Databases on GitHub Pages

#75

Post by noduerme »

goeiedaggoeie wrote: post #45
noduerme wrote: post #36

Earlier quoted context omitted.

Yeah but a key value system lacks all the really good things about SQL. And locally with sqlite you don't really have to worry about latency, so you should be able to just get atomicity and consistency on the thread. This shouldn't be a lot to ask from an embedded web DB. I think as with other standards bickering, ten years from now something (Canvas API) will come out that more or less replicates the technology that…

>Yeah but a key value system lacks all the really good things about SQL exactly >And locally with sqlite you don't really have to worry about latency, so you should be able to just get atomicity and consistency on the thread for high performance read you don't need much more. > think as with other standards bickering, ten years from now something (Canvas API) will come out that more or less replicates the technology…

It's really only the write / ACID features of WebSQL that are interesting to me, though. There are lots of ways to get a high performance read on key/value pairs if you've already downloaded all the data. We don't need a new methodology for that. What's interesting is the ability for web app storage to respect foreign key constraints and transactions within a local data model, without having to rely on rolling your own middleware to resolve inconsistencies.

Re: Hosting SQLite Databases on GitHub Pages

#76

Post by cafxx »

westurner wrote: post #68
ticklemyelmo wrote: post #44

Earlier quoted context omitted.

I think that's the core innovation here, smart HTTP block storage. I wonder if there has been any research into optimizing all http range requests at the client level in a similar way. i.e. considering the history of requests on a particular url and doing the same predictive exponential requests, or grabbing the full file asynchronously at a certain point.

> Methods for remotely accessing/paging data in from a client when a complete download of the dataset is unnecessary: > - Query e.g. parquet on e.g. GitHub with DuckDB: duckdb/test_parquet_remote.test https://github.com/duckdb/duckdb/blob/6c7c9805fdf1604039ebed... > - Query sqlite on e.g. GitHub with SQLite: [Hosting SQLite databases on Github Pages - (or any static file hoster) - phiresky's blog](...) >> The above q…

> Would BitTorrent be faster over HTTP/3 (UDP) or is that already a thing for web seeding?

The BT protocol itself runs on both TCP and UDP, but it has preferred the UDP variant for many years already.

Re: Hosting SQLite Databases on GitHub Pages

#77

Post by CRConrad »

goeiedaggoeie wrote: post #26

SQLite not being in browsers instead of indexdb saddens me today still. I designed a system 15 years ago that released dimensional star schemas for specific reports as sqllite databases into Adobe Air (or whatever the prerelease name was) for a large retailer in the UK. We would query the data warehouse, build the sqlite db file (I can't remember the exact db sizes but they weren't too big - 15mb or so) and the compu…

I've been thinking about inventing exactly that for the last week or so.

Re: Hosting SQLite Databases on GitHub Pages

#78

Post by CRConrad »

noduerme wrote: post #36
justsomeuser wrote: post #33

Earlier quoted context omitted.

It was deprecated because it was difficult to write a standard spec for the existing SQLite code (but a key value system is much easier to specify as there is no SQL language).

Yeah but a key value system lacks all the really good things about SQL. And locally with sqlite you don't really have to worry about latency, so you should be able to just get atomicity and consistency on the thread. This shouldn't be a lot to ask from an embedded web DB. I think as with other standards bickering, ten years from now something (Canvas API) will come out that more or less replicates the technology that…

Whoever that asshole was, he's received sooo much undeserved praise. There is pretty much nothing good about sliced bread.

Re: Hosting SQLite Databases on GitHub Pages

#80

Post by westurner »

cafxx wrote: post #76
westurner wrote: post #68

Earlier quoted context omitted.

> Methods for remotely accessing/paging data in from a client when a complete download of the dataset is unnecessary: > - Query e.g. parquet on e.g. GitHub with DuckDB: duckdb/test_parquet_remote.test https://github.com/duckdb/duckdb/blob/6c7c9805fdf1604039ebed... > - Query sqlite on e.g. GitHub with SQLite: [Hosting SQLite databases on Github Pages - (or any static file hoster) - phiresky's blog](...) >> The above q…

> Would BitTorrent be faster over HTTP/3 (UDP) or is that already a thing for web seeding? The BT protocol itself runs on both TCP and UDP, but it has preferred the UDP variant for many years already.

Thanks. There likely are relative advantages to HTTP/3 QUIC. Here's this from Wikipedia:

> Both HTTP/1.1 and HTTP/2 use TCP as their transport. HTTP/3 uses QUIC, a transport layer network protocol which uses user space congestion control over the User Datagram Protocol (UDP). The switch to QUIC aims to fix a major problem of HTTP/2 called "head-of-line blocking": because the parallel nature of HTTP/2's multiplexing is not visible to TCP's loss recovery mechanisms, a lost or reordered packet causes all active transactions to experience a stall regardless of whether that transaction was impacted by the lost packet. Because QUIC provides native multiplexing, lost packets only impact the streams where data has been lost.

And HTTP Pipelining / Multiplexing isn't specified by just UDP or QUIC:

> HTTP/1.1 specification requires servers to respond to pipelined requests correctly, sending back non-pipelined but valid responses even if server does not support HTTP pipelining. Despite this requirement, many legacy HTTP/1.1 servers do not support pipelining correctly, forcing most HTTP clients to not use HTTP pipelining in practice.

> Time diagram of non-pipelined vs. pipelined connection The technique was superseded by multiplexing via HTTP/2,[2] which is supported by most modern browsers.[3]

> In HTTP/3, the multiplexing is accomplished through the new underlying QUIC transport protocol, which replaces TCP. This further reduces loading time, as there is no head-of-line blocking anymore https://en.wikipedia.org/wiki/HTTP_pipelining

Post reply on HN