This seems cool, but doesn't Fossil (from the SQLite author) already do it "out of the box" ?
Hosting SQLite databases on GitHub Pages or any static file hoster
11–20 of 252 posts
Re: Hosting SQLite databases on GitHub Pages or any static file hoster
#12Incredibly clever use of paging and http range queries! Well done.
Re: Hosting SQLite databases on GitHub Pages or any static file hoster
#13This seems cool, but doesn't Fossil (from the SQLite author) already do it "out of the box" ? https://www.fossil-scm.org/home/doc/trunk/www/index.wiki
Fossil runs a server application.
Re: Hosting SQLite databases on GitHub Pages or any static file hoster
#14Over high latency links this would be virtually unusable. Why not just download the entire database into memory over XHR on page load? SQLite databases of pure data usually aren’t over 10MB in size.
Re: Hosting SQLite databases on GitHub Pages or any static file hoster
#15Modeling the DOM in SQL... Further evidence that anything we can imagine has some stable representation in third normal form. Is it fast? Maybe not. But is it correct? Provably.
Re: Hosting SQLite databases on GitHub Pages or any static file hoster
#16This is easily the most clever web programming hack I’ve seen this year. Bravo. I had seen this used for video or audio of course but it never occurred to me you could use it for databases. There are probably a ton of other formats this is good for too.
Re: Hosting SQLite databases on GitHub Pages or any static file hoster
#17Amazing, especially - for me - that the FTS5 full-text search just works. Longer term, I am if it were possible to split the DB code into read and write parts and cross-compile only read part for delivery to the browser.
Re: Hosting SQLite databases on GitHub Pages or any static file hoster
#18This is awesome. Got my wheels turning :)
Re: Hosting SQLite databases on GitHub Pages or any static file hoster
#19In the genomics world, Tabix indices enables similar use cases. An ordered TSV file is compressed in chunks (bgzip) and a Tabix index created to allow range based access by mapping from the index -> chunk. This allows a browser based genome browser zoomed into a section of the genome to fetch information from a multi gigabyte file.
Re: Hosting SQLite databases on GitHub Pages or any static file hoster
#20The innovation here is getting sql.js to use http and range requests for file access rather than all being in memory. I wonder when people using next.js will start using this for faster builds for larger static sites?
See also https://github.com/bittorrent/sqltorrent , same trick but using BitTorrent
Yeah, that was one of the inspirations for this. That one does not work in the browser though, would be a good project to do that same thing but with sqlite in wasm and integrated with WebTorrent instead of a native torrent program.
I actually did also implement a similar thing fetching data on demand from WebTorrent (and in turn helping to host the data yourself by being on the website): https://phiresky.github.io/tv-show-ratings/ That uses a protobufs split into a hashmap instead of SQLite though.