Hosting SQLite databases on GitHub Pages or any static file hoster

Post by gitgud » Sun, May 02, 2021, 11:51 PM UTC

city41 wrote: post #44

mvanaltvorst wrote: post #22

Earlier quoted context omitted.

I'm curious, in what manner could this method speed up Next.js builds? That's all done locally, which negates the effect of HTTP range requests, right?

I'm guessing they mean rather than build a static Next site that generates 10k+ pages (or whatever large means in the given context), it instead creates one page that just queries the data from the client. I have one Next static site that has about 20k pages and takes about 20 minutes to build and deploy. I think that's an acceptable build time. But I do know of other people around the net who have mentioned having s…

Post by nl » Sun, May 02, 2021, 11:55 PM UTC

WJW wrote: post #43

simonw wrote: post #41

Pop open the network pane in the browser tools and try running this SQL query for a demo of how clever this is: select country_code, long_name from wdi_country order by rowid desc limit 100 It fetches just 54.2KB of new data (across 49 small HTTP requests) to return 100 results - from a statically hosted database file that's 668.8MB! I have an animated GIF demo of this here: https://twitter.com/simonw/status/13889338…

So assuming no country has a name longer than 98 characters and that all country codes are 2 characters, that is over 500% overhead? Are you missing a /s in your post?

  select sum(length(country_code) + length(long_name)) from wdi_country;

  select sum(length(country_code) + length(long_name))/count(*) from wdi_country;

Post by joshuanapoli » Sun, May 02, 2021, 11:58 PM UTC

rzzzt wrote: post #31

arafalov wrote: post #17

Amazing, especially - for me - that the FTS5 full-text search just works. Longer term, I am if it were possible to split the DB code into read and write parts and cross-compile only read part for delivery to the browser.

If you are interested in full-text search on the client, Lunr is also an option: https://lunrjs.com/docs/index.html

Post by nicoburns » Mon, May 03, 2021, 12:10 AM UTC

gitgud wrote: post #111

city41 wrote: post #44

Earlier quoted context omitted.

I'm guessing they mean rather than build a static Next site that generates 10k+ pages (or whatever large means in the given context), it instead creates one page that just queries the data from the client. I have one Next static site that has about 20k pages and takes about 20 minutes to build and deploy. I think that's an acceptable build time. But I do know of other people around the net who have mentioned having s…

Generating 20k pages in 20mins is impressive, 16 pages a second on average. In my experience, it can take a couple of minutes just to deploy 20 pages, but that could just be the overhead of Typescript and SASS compilation too...

Post by CyberRabbi » Mon, May 03, 2021, 12:11 AM UTC

nl wrote: post #116

CyberRabbi wrote: post #105

Earlier quoted context omitted.

I made two claims: > this would be unusable over high latency links. That is objectively true > SQLite databases of pure data usually aren’t over 10MB in size. No one here has refuted this point. Any other counterargument is addressing a claim I did not make.

It's not anymore unusable over high latency links than most website. Also worth noting that the caching is very smart, so once things are downloaded it's very fast. But most high latency links are very slow (so downloading large databases is a horrible experience) and (more importantly) are often priced by the size of downloads.