Similar to Single-page applications (SPA), single-table application (STA) might become a thing. Just a shard a table on multiple keys and serve the shards as static files, provided that the data is Ok to share, similar to sharing static html content.
Show HN: 22 GB of Hacker News in SQLite
31–40 of 229 posts
Re: Show HN: 22 GB of Hacker News in SQLite
#32Don't miss how this works. It's not a server-side application - this code runs entirely in your browser using SQLite compiled to WASM, but rather than fetching a full 22GB database it instead uses a clever hack that retrieves just "shards" of the SQLite database needed for the page you are viewing. I watched it in the browser network panel and saw it fetch: https://hackerbook.dosaygo.com/static-shards/shard_1636.sqli…
Vfs support is amazing.
Re: Show HN: 22 GB of Hacker News in SQLite
#33The link seems to be down, was it taken down?
Probably just forgot to make it public.
Re: Show HN: 22 GB of Hacker News in SQLite
#34Earlier quoted context omitted.
Why do you say that?
Because anything that even slightly differs from the standard American phrasing of something must be "LLM generated" these days.
Sometimes I want to write more creatively, but then worry I’ll be accused of being an LLM. So I dumb it down. Remove the colorful language. Conform.
Re: Show HN: 22 GB of Hacker News in SQLite
#35That's pretty neat! I did something similar. I build a tool[1] to import the Project Arctic Shift dumps[2] of reddit into sqlite. It was mostly an exercise to experiment with Rust and SQLite (HN's two favorite topics). If you don't build a FTS5 index and import without WAL (--unsafe-mode), import of every reddit comment and submission takes a bit over 24 hours and produces a ~10TB DB. SQLite offers a lot of cool json…
You could check out SQLite's auto_vacuum which reclaims space without rebuilding the entire db https://sqlite.org/pragma.html#pragma_auto_vacuum
Re: Show HN: 22 GB of Hacker News in SQLite
#36What a reminder on how text is so much more efficient than video, its crazy! Could you imagine the same amount of knowledge (or dribble) but in video form? I wonder how large that would be.
Average high quality 1080p60 video has bitrate of 5Mbps, which is equivalent to 120k English words per second. With average English speech being 150wpm, we end up with text being 50 thousand times more space efficient.
Converting 22GB of uncompressed text into video essay lands us at ~1PB or 1000TB.
Re: Show HN: 22 GB of Hacker News in SQLite
#37[flagged]
> I'm really sorry to have to ask this, but this really feels like you had an LLM write it?
Ending a sentence with a question mark doesn’t automatically make your sentence a question. You didn’t ask anything. You stated an opinion and followed it with a question mark.
If you intended to ask if the text was written by AI, no, you don’t have to ask that.
I am so damn tired of the “that didn’t happen” and the “AI did that” people when there is zero evidence of either being true.
These people are the most exhausting people I have ever encountered in my entire life.
Re: Show HN: 22 GB of Hacker News in SQLite
#38[flagged]
I doubt it. "hacker news" spelled lowercase? comma after "beauty"? missing "in" after "it's"? i doubt an LLM would make such syntax mistakes. it's just good writing, that's also possible these days.
Re: Show HN: 22 GB of Hacker News in SQLite
#39Similar to Single-page applications (SPA), single-table application (STA) might become a thing. Just a shard a table on multiple keys and serve the shards as static files, provided that the data is Ok to share, similar to sharing static html content.
do you mean single database? it'd be quite hard if not impossible to make applications using a single table (no relations). reddit did it though, they have a huge table of "things" iirc.
Re: Show HN: 22 GB of Hacker News in SQLite
#40Earlier quoted context omitted.
Because anything that even slightly differs from the standard American phrasing of something must be "LLM generated" these days.
With the em dashes I see you. But at this point idrc so long as it reads well. Everyone uses spell check…
I add em dashes to everything I write now, solely to throw people who look for them off. Lots of editors add them automatically when you have two sequential dashes between words — a common occurrence, like that one. And this is is Chrome on iOS doing it automatically.
Ooh, I used “sequential”, ooh, I used an em dash. ZOMG AI IS COMING FOR US ALL