Live data from Hacker News

Show HN: 22 GB of Hacker News in SQLite

hackerbook.dosaygo.com

211–220 of 229 posts

Re: Show HN: 22 GB of Hacker News in SQLite

#211
post #22

Don't miss how this works. It's not a server-side application - this code runs entirely in your browser using SQLite compiled to WASM, but rather than fetching a full 22GB database it instead uses a clever hack that retrieves just "shards" of the SQLite database needed for the page you are viewing. I watched it in the browser network panel and saw it fetch: https://hackerbook.dosaygo.com/static-shards/shard_1636.sqli…

Is it possible to implement search this way?

Re: Show HN: 22 GB of Hacker News in SQLite

#212
post #24

What a reminder on how text is so much more efficient than video, its crazy! Could you imagine the same amount of knowledge (or dribble) but in video form? I wonder how large that would be.

Average high quality 1080p60 video has bitrate of 5Mbps, which is equivalent to 120k English words per second. With average English speech being 150wpm, we end up with text being 50 thousand times more space efficient. Converting 22GB of uncompressed text into video essay lands us at ~1PB or 1000TB.

Cheers to you for doing the math, I hope 2026 is excellent for you!

Re: Show HN: 22 GB of Hacker News in SQLite

#214

Earlier quoted context omitted.

So cool! Would it be impossible fro me to use them on the Archive stats page ( https://hackerbook.dosaygo.com/?view=archive ) ? If you're okay with that any links/credit line details? Totally cool if not, just super interesting!

yes, please, that'd be cool! you can link my site grimridge.net if you'd like. here are the numbers so you can plot graphs that fit the site's style: mean (all time): https://katb.in/yutupojerux mean (since 2020): https://katb.in/omoyibisava median (all time): https://katb.in/kilopofivet median (since 2020): https://katb.in/ukefetuyuhi

Ok i will put them in the next update! Thank you

Re: Show HN: 22 GB of Hacker News in SQLite

#215

Earlier quoted context omitted.

Really? Let me know how I can help. What would you like to see fixed?

Just following the ordinary guidelines when doing responsive designs, like increasing the text size and sizes of buttons and inputs, so my fat fingers don't missklick every other try. HN has gotten better, but is still below average, hence why I thought it was some kind of aesthetic choice.

I tried to keep to HN look. But I’m not a mobile design expert. I will give it a once over see if i can make it better for you

Re: Show HN: 22 GB of Hacker News in SQLite

#216

Earlier quoted context omitted.

Your page is served over sqlitevfs with Range queries? Let's try this here.

I did a similar VFS in Go. It doesn't run client-side in a browser. But you can use it (e.g.) in a small VPS to access a multi-TB database directly from S3.

That is cool. Maybe i look at the go code

Re: Show HN: 22 GB of Hacker News in SQLite

#218
post #202

Earlier quoted context omitted.

A JSON parser is less than a thousand lines of code.

And where most of CPU time will be wasted in, if you care about profiling/improving responsiveness.

At that point you're just io bound, no? I can easily parse json at 100+GB/s on commodity hardware, but I'm gonna have a much harder time actually delivering that much data to parse.

Re: Show HN: 22 GB of Hacker News in SQLite

#219

Earlier quoted context omitted.

So cool! Would it be impossible fro me to use them on the Archive stats page ( https://hackerbook.dosaygo.com/?view=archive ) ? If you're okay with that any links/credit line details? Totally cool if not, just super interesting!

yes, please, that'd be cool! you can link my site grimridge.net if you'd like. here are the numbers so you can plot graphs that fit the site's style: mean (all time): https://katb.in/yutupojerux mean (since 2020): https://katb.in/omoyibisava median (all time): https://katb.in/kilopofivet median (since 2020): https://katb.in/ukefetuyuhi

Added in the latest: https://hackerbook.dosaygo.com/?view=archive

Re: Show HN: 22 GB of Hacker News in SQLite

#220

Earlier quoted context omitted.

what dumb phone do you use? and why do you want wikipedia in your pocket, but not a smartphone? where do you draw the line? (doing a lot of work in that area, so i am asking to learn from someone who might think alike)

I use the Mudita Kompakt specifically cause it allows sideloading so I can still have a few extras. Right now I have Kiwix and Libby. It works really well. I have a $10 a month plan from US cellular with only 2gigs so I try to keep everything offline that I can. Honestly it's mostly the news... so I draw the line at browser, I'll never install a browser, that's basically something I can do when I sit down at a PC. I…

interesting. thank you. any way i can reach out to you regardibg a project i am working on?

your input would be very valuable.

Post reply on HN