Show HN: 22 GB of Hacker News in SQLite

#1

Post by keepamovin » Tue, Dec 30, 2025, 5:01 PM UTC

Community, All the HN belong to you. This is an archive of hacker news that fits in your browser. When I made HN Made of Primes I realized I could probably do this offline sqlite/wasm thing with the whole GBs of archive. The whole dataset. So I tried it, and this is it. Have Hacker News on your device.

Go to this repo (https://github.com/DOSAYGO-STUDIO/HackerBook): you can download it. Big Query -> ETL -> npx serve docs - that's it. 20 years of HN arguments and beauty, can be yours forever. So they'll never die. Ever. It's the unkillable static archive of HN and it's your hands. That's my Year End gift to you all. Thank you for a wonderful year, have happy and wonderful 2026. make something of it.

Show HN: 22 GB of Hacker News in SQLite
hackerbook.dosaygo.com

Re: Show HN: 22 GB of Hacker News in SQLite

#2

Post by [deleted] » Tue, Dec 30, 2025, 5:02 PM UTC

[deleted]

Re: Show HN: 22 GB of Hacker News in SQLite

#3

Post by asdefghyk » Tue, Dec 30, 2025, 5:37 PM UTC

How much space is needed? ...for the data .... Im wondering if it would work on a tablet? ....

Re: Show HN: 22 GB of Hacker News in SQLite

#4

Post by keepamovin » Tue, Dec 30, 2025, 5:47 PM UTC

asdefghyk wrote: post #3

How much space is needed? ...for the data .... Im wondering if it would work on a tablet? ....

~9GB gzipped.

Re: Show HN: 22 GB of Hacker News in SQLite

#5

Post by carbocation » Tue, Dec 30, 2025, 6:37 PM UTC

That repo is throwing up a 404 for me.

Question - did you consider tradeoffs between duckdb (or other columnar stores) and SQLite?

Re: Show HN: 22 GB of Hacker News in SQLite

#6

Post by wslh » Tue, Dec 30, 2025, 6:44 PM UTC

Is this updated regularly? 404 on GitHub as the other comment.

With all due respect it would be great if there is an official HN public dump available (and not requiring stuff such as BigQuery which is expensive).

Re: Show HN: 22 GB of Hacker News in SQLite

#7

Post by keepamovin » Tue, Dec 30, 2025, 6:46 PM UTC

carbocation wrote: post #5

That repo is throwing up a 404 for me. Question - did you consider tradeoffs between duckdb (or other columnar stores) and SQLite?

No, I just went straight to sqlite. What is duckdb?

Re: Show HN: 22 GB of Hacker News in SQLite

#8

Post by linhns » Tue, Dec 30, 2025, 6:47 PM UTC

carbocation wrote: post #5

That repo is throwing up a 404 for me. Question - did you consider tradeoffs between duckdb (or other columnar stores) and SQLite?

Not the author here. I’m not sure about DuckDB, but SQLite allows you to simply use a file as a database and for archiving, it’s really helpful. One file, that’s it.

Re: Show HN: 22 GB of Hacker News in SQLite

#9

Post by zX41ZdbW » Tue, Dec 30, 2025, 6:49 PM UTC

The query tab looks quite complex with all these content shards: https://hackerbook.dosaygo.com/?view=query

I have a much simpler database: https://play.clickhouse.com/play?user=play#U0VMRUNUIHRpbWUsI...

Re: Show HN: 22 GB of Hacker News in SQLite

#10

Post by 3eb7988a1663 » Tue, Dec 30, 2025, 6:50 PM UTC

carbocation wrote: post #5

That repo is throwing up a 404 for me. Question - did you consider tradeoffs between duckdb (or other columnar stores) and SQLite?

While I suspect DuckDB would compress better, given the ubiquity of SQLite, it seems a fine standard choice.