Live data from Hacker News

Show HN: 22 GB of Hacker News in SQLite

hackerbook.dosaygo.com

81–90 of 229 posts

Re: Show HN: 22 GB of Hacker News in SQLite

#81

Earlier quoted context omitted.

Because anything that even slightly differs from the standard American phrasing of something must be "LLM generated" these days.

Sometimes I want to write more creatively, but then worry I’ll be accused of being an LLM. So I dumb it down. Remove the colorful language. Conform.

[deleted]

Re: Show HN: 22 GB of Hacker News in SQLite

#82
post #67
post #52

Earlier quoted context omitted.

There was a UK government GitHub repo that did something interesting with this kind of trick against S3 but I checked just now and the repo is a 404. Here are my notes about what it did: https://simonwillison.net/2025/Feb/7/sqlite-s3vfs/ Looks like it's still on PyPI though: https://pypi.org/project/sqlite-s3vfs/ You can see inside it with my PyPI package explorer: https://tools.simonwillison.net/zip-wheel-explorer?p…

I recovered it from https://archive.softwareheritage.org/browse/origin/directory... and pushed a fresh copy to GitHub here: https://github.com/simonw/sqlite-s3vfs This comment was helpful in figuring out how to get a full Git clone out of the heritage archive: https://news.ycombinator.com/item?id=37516523#37517378 Here's a TIL I wrote up of the process: https://til.simonwillison.net/github/software-archive-recove...

I also have a locally cloned copy of that repo from when it was on GitHub. Same latest commit as your copy of it.

From what I see in GitHub in your copy of the repo, it looks like you don’t have the tags.

Do you have the tags locally?

If you don’t have the tags, I can push a copy of the repo to GitHub too and you can get the tags from my copy.

Re: Show HN: 22 GB of Hacker News in SQLite

#83
Beautiful !

2026 prayer: for all you AI junkies—please don’t pollute H/N with your dirty AI gaming.

Don’t bot posts, comments, or upvote/downvote just to maximize karma. Please.

We can’t identify anymore who’s a bot and who’s human. I just want to hang out with real humans here.

Re: Show HN: 22 GB of Hacker News in SQLite

#84
I tried "select * from items limit 10" and it is slowly iterating through the shards without returning. I got up to 60 shards before I stopped. Selecting just one shard makes that query return instantly. As mentioned elsewhere I think duckdb can work faster by only reading the part of a parquet file it needs over http.

I was getting an error that the users and user_domains tables aren't available, but you just need to change the shard filter to the user stats shard.

Re: Show HN: 22 GB of Hacker News in SQLite

#86
post #67

Earlier quoted context omitted.

I recovered it from https://archive.softwareheritage.org/browse/origin/directory... and pushed a fresh copy to GitHub here: https://github.com/simonw/sqlite-s3vfs This comment was helpful in figuring out how to get a full Git clone out of the heritage archive: https://news.ycombinator.com/item?id=37516523#37517378 Here's a TIL I wrote up of the process: https://til.simonwillison.net/github/software-archive-recove...

I also have a locally cloned copy of that repo from when it was on GitHub. Same latest commit as your copy of it. From what I see in GitHub in your copy of the repo, it looks like you don’t have the tags. Do you have the tags locally? If you don’t have the tags, I can push a copy of the repo to GitHub too and you can get the tags from my copy.

I don't have the tags! It would be awesome if you could push that.

Re: Show HN: 22 GB of Hacker News in SQLite

#88
post #86

Earlier quoted context omitted.

I also have a locally cloned copy of that repo from when it was on GitHub. Same latest commit as your copy of it. From what I see in GitHub in your copy of the repo, it looks like you don’t have the tags. Do you have the tags locally? If you don’t have the tags, I can push a copy of the repo to GitHub too and you can get the tags from my copy.

I don't have the tags! It would be awesome if you could push that.

Uploaded here:

https://github.com/Quantum-Nomad/sqlite-s3vfs

Re: Show HN: 22 GB of Hacker News in SQLite

#90
post #86

Earlier quoted context omitted.

I don't have the tags! It would be awesome if you could push that.

Uploaded here: https://github.com/Quantum-Nomad/sqlite-s3vfs

Thanks for that, though actually it turns out I had them after all - I needed to run:

  git push --tags origin
Post reply on HN