Ask HN: Has anybody built search on top of Anna's Archive?

Post by Aachen » Wed, Jun 04, 2025, 2:46 PM UTC

ThatPlayer wrote: post #48

tomthe wrote: post #16

Earlier quoted context omitted.

I wonder if you could implement it with only static hosting? We would need to split the index into a lot of smaller files that can be practically downloaded by browsers, maybe 20 MB each. The user types in a search query, the browser hashes the query and downloads the corresponding index file which contains only results for that hashed query. Then the browser sifts quickly through that file and gives you the result.…

I've done something similar with a static hosted site I'm working on. I opted to not reinvent the wheel, and just use WASM Sqlite in the browser. Sqlite already splits the database into fixed-size pages, so the driver using HTTP Range Requests can download only the required pages. Just have to make good indexes. I can even use Sqlite's full-text search capabilities!

Post by underlines » Wed, Jun 04, 2025, 2:52 PM UTC

Post by rkoots » Wed, Jun 04, 2025, 3:13 PM UTC

Post by mdp2021 » Wed, Jun 04, 2025, 3:41 PM UTC

andrepd wrote: post #70

namlem wrote: post #28

Earlier quoted context omitted.

It would be incredible for LLMs. Searching it, using it as training data, etc. Would probably have to be done in Russia or some other country that doesn't respect international copyright though.

> Would probably have to be done in Russia or some other country that doesn't respect international copyright though. Incredible, several years of major American AI companies showing that flaunting copyright only matters if it's college kids torrenting shows or enthusiasts archiving bootlegs on whatcd, but if it's big corpos doing it it's necessary for innovation. Yet some people still believe "it would have to be do…

Post by diggan » Wed, Jun 04, 2025, 4:07 PM UTC

freedomben wrote: post #94

executesorder66 wrote: post #42

Earlier quoted context omitted.

> or some other country that doesn't respect international copyright though. Like the US? OpenAI et al. don't give a shit.

> > or some other country that doesn't respect international copyright though. > Like the US? OpenAI et al. don't give a shit. OpenAI is not a country and therefore cannot make laws that don't respect international (or domestic) copyright. Also the US is a lot bigger than OpenAI and the big tech corps, and the law is very much on the side of copyright holders in the US.