21–30 of 45 posts
Earlier quoted context omitted.
Elastic Search and its Amazon fork Opensearch perhaps?
Xapian is a library, while elastic has a client-server model Xapian is more like sqlite while elastic would be mariadb
I dream about a similar thing that can do OCR on scanned docs and extract text from my also sprawling library of epub and mobi files. If someone builds something like this, with maybe a LOCAL LLM to extract text descriptions from photos and movies as well as indexing metadata for everything, subtitles from movies and lyrics for songs, and add that to a NAS appliance, it’d be a killer.
This project has been around and maintained for more than a decade! Small footprint, good speed. One downside might be GPL v2 for commercial use.
used also by recoll, the desktop search app: https://www.recoll.org/
I use recoll to index and search thousands of pdfs. Because I always have the author name in the filename I can filter queries like this: Cybernetics OR steering filename:Heidegger ext:pdf It's an absolute power tool.
Title Year Author Name.pdf
Same benefits as you mentioned. You can also filter by time that way.
I remember having used, a very long time ago, a self-hosted search engine on my library of PDFs, and it was unbelievably useful. I dream about a similar thing that can do OCR on scanned docs and extract text from my also sprawling library of epub and mobi files. If someone builds something like this, with maybe a LOCAL LLM to extract text descriptions from photos and movies as well as indexing metadata for everything…
used also by recoll, the desktop search app: https://www.recoll.org/
I remember having used, a very long time ago, a self-hosted search engine on my library of PDFs, and it was unbelievably useful. I dream about a similar thing that can do OCR on scanned docs and extract text from my also sprawling library of epub and mobi files. If someone builds something like this, with maybe a LOCAL LLM to extract text descriptions from photos and movies as well as indexing metadata for everything…
I remember having used, a very long time ago, a self-hosted search engine on my library of PDFs, and it was unbelievably useful. I dream about a similar thing that can do OCR on scanned docs and extract text from my also sprawling library of epub and mobi files. If someone builds something like this, with maybe a LOCAL LLM to extract text descriptions from photos and movies as well as indexing metadata for everything…
Evernote will do this, you can feed it a bunch of PDFs and other documents, it will OCR them and make them all searchable. it's not perfect, but you can also add manual tags for things you know are important.
I remember having used, a very long time ago, a self-hosted search engine on my library of PDFs, and it was unbelievably useful. I dream about a similar thing that can do OCR on scanned docs and extract text from my also sprawling library of epub and mobi files. If someone builds something like this, with maybe a LOCAL LLM to extract text descriptions from photos and movies as well as indexing metadata for everything…
I remember having used, a very long time ago, a self-hosted search engine on my library of PDFs, and it was unbelievably useful. I dream about a similar thing that can do OCR on scanned docs and extract text from my also sprawling library of epub and mobi files. If someone builds something like this, with maybe a LOCAL LLM to extract text descriptions from photos and movies as well as indexing metadata for everything…
The big AI players are probably already scraping the bottom of this "barrel" in their search for training data, I am sure ...