Live data from Hacker News

Turning a pile of documents into a searchable useable knowledge base

github.com

21–30 of 58 posts

Re: Turning a pile of documents into a searchable useable knowledge base

#21

I just installed this and, after a few hiccups, got it up and running on my Ubuntu system. Works great, looks great. Thank you for this. Half of my documents are OpenDocument format. Is there any chance you'll be supporting ODF in the future?

Yes, not supporting it is an oversight I will correct.

Re: Turning a pile of documents into a searchable useable knowledge base

#22

How do you feel about supporting an S3 compatible target as a feature request?

I'm actually thinking of this for a commercial product feature. However, if you use a tool like Rclone on Windows, Linux or Mac. Mount the s3 bucket and you can then run DocuBrowse as if the s3 bucket were local.

Re: Turning a pile of documents into a searchable useable knowledge base

#24
I learned a solution is to turn the documents into vectors in say PostgreSQL (with pgvector) and do a cosine similarity search with a search vector. Doing a search for embed models on HuggingFace shows nomic-ai/nomic-embed-text-v1.5 and Qwen/Qwen3-Embedding-0.6B. I might have used a larger one like Qwen/Qwen3-Embedding-4B.

There's some info for AnythingLLM[0] which supports RAG. AnythingLLM has LanceDB out of the box but also supports others including pgvector.

[0] https://docs.anythingllm.com/features/embedding-models

Re: Turning a pile of documents into a searchable useable knowledge base

#25

How do you feel about supporting an S3 compatible target as a feature request?

I'm actually thinking of this for a commercial product feature. However, if you use a tool like Rclone on Windows, Linux or Mac. Mount the s3 bucket and you can then run DocuBrowse as if the s3 bucket were local.

I love your project on many fronts. One, you're using Claude. Two, you used Python - but most importantly, you personally care about it.

I will be using this, and I will be making contributions to it as well.

> I'm actually thinking of this for a commercial product feature

Would you consider writing down which features you would like to make commercial product features and how you would like to price them?

Re: Turning a pile of documents into a searchable useable knowledge base

#26

I had an issue. A documents folder with over 12k objects in it. A hodgepodge of folders and sub-folders. That over time had created a mess that no amount of file movement was ever going to make it usable. I wanted: 1) To keep my data local 2) be able to filter out PII and other data 3) Be able to find and delete duplicates 4) Get short synopsis of what a document is 5) Semantic and keyword search 6) All of this kept…

Personal use? I need this at work, dragging useful info from tarpits like Teams and GitLab. Also need to search git repos including all branches and history (TIL/xkcd#153'd GitLab's web search can basically only do one branch at a time).

But how’d you access teams when it’s work teams and don’t have api access ?
Post reply on HN