Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: I have many PDFs – what is the best local way to leverage AI for search?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 92 posts

Re: Ask HN: I have many PDFs – what is the best local way to leverage AI for search?

#51

Post by fooker »

constantinum wrote: post #46

The primary challenge is not just about harnessing AI for search; it's about preparing complex documents of various formats, structures, designs, scans, multi-layout tables, and even poorly captured images for LLM consumption. This is a crucial issue. There is a 20 min read on why parsing PDFs is hell: https://unstract.com/blog/pdf-hell-and-practical-rag-applica... To parse PDFs for RAG applications, you'll need tool…

Modern LLMs are good enough at treating pdfs as images and groking the context.

Well, Claude and GPT-4 seem to be.

Re: Ask HN: I have many PDFs – what is the best local way to leverage AI for search?

#53

Post by ravetcofx »

andai wrote: post #35
bastien2 wrote: post #21

You don't. You use a full-text indexer and normal search tools. A chatbot is only going to decrease the integrity of query results.

I found that grep actually outperformed vector search for many queries. The only thing I was missing was when I didn't know how exactly to phrase something (the exact keyword to use). Do keyword search systems have workarounds for this? My own idea was for each keyword to generate a list of neighbor keywords in semantic space. I figured with such a dataset, I'd get something approximating vector search for free. I ma…

Do you know of any way to build a fast index you can run grep against? Would love to have something as instantaneous as "Everything" on windows for full text on Linux so I can just dump everything in a directory

Re: Ask HN: I have many PDFs – what is the best local way to leverage AI for search?

#54

Post by eviks »

yreg wrote: post #37
bastien2 wrote: post #21

You don't. You use a full-text indexer and normal search tools. A chatbot is only going to decrease the integrity of query results.

> decrease the integrity of query results What does that even mean. When you know the exact keywords then you use full-text. When you don't know them then other tools can be helpful.

It means you'd use the same tool since it's more convenient and get worse results in one tool vs. the other

Re: Ask HN: I have many PDFs – what is the best local way to leverage AI for search?

#55

Post by j45 »

pierre wrote: post #48
nl wrote: post #27

Earlier quoted context omitted.

Does the llamaindex PDF indexer correctly deal with multi-column PDFs? Most I've seen don't, and you get very odd results because of this.

Locally you can choose pypdf or mupdf wich are good but not perfect. If you can send your data online llamaparse is quite good.

Pulling the text out of the PDFs correctly and independently is correct.

Re: Ask HN: I have many PDFs – what is the best local way to leverage AI for search?

#56

Post by rahimnathwani »

bastien2 wrote: post #21

You don't. You use a full-text indexer and normal search tools. A chatbot is only going to decrease the integrity of query results.

Paperless-ngx set up using docker compose is good for this use case.

Re: Ask HN: I have many PDFs – what is the best local way to leverage AI for search?

#57

Post by rspoerri »

nl wrote: post #27
pierre wrote: post #15

RAG cli from llamaindex, allow you to do it 100% locally when used with ollama or llamacpp instead of OpenAI. https://docs.llamaindex.ai/en/stable/getting_started/starter...

Does the llamaindex PDF indexer correctly deal with multi-column PDFs? Most I've seen don't, and you get very odd results because of this.

i've made quite good conversions from pdf to markdown with https://github.com/VikParuchuri/marker . it's slow but worth a shot. Markdown should be easily parseable by a rag.

i'm trying to get a similar system setup on my computer.

Re: Ask HN: I have many PDFs – what is the best local way to leverage AI for search?

#58

Post by semi-extrinsic »

ravetcofx wrote: post #53
andai wrote: post #35

Earlier quoted context omitted.

I found that grep actually outperformed vector search for many queries. The only thing I was missing was when I didn't know how exactly to phrase something (the exact keyword to use). Do keyword search systems have workarounds for this? My own idea was for each keyword to generate a list of neighbor keywords in semantic space. I figured with such a dataset, I'd get something approximating vector search for free. I ma…

Do you know of any way to build a fast index you can run grep against? Would love to have something as instantaneous as "Everything" on windows for full text on Linux so I can just dump everything in a directory

Have you tried the more modern solutions like gripgrep, ack, etc.?

Or for something more comprehensive (to also search PDF, docx, etc.) there is ripgrep-all:

https://github.com/phiresky/ripgrep-all

Re: Ask HN: I have many PDFs – what is the best local way to leverage AI for search?

#59

Post by Capricorn2481 »

eviks wrote: post #54
yreg wrote: post #37

Earlier quoted context omitted.

> decrease the integrity of query results What does that even mean. When you know the exact keywords then you use full-text. When you don't know them then other tools can be helpful.

It means you'd use the same tool since it's more convenient and get worse results in one tool vs. the other

Because they're two different tools for two different tasks. If you expect to always know the exact phrase than, yes, grep will be better. But if you search a semantically similar phrase you will get nothing

Re: Ask HN: I have many PDFs – what is the best local way to leverage AI for search?

#60

Post by kkfx »

Honestly?

ocrmypdf + ripgrep-all, recoll (GUI+XLI xapian wrapper) if you prefer an indexed version, for mere full-text search, currently nothing gives better results. The semantic search it's still not there, Paperless-ngx, tagspaces and so on demand way too much time for adding just a single document to be useful at a certain scale.

My own personal version is org-mode, I keep all my stuff org-attached, so instead of searching the pdfs I search my notes linking them, a kind of metadata-rich, taggable, quick, full-text search however even if org-ql is there I almost not use it, just org-roam-node-find and counsel-rg on notes. Once done this allow for quick enough manual and variously automated archiving, do it on a large home directory it's a very long and tedious manual work. For me it's worth done since I keep adding documents and using them, but it took more than an year to be "almost done enough" and it's still unfinished after 4 years.

Post reply on HN