Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Open source alternative to ChatGPT and ChatPDF-like AI tools

github.com

Post reply on HN
Print view View original

51–60 of 65 posts

Re: Show HN: Open source alternative to ChatGPT and ChatPDF-like AI tools

#51

Post by d7y »

smeej wrote: post #10
d7y wrote: post #4

Earlier quoted context omitted.

Great questions. > 1. Does chat-with-pdfs function work with scanned PDFs? Not yet. We don't do OCR or anything to extract text from images yet. But that would be an awesome feature, so we would love to add it in the future. > 2. In the video example for chat-with-pdfs you show uploading a document interactively. The part of processing is quite slow. Can the tool be fed these documents offline as well? Not as of righ…

If I've already run OCR on my PDFs and that's added now as an invisible layer, would it work then? I've had a workflow digitizing my incoming paper documents, running OCR, and tagging them, all locally, and it would be great to have an easy front-end to talk to them.

I haven't tried this myself, but I think it should work. It would be worth trying at least, so I highly encourage you to play with it, and file issues if you find any issues with it.

Re: Show HN: Open source alternative to ChatGPT and ChatPDF-like AI tools

#53

Post by d7y »

hifreq wrote: post #32

I couldn't find this info in the readme... does this tool anonymize ChatGPT requests? What does it mean that it's a private an secure tool in the context of using ChatGPT?

It is secure because it allows you to fully customize where to process the data (i.e. LLM inference), where to store it, and data-retention policies, etc. You can choose to use a locally running LLM (like it does in my second video) or use a secure third-party service provider like Azure OpenAI.

For example, if you want GDPR compliance, then you can choose Azure OpenAI running in the EU region. For HIPAA compliance, you should choose a service provider that provides the Business Associate Agreement (BAA). You can even run it in air-gapped facilities (like GitLab's offline mode [1]). In all of these cases, you can always run an Ollama-like inference service on your infra and point SecureAI Tools to it)

[1]: https://docs.gitlab.com/ee/topics/offline/

Re: Show HN: Open source alternative to ChatGPT and ChatPDF-like AI tools

#54

Post by d7y »

vunderba wrote: post #5

Great job. This is a relatively crowded area, particularly RAG style chat systems. It might be nice for SecureAI to call out what makes their product different from other open source players in the same space, specifically Khoj and Danswer, both of which allow you to chat with your documents, offer network authentication, and allow you to plug in your own LLM. Danswer https://github.com/danswer-ai/danswer Khoj https:…

A great question.

We are trying to build a single platform for all the AI tool needs. Chat-with-LLM and chat-with-documents are just a couple of apps or experiences that we have started with, but we have ambitious goals. In future, we would love to provide an SDK that exposes common abstractions and lets everyone build apps/experiences for the long tail of use cases.

Re: Show HN: Open source alternative to ChatGPT and ChatPDF-like AI tools

#55

Post by number6 »

suslik wrote: post #42
number6 wrote: post #31

Earlier quoted context omitted.

What you guys are referring to as Linux, is in fact, GNU/Linux, or as I've recently taken to calling it, GNU plus Linux plus Systemd. Linux is not an operating system unto itself, but rather another free component of a fully functioning GNU system made useful by the GNU corelibs, shell utilities, and vital system components comprising a full OS as defined by POSIX.Many computer users run a modified version of the GNU…

Cutting off "I’d just like to interject for a moment" from this pasta is herecy.

I botched it...

Re: Show HN: Open source alternative to ChatGPT and ChatPDF-like AI tools

#56

Post by freedomben »

htsh wrote: post #47
saberience wrote: post #44

Earlier quoted context omitted.

How can you chat with a pdf which doesn’t fit in the context window? I mean with a 500 page pdf you might need 100 context windows to fully grok it. Basically it makes no sense to “chat” with a 500 page pdf with todays LLMs.

That is what the RAG system does. The PDF is chunked and thrown into a vector store. And then when prompted, only the relevant bits are retrieved and stuffed into the context and sent to the LLM. So yeah it's kinda smoke and mirrors. In some cases, for some long PDFs, it works really well. If it's a 500 page PDF with many disparate topics, it may do fine.

Indeed. Would only add, context windows are continually multiplying in size. Who knows how long Moore's Law will apply here, but it's a continually improving window.

Re: Show HN: Open source alternative to ChatGPT and ChatPDF-like AI tools

#57

Post by qingcharles »

caseyf7 wrote: post #13

How do you get value from chatting with documents? I can scan and read a pdf faster than I can chat with an AI about it. There must be more to it than I realize.

I use Claude 2.1 to create summaries and TOCs of the magazines on my magazine encyclopedia. There is no way I could do that by hand for several million magazines averaging 100 pages each.

Re: Show HN: Open source alternative to ChatGPT and ChatPDF-like AI tools

#58

Post by saberience »

htsh wrote: post #47
saberience wrote: post #44

Earlier quoted context omitted.

How can you chat with a pdf which doesn’t fit in the context window? I mean with a 500 page pdf you might need 100 context windows to fully grok it. Basically it makes no sense to “chat” with a 500 page pdf with todays LLMs.

That is what the RAG system does. The PDF is chunked and thrown into a vector store. And then when prompted, only the relevant bits are retrieved and stuffed into the context and sent to the LLM. So yeah it's kinda smoke and mirrors. In some cases, for some long PDFs, it works really well. If it's a 500 page PDF with many disparate topics, it may do fine.

That assumes that only one part of the PDF, which fits in the context window, is relevant to the prompt, which seems like a fairly big assumption.

Re: Show HN: Open source alternative to ChatGPT and ChatPDF-like AI tools

#59

Post by saberience »

freedomben wrote: post #56
htsh wrote: post #47

Earlier quoted context omitted.

That is what the RAG system does. The PDF is chunked and thrown into a vector store. And then when prompted, only the relevant bits are retrieved and stuffed into the context and sent to the LLM. So yeah it's kinda smoke and mirrors. In some cases, for some long PDFs, it works really well. If it's a 500 page PDF with many disparate topics, it may do fine.

Indeed. Would only add, context windows are continually multiplying in size. Who knows how long Moore's Law will apply here, but it's a continually improving window.

I've found that the longer context windows don't seem to be a linear improvement in responses though. It's like the longer the context window, the quality of the response is perhaps broader, but less sharp or accurate. I've been using GPT4-turbo with the longer context window for coding tasks but it doesn't seem to have improved the responses as much as you would think, it seems to be more "distracted" now, which perhaps makes some intuitive sense.

I can give gpt4-turbo many full code files to try and solve a complex coding task but despite the larger window it seems to fail more often or ignore parts of the context window or just doesn't really answer the question.

Post reply on HN