PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
1–10 of 143 posts
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#2The most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI.
Stop doing that.
You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around.
The best way to make these apps IMO is to make them work entirely locally, with an easy string that's swappable in a .toml file to any huggingface model. Then if you really want OpenAI crap, you can make it happen with some other docker secret or `pass` chain or something with a key, while changing up the config.
The default should be local first, do as much as possible, and then if the user /really/ wants to, make the collated prompt send a very few set of tokens to openAI.
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#3Is there a company that makes a hosted version of something like this? I quite want a little AI that I can feed all my data to to ask questions to.
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#4The most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI. Stop doing that. You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around. The best way to make these apps IMO is to make them work entirely local…
Have you seen PrivateGPT. It's quite good and free.
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#5Keep your data private and don't leak it to third parties. Use something like privateGPT (32k stars). Not your keys, not your data.
"Interact privately with your documents using the power of GPT, 100% privately, no data leaks"[0]
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#6The author has a demo of this here: https://www.swamisivananda.ai/
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#7Is there a company that makes a hosted version of something like this? I quite want a little AI that I can feed all my data to to ask questions to.
https://libraria.dev/ offers this and more as a service. It has added conveniences like integration with your google drive, youtube videos, and such
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#8gpt4all has this truly locally. I recommend those with a decent GPU to give it a go.
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#9Anyone know how milvus, quickwit, pinecone compares?
I've been thinking about seeing if there's consulting opportunities for local businesses for LLMs, finetuning/vector search, chat bots. Also making tools to make it easier to drag and drop files and get personalized inference. Recently I saw this one pop into my linkedin feed, https://gpt-trainer.com/ . There's been a few others for documents I've found
Nope nope, wouldn't want to compete with that on pricing. Local open source LLMs on a 3090 would also be a cool service, but wouldn't have any scalability.
Are there any other finetuning or vector search context startups you've seen?
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#10The most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI. Stop doing that. You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around. The best way to make these apps IMO is to make them work entirely local…
Have you seen PrivateGPT. It's quite good and free.
what hardware do you need for that?