Don't build a personal ChatGPT, and don't let OpenAI, Microsoft and their business partners (and probably the US government) have a bunch of your personal and private information.
So avoid all Microsoft products too?
[deleted]
PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
61–70 of 143 posts
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#62Earlier quoted context omitted.
It's difficult to compete. A small business might answer 10,000 requests to their chat bot. The options are - Pay openai less than $50mo - Manage cloud gpus, hire ml engineers > $1000/mo - Buy a local 4090 and put it under someone's desk, $no reliability +$1500 fixed Any larger business will need scalability and you still can't compete with openai pricing. Maybe one of you startup inclined people can make an openllam…
Doing this. We soft launched yesterday with a paid Falcon-40B playground - 3 models for now Falcon 40b instruct, uncensored, and base. Adding API and per token pricing this week. https://api.llm-utils.org/ And more models coming soon. Vector storage isn’t on the roadmap (what stops using a separate vector store from working well? Could add to roadmap but want to add understand more first), and we could add fine tunin…
Lots of people using LLMs to make chat bots from their existing datasets: customer service troubleshooting, FAQs, billing, scheduling. Being able to upload their own pdfs, spreadsheets, docx, crawl their home page, lets the chat bot become personalized to their use case. While you could locally query your own vectordb before prompting, people buy paid service so they won't have to manage any of the technical details.
If people can drag and drop some files from their nas, you parse them with apache tika or similar https://tika.apache.org/ , they can start using personalized branded bots. It also lets you do things like refusing to answer, if the vector database returns nothing and the use case requires a specific answer from the docs only (not the llm to make stuff up).
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#63Is there a company that makes a hosted version of something like this? I quite want a little AI that I can feed all my data to to ask questions to.
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#64The most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI. Stop doing that. You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around. The best way to make these apps IMO is to make them work entirely local…
It's difficult to compete. A small business might answer 10,000 requests to their chat bot. The options are - Pay openai less than $50mo - Manage cloud gpus, hire ml engineers > $1000/mo - Buy a local 4090 and put it under someone's desk, $no reliability +$1500 fixed Any larger business will need scalability and you still can't compete with openai pricing. Maybe one of you startup inclined people can make an openllam…
I’ve got an expensive GPU at home I’m not even using because there aren’t that many things to do with it. Give me more local options.
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#65Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#66Keep your data private and don't leak it to third parties. Use something like privateGPT (32k stars). Not your keys, not your data. "Interact privately with your documents using the power of GPT, 100% privately, no data leaks"[0] [0] https://github.com/imartinez/privateGPT
Is this robust enough to feed all your emails and chat logs into it and have convos with it? Will it be able to extract context to figure out questions to recent logs, etc?
In theory, yes.
I've not got it to work yet though, it ends up hallucinating answers to all the questions about documents I feed it.
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#67Earlier quoted context omitted.
It's difficult to compete. A small business might answer 10,000 requests to their chat bot. The options are - Pay openai less than $50mo - Manage cloud gpus, hire ml engineers > $1000/mo - Buy a local 4090 and put it under someone's desk, $no reliability +$1500 fixed Any larger business will need scalability and you still can't compete with openai pricing. Maybe one of you startup inclined people can make an openllam…
I’ve got an expensive GPU at home I’m not even using because there aren’t that many things to do with it. Give me more local options.
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#68The most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI. Stop doing that. You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around. The best way to make these apps IMO is to make them work entirely local…
What do you (or anyone else, feel free to chime in) do with other LLMs that makes them useable for anything that is not strictly tinkering? Here is my premise: We are past the wonder stage. I want to actually get stuff done efficiently. From what I have tested so far, the only model that allows me to do that halfway reliably is GPT-4. Am I incompetent or are we really just wishfully thinking in HN spirit that other L…
This line of thinking only works if it's impossible to imagine a world where OpenAI isn't the leader. In 2 years if the non OpenAI models are better then it will serve us much better to allow these tools to work with other models as well.
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#69Earlier quoted context omitted.
Doing this. We soft launched yesterday with a paid Falcon-40B playground - 3 models for now Falcon 40b instruct, uncensored, and base. Adding API and per token pricing this week. https://api.llm-utils.org/ And more models coming soon. Vector storage isn’t on the roadmap (what stops using a separate vector store from working well? Could add to roadmap but want to add understand more first), and we could add fine tunin…
Lots of people using LLMs to make chat bots from their existing datasets: customer service troubleshooting, FAQs, billing, scheduling. Being able to upload their own pdfs, spreadsheets, docx, crawl their home page, lets the chat bot become personalized to their use case. While you could locally query your own vectordb before prompting, people buy paid service so they won't have to manage any of the technical details.…
For those use cases the “custom ChatGPT” tools I linked here might be better https://news.ycombinator.com/item?id=36649777
Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
#70The most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI. Stop doing that. You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around. The best way to make these apps IMO is to make them work entirely local…
https://mudler.pm/posts/smart-slackbot-for-teams/ with LocalAI?