Live data from Hacker News

PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

github.com

141–143 of 143 posts

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#141
post #2

The most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI. Stop doing that. You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around. The best way to make these apps IMO is to make them work entirely local…

It's difficult to compete. A small business might answer 10,000 requests to their chat bot. The options are - Pay openai less than $50mo - Manage cloud gpus, hire ml engineers > $1000/mo - Buy a local 4090 and put it under someone's desk, $no reliability +$1500 fixed Any larger business will need scalability and you still can't compete with openai pricing. Maybe one of you startup inclined people can make an openllam…

> Maybe one of you startup inclined people can make an openllama startup that charges by request

I'm currently building www.lalamon.us specifically to provide a fully hosted open source model experience. One slight difference is that I'm providing a private chat instance for each user, so charging based on hours of active chat usage seemed to make more sense. Per-request charging seems more unpredictable for users, but I'd be interested in hearing the case either way.

Feel free to reach out with more questions if interested; my email is in my profile.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#142
post #59

Earlier quoted context omitted.

It's difficult to compete. A small business might answer 10,000 requests to their chat bot. The options are - Pay openai less than $50mo - Manage cloud gpus, hire ml engineers > $1000/mo - Buy a local 4090 and put it under someone's desk, $no reliability +$1500 fixed Any larger business will need scalability and you still can't compete with openai pricing. Maybe one of you startup inclined people can make an openllam…

Doing this. We soft launched yesterday with a paid Falcon-40B playground - 3 models for now Falcon 40b instruct, uncensored, and base. Adding API and per token pricing this week. https://api.llm-utils.org/ And more models coming soon. Vector storage isn’t on the roadmap (what stops using a separate vector store from working well? Could add to roadmap but want to add understand more first), and we could add fine tunin…

Shouldn't you use a .com tld?

Will your pricing be competitive with Replicate?

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#143
I’m working for a company that works as a security layer between any sensitive enterprise data and the LLMs. Regardless of the model (HF, ChatGPT, Bard), and regardless of the medium - conversational data, pdf, knowledge bases like Notion etc. It hides the sensitive data, preventing risky use and fact checking at the same time. Happy to make an intro if that’s what you’re looking for! tothepoint.tech
Post reply on HN