Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

github.com

Post reply on HN
Print view View original

101–110 of 143 posts

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#101

Post by bicx »

chaxor wrote: post #2

The most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI. Stop doing that. You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around. The best way to make these apps IMO is to make them work entirely local…

> Stop doing that.

This commanding attitude on HN seems to be getting worse lately. Not a fan.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#102

Post by atenni »

halfcat wrote: post #74
gumby wrote: post #44

Earlier quoted context omitted.

> OpenAI not actually having any homegrown certification program A bit off topic but where are certifications (e.g. Cisco, Microsoft) useful? I am sure they are useful (both to candidates and companies) because people go to the effort to get these certs, and if they were useless everyone would have stopped long ago. I don't assume people do it for ego satisfaction. But I've never worked anywhere where it has come up…

Years ago, companies could get discounts if they were a “certified gold partner” or whatever. To be a partner, the company would need a certain number of certifications among their employees, so there was tangible value to companies who either used a ton of Microsoft licensing or Cisco/Dell hardware, or resold those to their own clients (better discount equating to higher margin). In some cases, getting the higher le…

Definitely still a thing with Azure and Atlassian

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#103

Post by woeirua »

hi wrote: post #5

Keep your data private and don't leak it to third parties. Use something like privateGPT (32k stars). Not your keys, not your data. "Interact privately with your documents using the power of GPT, 100% privately, no data leaks"[0] [0] https://github.com/imartinez/privateGPT

It’s significantly worse than OpenAIs offerings, and I’m tired of people pretending as though these models are totally interchangeable yet. They are not.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#104

Post by theonlybutlet »

BeetleB wrote: post #78
cloudking wrote: post #43

Am I the only one who doesn't need to search across my data? What are the use cases here

Example use case: We have a group at work that meets and discusses various investment topics. The guy organizing it is fairly well connected and every week he tries to get an external speaker to come and present. Very educational. I have raw notes for each of these presentations. My goal has always been to go through those notes, and properly organize the knowledge in there into a wiki of sorts. It's been 3 years sin…

You're creating additional hardship for yourself. Why create a pdf only to convert it out of pdf again. Just insert all your notes into the LLM model.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#106

Post by rolisz »

JimmyRuska wrote: post #30
chaxor wrote: post #2

The most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI. Stop doing that. You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around. The best way to make these apps IMO is to make them work entirely local…

It's difficult to compete. A small business might answer 10,000 requests to their chat bot. The options are - Pay openai less than $50mo - Manage cloud gpus, hire ml engineers > $1000/mo - Buy a local 4090 and put it under someone's desk, $no reliability +$1500 fixed Any larger business will need scalability and you still can't compete with openai pricing. Maybe one of you startup inclined people can make an openllam…

FastChat-T5 can work for such a use case and it runs on (beefy) CPUs. With a 700$/month instance, it can do 4 conversations simultaneously, without needing GPUs.

The instant a company has sensitive data, this becomes very viable.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#107

Post by rolisz »

ben_w wrote: post #39
space_fountain wrote: post #25

Earlier quoted context omitted.

Do you have citations on OpenAI embeddings being some of the cheapest and best? The signs I've seen points almost in the opposite direction?

The only embeddings I currently see listed on https://openai.com/pricing are Ada v2, at $0.1/million tokens. Even if the alternative is free, how much do you value your time, how long will it take to set up an alternative, and how much use will you get out of it? If you're getting less than a million tokens and it takes half an hour longer to set up, you'd better be a student with zero literally income because that c…

If you've never coded or used Python before, yeah, go with OpenAI. Otherwise, generating embeddings with SentenceBERT takes 5 minutes.

And from my personal experience Ada embeddings are not the best. They are large (makes aproximate searching harder), are distributed weirdly, and zimply put, other embeddings give better results for retrieval.

Another advantage is that you are not an OA's whim: they just announced the deprecation of some previous model. What are you going to do when they will deprecate Ada v2 and you've built a huge system on top of it? You'll have to regenerate embeddings and hope everything still works just as well.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#108

Post by Templa »

hi wrote: post #5

Keep your data private and don't leak it to third parties. Use something like privateGPT (32k stars). Not your keys, not your data. "Interact privately with your documents using the power of GPT, 100% privately, no data leaks"[0] [0] https://github.com/imartinez/privateGPT

Having something that could be used with confluence would be so nice. Having documentation written and just asking questions about it.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#109

Post by simonw »

gigel82 wrote: post #34

I don't get it, GPT-2 is (one of the few) open models from OpenAI, you can just run it locally, why would you use their API for this? https://github.com/openai/gpt-2

It's not using GPT-2 - the README is incorrect.

It's using "from langchain.embeddings import OpenAIEmbeddings" - which is the OpenAI embeddings API, text-embedding-ada-002

The only aspect of GPT-2 this is using is GPT2TokenizerFast.from_pretrained("gpt2") - which it uses as the length function to count tokens for the RecursiveCharacterTextSplitter() langchain utility.

Which doesn't really make sense - why use the GPT-2 tokenizer for that? May as well just count characters or even count words based on .split(), it's not particularly important how the counting works here.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#110

Post by freediver »

jstummbillig wrote: post #29
chaxor wrote: post #2

The most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI. Stop doing that. You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around. The best way to make these apps IMO is to make them work entirely local…

What do you (or anyone else, feel free to chime in) do with other LLMs that makes them useable for anything that is not strictly tinkering? Here is my premise: We are past the wonder stage. I want to actually get stuff done efficiently. From what I have tested so far, the only model that allows me to do that halfway reliably is GPT-4. Am I incompetent or are we really just wishfully thinking in HN spirit that other L…

You are correct in this assesment. A majority of individuals and startups playing around with turning LLMs into products aim to be prepared for the arrival of the subsequent generation of models. When that occurs, they'll already have a product or company in place and can simply integrate the new models.

Models are getting commoditized, well executed ideas are not.

Post reply on HN