Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

github.com

Post reply on HN
Print view View original

131–140 of 143 posts

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#131

Post by trolan »

rolisz wrote: post #107
ben_w wrote: post #39

Earlier quoted context omitted.

The only embeddings I currently see listed on https://openai.com/pricing are Ada v2, at $0.1/million tokens. Even if the alternative is free, how much do you value your time, how long will it take to set up an alternative, and how much use will you get out of it? If you're getting less than a million tokens and it takes half an hour longer to set up, you'd better be a student with zero literally income because that c…

If you've never coded or used Python before, yeah, go with OpenAI. Otherwise, generating embeddings with SentenceBERT takes 5 minutes. And from my personal experience Ada embeddings are not the best. They are large (makes aproximate searching harder), are distributed weirdly, and zimply put, other embeddings give better results for retrieval. Another advantage is that you are not an OA's whim: they just announced the…

Yes, when new technology comes you may need to upgrade. OpenAI aren't hero's but they are covering the cost to move people from old to new embedding models

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#132

Post by NavinF »

atenni wrote: post #102
halfcat wrote: post #74

Earlier quoted context omitted.

Years ago, companies could get discounts if they were a “certified gold partner” or whatever. To be a partner, the company would need a certain number of certifications among their employees, so there was tangible value to companies who either used a ton of Microsoft licensing or Cisco/Dell hardware, or resold those to their own clients (better discount equating to higher margin). In some cases, getting the higher le…

Definitely still a thing with Azure and Atlassian

Could you elaborate or drop some links? Are people getting discounts on Azure cloud bills with certifications?

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#133

Post by theonlybutlet »

BeetleB wrote: post #126
theonlybutlet wrote: post #104

Earlier quoted context omitted.

You're creating additional hardship for yourself. Why create a pdf only to convert it out of pdf again. Just insert all your notes into the LLM model.

You are probably replying to the wrong thread - I didn't say anything about a pdf.

Apologies

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#134

Post by theonlybutlet »

muspimerol wrote: post #119
theonlybutlet wrote: post #104

Earlier quoted context omitted.

You're creating additional hardship for yourself. Why create a pdf only to convert it out of pdf again. Just insert all your notes into the LLM model.

Because that requires retraining the model every time you take new notes. And this way you also still have the raw notes as similarity matches from the vector db, rather than them "disappearing" into the LLM model.

I see thanks for the insight.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#135

Post by rmbyrro »

serjester wrote: post #99
rmbyrro wrote: post #93

Earlier quoted context omitted.

It depends heavily on the use case, not org size. I consult for a ~70 people org that needs to process ~1M tokens per day. That costs $30K per day on OpenAI ChatGPT API. I'm sure this is not an extraordinary case.

Each person in the org needs 1M GPT-4 token and semantic search can’t be used to trim queries? I would be super curious to know more about this use case.

The data doesn't scale according to employee size. If they manage to cut the headcount in half, they'd still need to process the same amount of info.

The use case is based on public information on the internet. News articles, PRs, social media posts, etc.

LLMs are used to extract info from text in a structured format. It used to have several classification and NLP models to do the job, but now a single LLM can do it faster and with better accuracy.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#136

Post by raghavankl »

chaxor wrote: post #2

The most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI. Stop doing that. You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around. The best way to make these apps IMO is to make them work entirely local…

When I was creating this tool, I made sure to abstract out the reliance on just one LLM or vector db. Instead, I focused on using langchain/huggingface for tokenization, embedding, and conversational modeling. This was done purposely so that it would be simple to replace the OpenAI dependencies with any other LLMs if needed.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#137

Post by raghavankl »

Imnimo wrote: post #100

This readme is very confusing. It says we're going to use the GPT-2 tokenizer, and use GPT-2 as an embedding model. But looking at the code, it seems to use the default LangChain OpenAIEmbeddings and OpenAI LLM. Aren't those text-embedding-ada-002 and text-davinci-003, respectively? I don't understand how GPT-2 enters into this at all.

The embedding model used is the default OpenAI API embedding which is text-embedding-ada-002. GPT2 is only used during the tokenization process to efficiently calculate token lengths.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#138

Post by raghavankl »

emmender wrote: post #127

Please provide this reference in your readme / blog as it is the original source for your work... and provides the background for the tradeoff between the 2 approaches: 1) fine-tuning vs 2) Search-ask https://github.com/openai/openai-cookbook/blob/main/examples...

I respect OpenAI for creating a comprehensive cookbook, and my tooling uses OpenAI for embeddings and chat completion which I have mentioned in the Readme. However, it was not built using a single reference or code example, and rather it is a combination of ideas from huggingface, openAI and langchain documentation.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#139

Post by raghavankl »

quickthrower2 wrote: post #6

The author has a demo of this here: https://www.swamisivananda.ai/

I appreciate you finding and sharing this demo. I have also written a blog post on the vision of building a personal ChatGpt here https://devden.raghavan.studio/p/chatgpt-using-your-own-data

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#140

Post by raghavankl »

gigel82 wrote: post #34

I don't get it, GPT-2 is (one of the few) open models from OpenAI, you can just run it locally, why would you use their API for this? https://github.com/openai/gpt-2

The embedding model used is the default OpenAI API embedding, text-embedding-ada-002. GPT2 is only used during the tokenization process to calculate token lengths efficiently. I have updated my readme to reflect this information correctly.
Post reply on HN