Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

github.com

Post reply on HN
Print view View original

71–80 of 143 posts

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#71

Post by codingdave »

gumby wrote: post #44
akira2501 wrote: post #35

Earlier quoted context omitted.

> is that pretty much all of them require OpenAI. They're not here to release an actual product. They're here to release part of a CV proving they have "OpenAI" experience. I'm assuming this is the result of OpenAI not actually having any homegrown certification program of their own.

> OpenAI not actually having any homegrown certification program A bit off topic but where are certifications (e.g. Cisco, Microsoft) useful? I am sure they are useful (both to candidates and companies) because people go to the effort to get these certs, and if they were useless everyone would have stopped long ago. I don't assume people do it for ego satisfaction. But I've never worked anywhere where it has come up…

It is a consulting / business partner thing. Different levels in the business partner programs require minimum number of certified employees in your consulting firm. So if you work in that slice of the industry, certifications matter. Outside of that... not so much.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#72

Post by mcmoor »

jstummbillig wrote: post #29
chaxor wrote: post #2

The most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI. Stop doing that. You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around. The best way to make these apps IMO is to make them work entirely local…

What do you (or anyone else, feel free to chime in) do with other LLMs that makes them useable for anything that is not strictly tinkering? Here is my premise: We are past the wonder stage. I want to actually get stuff done efficiently. From what I have tested so far, the only model that allows me to do that halfway reliably is GPT-4. Am I incompetent or are we really just wishfully thinking in HN spirit that other L…

I still wonder what makes GPT-4 so much better than its contemporaries. That's why I saw tons of people trying to explain how GPT-4 works starting from simple neural network distasteful, tons of people already knew and do that but none of them is nearly close to GPT-4.l

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#73

Post by tikkun »

xigency wrote: post #64
JimmyRuska wrote: post #30

Earlier quoted context omitted.

It's difficult to compete. A small business might answer 10,000 requests to their chat bot. The options are - Pay openai less than $50mo - Manage cloud gpus, hire ml engineers > $1000/mo - Buy a local 4090 and put it under someone's desk, $no reliability +$1500 fixed Any larger business will need scalability and you still can't compete with openai pricing. Maybe one of you startup inclined people can make an openllam…

I’ve got an expensive GPU at home I’m not even using because there aren’t that many things to do with it. Give me more local options.

https://faraday.dev/

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#74

Post by halfcat »

gumby wrote: post #44
akira2501 wrote: post #35

Earlier quoted context omitted.

> is that pretty much all of them require OpenAI. They're not here to release an actual product. They're here to release part of a CV proving they have "OpenAI" experience. I'm assuming this is the result of OpenAI not actually having any homegrown certification program of their own.

> OpenAI not actually having any homegrown certification program A bit off topic but where are certifications (e.g. Cisco, Microsoft) useful? I am sure they are useful (both to candidates and companies) because people go to the effort to get these certs, and if they were useless everyone would have stopped long ago. I don't assume people do it for ego satisfaction. But I've never worked anywhere where it has come up…

Years ago, companies could get discounts if they were a “certified gold partner” or whatever.

To be a partner, the company would need a certain number of certifications among their employees, so there was tangible value to companies who either used a ton of Microsoft licensing or Cisco/Dell hardware, or resold those to their own clients (better discount equating to higher margin).

In some cases, getting the higher level certifications like Cisco CCIE was a virtual guarantee of a good job.

I feel like this has become less of a thing in recent years, but I’m not involved in that space anymore.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#75

Post by cj »

WhackyIdeas wrote: post #50
einpoklum wrote: post #28

Don't build a personal ChatGPT, and don't let OpenAI, Microsoft and their business partners (and probably the US government) have a bunch of your personal and private information.

So avoid all Microsoft products too?

Does Microsoft have an AI opt out?

AWS has an AI opt out at the organizational level that prevents them from using your data to “improve” their other services.

(I personally recommend everyone opt out now in AWS if you haven’t already…)

https://docs.aws.amazon.com/organizations/latest/userguide/o...

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#76

Post by behnamoh »

anton5mith2 wrote: post #70
chaxor wrote: post #2

The most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI. Stop doing that. You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around. The best way to make these apps IMO is to make them work entirely local…

https://mudler.pm/posts/smart-slackbot-for-teams/ with LocalAI?

This is a well-written tutorial and it's exactly what I was looking for! Thanks so much.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#77

Post by swader999 »

JimmyRuska wrote: post #30
chaxor wrote: post #2

The most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI. Stop doing that. You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around. The best way to make these apps IMO is to make them work entirely local…

It's difficult to compete. A small business might answer 10,000 requests to their chat bot. The options are - Pay openai less than $50mo - Manage cloud gpus, hire ml engineers > $1000/mo - Buy a local 4090 and put it under someone's desk, $no reliability +$1500 fixed Any larger business will need scalability and you still can't compete with openai pricing. Maybe one of you startup inclined people can make an openllam…

Wait until winter time and heat your house!

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#78

Post by BeetleB »

cloudking wrote: post #43

Am I the only one who doesn't need to search across my data? What are the use cases here

Example use case:

We have a group at work that meets and discusses various investment topics. The guy organizing it is fairly well connected and every week he tries to get an external speaker to come and present. Very educational.

I have raw notes for each of these presentations. My goal has always been to go through those notes, and properly organize the knowledge in there into a wiki of sorts. It's been 3 years since this all started and I still haven't found the time to do it. If I want to be realistic, I should accept that it'll never happen.

How do I go about finding information that I have in those notes? I could use text search but it's too sensitive to my search string - I'll often fail to find what I need. Also, the information may be scattered across several files, and I'd have to open all the hits and scan to find what I need.

With technology like this, I can put all my notes into some vector DB, and then use AI to ask in plain English what I need. Locally the system interprets my query and finds the most relevant documents in the DB. It then sends my query and those hits to OpenAI to interpret my question, and find the answer amongst my notes. A while ago I used Langchain to set it up and I got it working as a proof of concept. An Aha moment was when I asked it something and it gave me a response with information that was scattered over two different presentations. My challenge is that there are so many parameters I could play with, and I haven't yet thought of a way/metric to assess the performance of my system (any pointers would be appreciated!)

There's nothing personal in these notes, so no privacy concerns. I did want to set a similar thing up with over 20 years of emails, but didn't due to privacy. Also, I use a mail indexer (notmuch) which is fairly good so the need to use AI is not as strong.

But for other (non-personal) notes? If I can get this system working fairly well, it'd be a life saver. I've made so many notes on so many topics over the years, and it's worth real money not to have to organize it well. Just let me write my notes, and use an AI to retrieve what I need.

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#79

Post by eddieweng »

JimmyRuska wrote: post #9

Anyone know how milvus, quickwit, pinecone compares? I've been thinking about seeing if there's consulting opportunities for local businesses for LLMs, finetuning/vector search, chat bots. Also making tools to make it easier to drag and drop files and get personalized inference. Recently I saw this one pop into my linkedin feed, https://gpt-trainer.com/ . There's been a few others for documents I've found https://www…

qdrant is better in my opinion

Re: PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

#80

Post by EGreg »

3rdrodeo wrote: post #46
EGreg wrote: post #32

Earlier quoted context omitted.

I see, so when it comes to the top experts in AI screaming “we made a mistake, please be careful” we should use nuance and actually conclude the opposite — that we should press ahead and they’re wrong. But with Web3, we should just listen to a bunch of random no-name haters say “there are NO GOOD APPLICATIONS, trust us, period, stop talking about it”, use no nuance or critical thinking of our own, and simply stop bui…

The crypto group had a lot of time and even more money to make a compelling product that took off and so far they've failed. We've watched fraud after fraud as they've shown themselves to just be ignorant and arrogant ideologues who don't understand how the "modern" finance system came to be, what the average user wants out of financial or social products, or just outright scammers. We can keep sinking money into a b…

The AI crowd has been working for multile decades and only now has made progress that people care about. Also I’m pretty sure the “eye-watering amounts of money” Sam Altman referred to exceed what developers of even the crypto projects had, when they built eg Bitcoin or Ethereum. That’s what it took to make AI turn heads. Until AlphaGo you could also yell that AI has no real applications.

The personal computer crowd had hobbyists like Wozniak coming to meetups for decades and computers were the province of nerds, now everyone is addicted to them. Decades took place

You are like a person yelling at the video game industry: “pong and space invaders are a stupid waste of time with ugly graphics!! Don’t play or make video games!!” Until a few decades later we have Halo, Call of Duty etc.

Post reply on HN