Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

news.ycombinator.com

Post reply on HN
Print view View original

161–170 of 247 posts

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#161

Post by rolisz »

throwup238 wrote: post #152
ankit219 wrote: post #133

Earlier quoted context omitted.

Some caution here. Not everything needs to go into a RAG pipeline (Eg: a database table would not necessarily need to be embedded, but its schema should be.). There would be a lot of repetitions, lots of junk and useless data, and numerical data and parsing through that would be a pain. Then comes how the users would behave. You need a longer string to get accurate results. Most non tech users would rather write shor…

"Toy" is the wrong word to describe it but it seems like another order of magnitude or two increase in context size will solve all their problems. On the other hand I've got a terabyte of text extracted from LibGen - let's say I can ignore the half that is fiction and I can dedupe the rest further by 80% - that's still 100gb. On top of that I've got 300gb of text extracted from court documents and that's just from Ca…

What are you trying to achieve with that dataset from LibGen? I kinda expect that GPT4 was trained on the data that is available on LibGen

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#162

Post by rolisz »

hrdwdmrbl wrote: post #115
throwup238 wrote: post #109

Earlier quoted context omitted.

> I talked to a firm with 70GB worth of data. No way a RAG pipeline would give them results. They are struggling to get LLMs to work for them. Wow so RAG is basically a toy for demos and low effort MVPs. 70GB is tiny, it’d barely qualify as “big data” 20 years ago. Is anyone trying more advanced stuff like knowledge graph augmented generation to try to expand on that?

I have a _small_ e-commerce company and we have >300GB. Most of that bulk is photos and videos though, but in an ideal world I’d like my AI assistant to find that stuff too: “I’m making a Boxing Day ad campaign. Can you show me the ads that we’ve made in previous years and all of the photos that we’ve taken of our new Reindeer and Elf designs?”

Photos and videos are very different from text. 300 GB of text is not comparable to 300 GB of photos.

You can do something using image embeddings to get what you want.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#163

Post by CommieBobDole »

FergusArgyll wrote: post #81
seedless-sensat wrote: post #15

Earlier quoted context omitted.

What is RAG? That's hard to search for

Off Topic; It fascinates me how much variance there is in peoples searching skills. some people think they are talking to a person when searching e.g 'what is the best way that i can {action}' I think the number one trick is to forget grammar and other language niceties and just enter concepts e.g. 'clean car best'

Over the last couple of years, at least with Google, I've found that no strategy really seems to work all that well - Google just 'interprets' my request and assumes that I'm searching for a similar thing that has a lot more answers than what I was actually searching for, and shows me the results for that.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#165

Post by _giorgio_ »

baobun wrote: post #153
_giorgio_ wrote: post #118

If you want a simpler task, like training a mistral llama etc on your documents, to act as a document completer , how would you proceed instead? Probably much easier. Thanks

Markov chains, or, if you need to get fancy, Hidden Markov models?

Is this a joke? I'd like to fine-tune an existing model, let's say mistral, on a new dataset using (existing tools),

I've seen that there are a lot of approaches, but none that has gained traction, there isn't a clear consensus...

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#166

Post by wackget »

pc86 wrote: post #103
gianpaj wrote: post #90

Earlier quoted context omitted.

Ask chatgpt next time. "What is rag in context of AI?"

Or just using a traditional search engine and "rag" plus literally any ML/AI/LLM term will yield a half dozen results at the top with "Retrieval-augmented generation" in the page title.

Or people could just not use obscure acronyms when discussing specialised topics on an open forum?

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#167

Post by jncfhnb »

ilaksh wrote: post #5

You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for. You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG. Test it out. If it…

RAG is a funny thing. It’s like going back to Watson for specifics but letting the LLM handle the generic stuff.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#168

Post by AndrewKemendo »

hrdwdmrbl wrote: post #116
xuancanh wrote: post #86

Earlier quoted context omitted.

It's doable with Amazon Q, but it's in Preview phase now. https://aws.amazon.com/q/

Is Q the answer to this whole thread? People are talking about AWS Bedrock but that seems like something a startup would build upon and offer something like Q, eh?

I’m going to try it this week so maybe we have a follow up thread??

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#169

Post by baobun »

_giorgio_ wrote: post #165
baobun wrote: post #153

Earlier quoted context omitted.

Markov chains, or, if you need to get fancy, Hidden Markov models?

Is this a joke? I'd like to fine-tune an existing model, let's say mistral, on a new dataset using (existing tools), I've seen that there are a lot of approaches, but none that has gained traction, there isn't a clear consensus...

Not a joke. Without more specifics, doesn't sound like LLMs are what you need/want.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#170

Post by gosub100 »

pc86 wrote: post #103
gianpaj wrote: post #90

Earlier quoted context omitted.

Ask chatgpt next time. "What is rag in context of AI?"

Or just using a traditional search engine and "rag" plus literally any ML/AI/LLM term will yield a half dozen results at the top with "Retrieval-augmented generation" in the page title.

What percentage of people could you fool if you told them it was AI and replayed standard search results but with the "karaoke-like" prompt that highlights each word (as if we're 2nd graders in Special Ed learning how to string more than 2 sentences together)
Post reply on HN