Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

news.ycombinator.com

Post reply on HN
Print view View original

221–230 of 247 posts

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#221

Post by feitingen »

CommieBobDole wrote: post #163
FergusArgyll wrote: post #81

Earlier quoted context omitted.

Off Topic; It fascinates me how much variance there is in peoples searching skills. some people think they are talking to a person when searching e.g 'what is the best way that i can {action}' I think the number one trick is to forget grammar and other language niceties and just enter concepts e.g. 'clean car best'

Over the last couple of years, at least with Google, I've found that no strategy really seems to work all that well - Google just 'interprets' my request and assumes that I'm searching for a similar thing that has a lot more answers than what I was actually searching for, and shows me the results for that.

Some concepts seems to be permanently defined as a spelling error and will just be impossible to search for.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#222

Post by throwup238 »

fennecbutt wrote: post #216
throwup238 wrote: post #152

Earlier quoted context omitted.

"Toy" is the wrong word to describe it but it seems like another order of magnitude or two increase in context size will solve all their problems. On the other hand I've got a terabyte of text extracted from LibGen - let's say I can ignore the half that is fiction and I can dedupe the rest further by 80% - that's still 100gb. On top of that I've got 300gb of text extracted from court documents and that's just from Ca…

As a human would you read 100GB of data all at once? Or would you read it bit by bit, taking notes and summarising as you went along. Then compiling your notes/summaries into a final report? Because I don't see why we expect these models to be so superhuman when a 100K context would already be considered superhuman memory. Imagine me regurgitating 100k tokens worth of dialogue at you and expecting you to take into ac…

As a human would you do tens of billions of multiplies and additions per second? Store tens of thousands of books on something the size of a finger nail and recall them with perfect fidelity every time? Communicate with another human via optical signals using thousand mile long optical fiber across the entire Pacific ocean? Eat electricity instead of food? Project images from your eyes? Can you stick an audio cable in your butt to power speakers?

I'm talking about computers, not humans.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#224

Post by tomgs »

We do just that at Flexor!

We've built what we call an Unstructured Data Transformation Layer. Think about it like an assembly line from raw text to tables in your data warehouse.

We don't use Llamaindex, we have our own (proprietary) piece of tech that does this. We can and have been outputting gold-standard tables on top of a lot of different types of context (legal docs, call transcripts, customer reviews, chat logs, emails, blog posts, social media posts, etc...) and looking to expand to more interesting domains soon.

If anyone wants to hear more hit me up at tom [at] flexor [dot] ai (does this still work or are scrapers smart enough nowadays to just grep for this too lol)

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#225

Post by sivasurend »

legendofbrando wrote: post #218
ilaksh wrote: post #5

You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for. You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG. Test it out. If it…

What’s the benefit of llamaindex over just storing documents in chroma and using chroma to query? I’ve done the latter and trying to understand if there’s a performance gain to the former?

Not much, actually. For lower volumes of documents, vector stores like Chroma or Weaviate provide inbuilt RAG.

Things get messy when the number and type of documents increase. Below are the reasons why you may need advanced RAG.

1. Intelligent Data Parser 2. Chunking efficiently 3. Choice of embedding models 4. Query transformation 5. RAG technique 6. Prompt design 7. Feedback loop

Check out my blog on the 27 parameters, considerations and techniques one could follow to build a State-of-the-Art Chatbot.

https://www.lyzr.ai/27-parameters-techniques-considerations-...

So here is the quick guide.

For simpler usecases - inbuilt vector database RAG is sufficient For more complex ones - LlamaIndex or Langchain options are suitable For enterprise grade production use cases - Lyzr's SOTA RAG architecture comes in handy

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#226

Post by samstave »

ohthehugemanate wrote: post #204

NGL I think this one has passed the point on the tech maturityc curve where it makes sense to roll your own I played with MS office's copilot builder the other day and it's amazing. Point it at a set of base URLs, uploaded files, public or behind authentication. In literal seconds you have a copilot that can be embedded anywhere, including messengers. I gave it the root of Azure documentation, the root of red hat doc…

Can you do me an experiment.

Feed it a subreddit. Post results.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#227

Post by user32489318 »

BlueGh0st wrote: post #108
FergusArgyll wrote: post #81

Earlier quoted context omitted.

Off Topic; It fascinates me how much variance there is in peoples searching skills. some people think they are talking to a person when searching e.g 'what is the best way that i can {action}' I think the number one trick is to forget grammar and other language niceties and just enter concepts e.g. 'clean car best'

I used to do this. Then when Google's search results started declining in quality, I often found it better to search by what the average user would probably write.

and what would an average user write?

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#228

Post by user32489318 »

ercan wrote: post #171
FergusArgyll wrote: post #81

Earlier quoted context omitted.

Off Topic; It fascinates me how much variance there is in peoples searching skills. some people think they are talking to a person when searching e.g 'what is the best way that i can {action}' I think the number one trick is to forget grammar and other language niceties and just enter concepts e.g. 'clean car best'

I found something very annoying while looking for technical data ( a service manual for an ancient medical device - build around 2001). The same term was the name of the device + something about the power source. The result from the client network - my phone/client computer nothing related to the search for 4-5 pages. Same search from work - second result was what I was looking. So it seems there is a relation with y…

same experience. I'm generally getting better results at client's (VPN) network, we are all googling for the same stuff, I guess.

It must be possible to create a fixed set of google searches and rate the location based on the results. So you could physically travel to a Starbucks 20miles away to get the best results for the 'best USB-C dongle reddit'.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#229

Post by amelius »

user32489318 wrote: post #227
BlueGh0st wrote: post #108

Earlier quoted context omitted.

I used to do this. Then when Google's search results started declining in quality, I often found it better to search by what the average user would probably write.

and what would an average user write?

An entire question instead of a bunch of keywords.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#230

Post by dbish »

noiv wrote: post #208
dbish wrote: post #138

Earlier quoted context omitted.

Just use your embeddings model of choice that works with your language, I believe Ada from openai is multilingual but I don’t know what languages it works well on, there are many embedding models out there, huggingface is your friend in this search. The output is just a vector and the rest of the system can basically stay the same. The only other thing that may need to change depending on language is any text preproc…

Well, thanks, how do they deal with technical terms never seen during training?

Unknown words can’t really be handled other than mapping to a placeholder “unknown” vector in most cases. This can be dealt with in a variety of ways, text preprocessing being one of them
Post reply on HN