Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

news.ycombinator.com

Post reply on HN
Print view View original

191–200 of 247 posts

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#191

Post by thorum »

ilaksh wrote: post #5

You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for. You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG. Test it out. If it…

Has anyone tried using an LLM for the retrieval stage? Instead of using vector embeddings, have a (small, fast) LLM literally scan the entire corpus in chunks extracting relevant sections?

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#192

Post by spullara »

thorum wrote: post #191
ilaksh wrote: post #5

You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for. You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG. Test it out. If it…

Has anyone tried using an LLM for the retrieval stage? Instead of using vector embeddings, have a (small, fast) LLM literally scan the entire corpus in chunks extracting relevant sections?

That would still be very slow with any reasonably large corpus.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#193

Post by warthog »

ilaksh wrote: post #5

You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for. You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG. Test it out. If it…

Why Llamaindex instead of Langchain?

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#194

Post by bmgoau »

patja wrote: post #132
bmgoau wrote: post #92

Run https://github.com/imartinez/privateGPT Then make ingest /path/to/folder/with/files Then chat to the LLM. Done. Docs: https://docs.privategpt.dev/overview/welcome/quickstart

I've tried LocalGPT, PrivateGPT, and H2OGPT. Have you been satisfied with the responses you get from PrivateGPT? When I tried it, it seemed very shallow/cursory in its responses. I saw much more detailed and complete responses when trying H2OGPT.

The models released over the last two weeks are much much better than the defaults. Try changing settings.yaml to use https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGU... .

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#195

Post by staranjeet »

You can use embedchain[1] to connect various data sources and then get a RAG application running on your local and production very easily. Embedchain is an open source RAG framework and It follows a conventional but configurable approach.

The conventional approach is suitable for software engineer where they may not be less familiar with AI. The configurable approach is suitable for ML engineer where they have sophisticated uses and would want to configure chunking, indexing and retrieval strategies.

[1]: https://github.com/embedchain/embedchain

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#196

Post by newstio »

ametrau wrote: post #136
newstio wrote: post #124

how to run a local llm model for RAG apps. Retrieval documents are turkish. But ı would to analyze this documents with llm. But ı have not a turkish local llm. How to solve this problem. Out of fine-tune and training.

If the LLM you use supports Turkish (I am pretty sure chatgpt does) then the language doesn’t matter. Augment the Generation by Retrieving turkish documents / snippets.

Thank you. but I don't want to use gpt. I want to use local llm. LLM does'not support Turkish a good level.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#197

Post by npamart »

ParetoOptimal wrote: post #180
ZunarJ5 wrote: post #84

I'm a fan of Khoj. Been using it for months. https://github.com/khoj-ai/khoj

I need to give Khoj a try again. I tried it on my org inbox on a whim, couldn't get useful results, and promptly forgot about it.

Nice pun

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#198

Post by _giorgio_ »

baobun wrote: post #169
_giorgio_ wrote: post #165

Earlier quoted context omitted.

Is this a joke? I'd like to fine-tune an existing model, let's say mistral, on a new dataset using (existing tools), I've seen that there are a lot of approaches, but none that has gained traction, there isn't a clear consensus...

Not a joke. Without more specifics, doesn't sound like LLMs are what you need/want.

Thanks for wasting my time. I ask about fine tuning llama or mixtral and he answers with a nonsense, telling me that I don't want what I want.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#199

Post by asah »

kkielhofner wrote: post #154

As others have said you want RAG. The most feature complete implementation I've seen is h2ogpt[0] (not affiliated). The code is kind of a mess (most of the logic is in an ~8000 line python file) but it supports ingestion of everything from YouTube videos to docx, pdf, etc - either offline or from the web interface. It uses langchain and a ton of additional open source libraries under the hood. It can run directly on…

I just tried installing this on a fresh GCP instance with NVidia T4 GPU, and let's just say it was non-trivial. The CPU version running on my mac was mostly an OK install and worked pretty well.
Post reply on HN