Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 247 posts

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#51

Post by nmstoker »

seedless-sensat wrote: post #15
ilaksh wrote: post #5

You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for. You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG. Test it out. If it…

What is RAG? That's hard to search for

Seems fairly easy to search for to me - top results are all relevant:

https://kagi.com/search?q=ml+rag

https://www.google.com/search?q=ml+rag

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#52

Post by monkeydust »

Did this in the summer via RAG. One thing we realised is that pure vector embeddings retrieval doesn't work so well for docs with acronyms (which let's face it all businesses have). Created a hybrid solution using embeddings and BM25 which is traditional ranking tool. This hybrid gave best results.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#53

Post by yu3zhou4 »

isoprophlex wrote: post #50
yu3zhou4 wrote: post #49

Earlier quoted context omitted.

Is Llamaindex + hosted model on Azure OpenAI Services still the best option?

There's Azure AI studio which is kinda like AWS bedrock. It's not bad, but for max control and versatility i'd start out rolling my own w/ for example llamaindex+azure-branded openai like you say.

Thanks. Is it possible to have persistent RAG in Azure AI Studio though? I found only a preview version of uploading files that are available to model, but when using this model through API, this uploaded data is not available to the model

Likely I misunderstood about how RAG works with Azure AI Studio, so sorry in advance

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#54

Post by monkeydust »

isoprophlex wrote: post #29
dfhg wrote: post #10

Earlier quoted context omitted.

Another question, which one is preferred, LlamaIndex or Langchain, for RAG? Thanks in advance for your insights.

You basically don't use langchain for anything besides 30 minute demos that you copied from someone else's github. It has a completely spaghettified API, is not performant, and forces you into excessive mental contortions to reason about otherwise simple tasks. LlamaIndex is pretty good.

Yea discovered this with Langchain last week. Was great for a demo then started to push it harder and spent ages trawling Reddit, discord, GitHub trying to find solutions to issues only to discover what was supposed to be supported was deprecated. Got a massive headache for what should have been a simple change. Moved on now.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#55

Post by gardnr »

monkeydust wrote: post #52

Did this in the summer via RAG. One thing we realised is that pure vector embeddings retrieval doesn't work so well for docs with acronyms (which let's face it all businesses have). Created a hybrid solution using embeddings and BM25 which is traditional ranking tool. This hybrid gave best results.

I was going to ask how you integrated BM25 but then I found this: https://docs.llamaindex.ai/en/stable/examples/retrievers/bm2...

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#57

Post by isoprophlex »

yu3zhou4 wrote: post #53
isoprophlex wrote: post #50

Earlier quoted context omitted.

There's Azure AI studio which is kinda like AWS bedrock. It's not bad, but for max control and versatility i'd start out rolling my own w/ for example llamaindex+azure-branded openai like you say.

Thanks. Is it possible to have persistent RAG in Azure AI Studio though? I found only a preview version of uploading files that are available to model, but when using this model through API, this uploaded data is not available to the model Likely I misunderstood about how RAG works with Azure AI Studio, so sorry in advance

https://github.com/azure/aistudio-copilot-sample

Check it out, specifically steps 3 and 4. As with almost every Microsoft CLI tool and SDK, it's clunky... and you can tell everyone is rushing this AI shit out as fast as they can to stay in the game. But what you want should be doable.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#58

Post by drakonka »

As mentioned above, I don't think you'd need to train your own model for this (or for most use cases of this, anyway). You'd use a RAG.

I've tried out working with custom documents in two different ways for different types of data:

* Once using LlamaIndex + Chroma[0] to transcribe and then conversationally query video contents (using GPT 3.5 or 4 as the backing LLM).

* Once using GPT Plus, uploading long-form PDFs of my own fiction books to the GPT's knowledge base. I use this to help me remember character names and timelines (not always accurate, so results need to be treated with caution) and help brainstorm story or tech ideas for my world.

Both work for what I'm using them for. I feel like option one is more customizable and easier to tweak for the types of results I would want, if I have concrete requirements about what kind of output I'm looking for. Option two has a lower barrier to entry and is just a little lower effort (no need to run your own app).

For the next iteration, I'd like to try out AWS Bedrock and compare the workflow and results.

[0] https://www.daily.co/blog/search-your-video-content-library-...

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#59

Post by NKosmatos »

There was something similar about Retrieval Augmented Generation (RAG) recently on HN: https://news.ycombinator.com/item?id=38491251

Early next year I’m preparing something similar for my team, so I’ll surely look into the useful links/recommendations posted by fellow HNers :-)

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#60

Post by mathis-l »

dfhg wrote: post #10
ilaksh wrote: post #5

You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for. You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG. Test it out. If it…

Another question, which one is preferred, LlamaIndex or Langchain, for RAG? Thanks in advance for your insights.

Haystack [1] is another good option. It‘s modular, doesn’t get in your way and is particularly strong at retrieval. People like the documentation too.

Disclaimer: I work at deepset

[1] https://github.com/deepset-ai/haystack

Post reply on HN