Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

news.ycombinator.com

Post reply on HN
Print view View original

101–110 of 247 posts

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#101

Post by hhh »

tesdinger wrote: post #96
benjaminwootton wrote: post #3

AWS Bedrock is fairly easy. You can do it in 5 or 6 clicks. You have to upload your documents to S3, create a “Knowledge Base” then sync your documents into a vector database like OpenSearch or PineCone. You are then good to go via their playground or the AWS API. I made a video here describing the process, check around 14 minutes in: https://ensembleanalytics.io/blog/introducing-bedrock-knowle... Bedrock is a decent…

How does bedrock satisfy the non-hallucinating requirement?

It doesn't. You need to try and reduce hallucination as much as possible with your prompts, and then benchmark it.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#103

Post by pc86 »

gianpaj wrote: post #90
seedless-sensat wrote: post #15

Earlier quoted context omitted.

What is RAG? That's hard to search for

Ask chatgpt next time. "What is rag in context of AI?"

Or just using a traditional search engine and "rag" plus literally any ML/AI/LLM term will yield a half dozen results at the top with "Retrieval-augmented generation" in the page title.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#104

Post by ksjskskskkk »

seedless-sensat wrote: post #15
ilaksh wrote: post #5

You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for. You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG. Test it out. If it…

What is RAG? That's hard to search for

RAG: having a LLM spew search queries for you because your search foo is worse than a chat bot alucinations.

or because you want to charge your client the "ai fee".

or because your indexing is so bad you hide it from your user and blame the llm assistant dept.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#105

Post by KaoruAoiShiho »

Der_Einzige wrote: post #99
xnx wrote: post #9

GPT-4 Turbo has a 128K (~300 pages) context window, which probably handles a lot of use cases which might have previously needed extra training/refinement.

Long context length models are still mostly a mirage with the "lost in the middle" phenomenon rearing it's ugly little head on actual production use-cases of this.

Not true.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#108

Post by BlueGh0st »

FergusArgyll wrote: post #81
seedless-sensat wrote: post #15

Earlier quoted context omitted.

What is RAG? That's hard to search for

Off Topic; It fascinates me how much variance there is in peoples searching skills. some people think they are talking to a person when searching e.g 'what is the best way that i can {action}' I think the number one trick is to forget grammar and other language niceties and just enter concepts e.g. 'clean car best'

I used to do this. Then when Google's search results started declining in quality, I often found it better to search by what the average user would probably write.

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#109

Post by throwup238 »

ankit219 wrote: post #75

I think the answer depends on how many documents you have. To think in terms of tokens (assuming 750-1000 tokens is a page), if you have a good estimate of number of pages you want to query on, you can decide on the approach. Three popular approaches: 1. RAG: Most popular and works really well on smaller datasets. It is limited by number of vectors/embeddings. A typical embedding could be of 1000 tokens in size. Llam…

> I talked to a firm with 70GB worth of data. No way a RAG pipeline would give them results. They are struggling to get LLMs to work for them.

Wow so RAG is basically a toy for demos and low effort MVPs. 70GB is tiny, it’d barely qualify as “big data” 20 years ago.

Is anyone trying more advanced stuff like knowledge graph augmented generation to try to expand on that?

Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

#110

Post by min76 »

I have a related question. I have a fair idea of the LLM ecosystem. (Thanks to this very nice blog called Emerging Architectures for LLM Applications). The problem is, there are way too options in each component. ( For E.g, too many vector store implementations, ingestion engines etc) What is the easiest way to get started? Primarily around RAG on my own pdf files. Also, what is the best/easiest option for hosting?. That blog lists vercel,streamlit, streamship and modal. I know vercel at a high level and found it very good. I am not well versed with javascript/typescript though. I believe the best option for UI generation is to use one of their templates.
Post reply on HN