What are you trying to do more specifically? You can use https://docalysis.com/ for most document RAG tasks.
Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?
21–30 of 247 posts
Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?
#22AWS Bedrock is fairly easy. You can do it in 5 or 6 clicks. You have to upload your documents to S3, create a “Knowledge Base” then sync your documents into a vector database like OpenSearch or PineCone. You are then good to go via their playground or the AWS API. I made a video here describing the process, check around 14 minutes in: https://ensembleanalytics.io/blog/introducing-bedrock-knowle... Bedrock is a decent…
Is there a limit? Could I create a knowledge base with 10,000 documents? 100k? 1M?
Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?
#23Earlier quoted context omitted.
Is there a limit? Could I create a knowledge base with 10,000 documents? 100k? 1M?
https://docs.aws.amazon.com/bedrock/latest/userguide/quotas....
I’m sorry, I don’t understand those limits. It uses a lot of unfamiliar terms like “batch inference” and “modality”. I just want a nice UI that I can give my hard-drive to and then ask it questions.
Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?
#24AWS Bedrock is fairly easy. You can do it in 5 or 6 clicks. You have to upload your documents to S3, create a “Knowledge Base” then sync your documents into a vector database like OpenSearch or PineCone. You are then good to go via their playground or the AWS API. I made a video here describing the process, check around 14 minutes in: https://ensembleanalytics.io/blog/introducing-bedrock-knowle... Bedrock is a decent…
Is there a limit? Could I create a knowledge base with 10,000 documents? 100k? 1M?
The documents are encoded as vectors and stored in a database, so I suspect it would be effectively unlimited. You would just pay for storage and compute.
AWS OpenSearch has fairly good integration so you could look up costs for that. It’s not the cheapest AWS service to run and not exactly serverless as you pay by the hour.
Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?
#25Easiest is OpenAI assistants api. Use the playground and it’s a no code experience.
Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?
#26Slightly off topic but is there recommended advice on how to tune / train not for document retrieval but for consistent JSON output with specific enums? i.e given a text, always return back a certain set of fields. For some keys here is the possible set of enums etc. One shot prompting does work but curious how others approach this if you have training data on hand.
You want grammars to restrict the output, search for "gbnf grammar". That and combined with a good prompt with an example, also check out outlines.dev
Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?
#27You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for. You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG. Test it out. If it…
Are there public examples of working products using RAG, compared with fine-tuning or training from scratch?
The OpenAI assistants API is an implementation of a RAG pipeline. It performs both RAG on any documents you upload, and on any conversation you have with it that exceeds the context.
Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?
#28If you’re looking for something that is hosted for you, at Notion we launched a feature for this a few weeks ago and it works quite well in my experience. RAG is one of the techniques used. https://www.notion.so/blog/introducing-q-and-a
Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?
#29You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for. You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG. Test it out. If it…
Another question, which one is preferred, LlamaIndex or Langchain, for RAG? Thanks in advance for your insights.
You basically don't use langchain for anything besides 30 minute demos that you copied from someone else's github. It has a completely spaghettified API, is not performant, and forces you into excessive mental contortions to reason about otherwise simple tasks.
LlamaIndex is pretty good.
Re: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?
#30Slightly off topic but is there recommended advice on how to tune / train not for document retrieval but for consistent JSON output with specific enums? i.e given a text, always return back a certain set of fields. For some keys here is the possible set of enums etc. One shot prompting does work but curious how others approach this if you have training data on hand.
There are many interesting tools that achieve this, like Outlines[0] and jsonformer[1].
I haven't tried them myself but they look very promising.
[0]: https://github.com/outlines-dev/outlines [1]: https://github.com/1rgs/jsonformer