Does fine tuning it on a set of docs in your “knowledge base” help for generalizing it so it can answer questions pertaining to new documents that come in (with a “similar” style/structure but with different content/fscts)?
[deleted]
How we got fine-tuning Mistral-7B to not suck
31–40 of 50 posts
Re: How we got fine-tuning Mistral-7B to not suck
#32Glad to see that more people outside the big ai labs are figuring out how to do fine tuning. Some open source LLM authors also seem to have figured it out. I think many users get put off it because just pushing a button doesn’t work and the whole thing seems like a black box that you don’t know how to fix when it breaks. It turns out that finetuning can be debugged, but the methods aren’t well documented (yet), eg by…
> RAG is easy to setup - it’s push button
I'm interested to hear about push-button solutions for RAG that aren't a SaaS.
Re: How we got fine-tuning Mistral-7B to not suck
#33I've done fine tuning too, but the reasons they mention in "Why not just use RAG?" aren't very good. People way understimate what RAG can do, even if in general people don't talk about the right things. For example LlamaIndex spends a lot of time talking about various extractors which is the easy part. The hard thing is deciding what you are actually searching for given a chat context. RAG is a horrible hack (and the…
Is that lossy?
Isn't the vector representation of the text (and the ANN index itself) lossy, and the source text itself the source of truth?
Re: How we got fine-tuning Mistral-7B to not suck
#34Glad to see that more people outside the big ai labs are figuring out how to do fine tuning. Some open source LLM authors also seem to have figured it out. I think many users get put off it because just pushing a button doesn’t work and the whole thing seems like a black box that you don’t know how to fix when it breaks. It turns out that finetuning can be debugged, but the methods aren’t well documented (yet), eg by…
> RAG is easy to setup - it’s push button I'm interested to hear about push-button solutions for RAG that aren't a SaaS.
[deleted]
Re: How we got fine-tuning Mistral-7B to not suck
#35If you look at the source [1] you can see how they solved their what are the doctors going to do problem. It is literally included in one of the prompts now :-) Users tend to ask broad, vague questions of the document in order to test that the system is working. We want those queries to work well. For example, a user would ask "what are the doctors going to do?" of a document that is about a junior doctors' strike. T…
That's right: the question is just does it generalize? :-D
Re: How we got fine-tuning Mistral-7B to not suck
#36Glad to see that more people outside the big ai labs are figuring out how to do fine tuning. Some open source LLM authors also seem to have figured it out. I think many users get put off it because just pushing a button doesn’t work and the whole thing seems like a black box that you don’t know how to fix when it breaks. It turns out that finetuning can be debugged, but the methods aren’t well documented (yet), eg by…
> RAG is easy to setup - it’s push button I'm interested to hear about push-button solutions for RAG that aren't a SaaS.
https://github.com/lamini-ai/lamini-sdk/tree/main/03_RAG
You can implement RAG in 80 lines of python and 0 SaaS libraries.
It's extremely easy.
1. Load your data as a giant string (streaming)
2. Chunk it (big chunk size, small chunk steps)
3. Call an LLM to convert chunk -> embedding, store in an index (or just concat it onto a numpy array)
4. Call an LLM to convert query -> embedding
5. Compute cosine similarity between the embeddings, pick the max
6. Insert the picked chunks into the LLM prompt
That's it. I'd encourage you to try to implement it yourself.
Anything beyond this is unnecessary complexity.
I walk through the code/whiteboard of the whole thing in this video: https://www.youtube.com/watch?v=Xkzd_YNbWmc&t=6003s
Re: How we got fine-tuning Mistral-7B to not suck
#37Does fine tuning it on a set of docs in your “knowledge base” help for generalizing it so it can answer questions pertaining to new documents that come in (with a “similar” style/structure but with different content/fscts)?
Fine tuning on your documents will really help to answer questions in the style and tone of those documents, so in that way, yes it helps.
It would be possible to include some parts of the new documents in the prompt so you can answer questions about new facts in the style and tone of your old documents, which we feel is useful. We are also experimenting with adding Retrieval Augmented Generation alongside fine tuning to see if the results are better than either or.
disclaimer: I work on Helix
Re: How we got fine-tuning Mistral-7B to not suck
#38For helix, I notice that GitHub is listed as a data source, but there's nothing in the docs about this. I'd really love to see what a model trained on my commonly used git repos (which generally are newer than The Stack etc), and in particular their commit history. Ideally these would make it easier for code completion to have the historical context as well as the current code to play with in determining what to writ…
This is spot on, the thing we've not yet done is make it easy to import a repo(s) code and the associated metadata into a fine tuning session easily.
> I often wonder how you'd go about organizing training data for a full historic github repo in a way that makes sense for training (or RAG)?
This is the hard part :-) But you are right - it would be intriguing to see what the output of a fune-tuned & RAG model would look like for this use-case. We are currently experimenting with adding RAG alongside the fine tuned model (so it's both, not either or) to see if it produces better results.
I will make sure we take a look at the gihub repo use case because it feels like that would be an interesting experiment to do!
disclaimer: I work on Helix
Re: How we got fine-tuning Mistral-7B to not suck
#39For helix, I notice that GitHub is listed as a data source, but there's nothing in the docs about this. I'd really love to see what a model trained on my commonly used git repos (which generally are newer than The Stack etc), and in particular their commit history. Ideally these would make it easier for code completion to have the historical context as well as the current code to play with in determining what to writ…
This is spot on, the thing we've not yet done is make it easy to import a repo(s) code and the associated metadata into a fine tuning session easily. > I often wonder how you'd go about organizing training data for a full historic github repo in a way that makes sense for training (or RAG)? This is the hard part :-) But you are right - it would be intriguing to see what the output of a fune-tuned & RAG model would lo…
Reading through the dataprep stuff, I wonder if doing more RAG during the prep stage might help this sort of task on structured daa. E.g. pre-indexing related parts and using those to build summaries / QA pairs. I took a look at the current prompts that are very research focused ("professors" creating questions), and could extrapolate from that to a dev mindset nicely.
Re: How we got fine-tuning Mistral-7B to not suck
#40Earlier quoted context omitted.
Fine tuning is just more training -- so it's definitely possible to teach the model facts this way too. In practice we've found that it's a bit of a balancing act to teach the model the new knowledge without destroying existing knowledge, but it's just a matter of tuning the parameters carefully. We're also researching whether we can fine-tune a brand new expert in a MoE model like Mixtral, I've also seen work on fin…
Have you tried generating two sets of qapairs, one with bad answers, and using DPO?
Not yet, sounds promising!