Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fine-tune Google's Gemma 3

unsloth.ai

Post reply on HN
Print view View original

61–70 of 80 posts

Re: Fine-tune Google's Gemma 3

#61

Post by t1amat »

Tostino wrote: post #40
t1amat wrote: post #31

Earlier quoted context omitted.

I would like to see more knowledgeable people with experience talk about this. Is it just a matter of assembling Q/A pairs like: “What’s class X?”, “class X { … }” Do you really need to do this training on the base model instead, which means you have to fine tune chat on it afterward? How does this work?

I've not done fine tuning on code bases but I have done other fine tuning. You will generally get better results when you fine-tune the base model on your data. Since you still want to use it with the chat template in the end, you fine-tune the base model with the chat template with your specific data. From there you'll have a lora that knows your data alright, but still doesn't really work for chatting. You take tha…

Thank you, this was a great explanation!

Re: Fine-tune Google's Gemma 3

#63

Post by nl »

anon373839 wrote: post #26
minimaxir wrote: post #8

Earlier quoted context omitted.

Finetuning is easy and worthwhile, especially with LoRAs as these Unsloth demos do. The bottleneck then becomes how to self-host the finetuned model in a way that's cost-effective and scalable. In practice prompt engineering and few-shot prompting with modern LLMs, due to their strong-and-only-getting-better-over-time prompt adherence, tends to be more pragmatic.

There are inference providers such as Together AI that will serve your LoRA adapters at no extra cost above the model price. Then, there’s basically no difference between using your fine-tuned model or an API model off the shelf (except for the benefits you get from fine-tuning).

This (Serverless LoRA providers) is what most people want even if they don't know it.

Re: Fine-tune Google's Gemma 3

#65

Post by weird-eye-issue »

zk wrote: post #62

Is there a version of Gemma 3 that has tool calling? Google's blog claimed it supports tools but it doesn't seem like it actually does.

Not sure but you can just implement tool calling with some custom prompting yourself, it's really not too hard and is what we do

Re: Fine-tune Google's Gemma 3

#66

Post by fine_tune »

danielhanchen wrote: post #53
fine_tune wrote: post #34

Earlier quoted context omitted.

> I'm interested to know if anyone is using fine-tuning to train a model on proprietary or in-house codebases and documentation. I've done it, 1/2 the team though it was great 20% of the time, 1/2 the team hated it from day 0. I used roughly 500K lines of code. > How much effort is required to turn code into something one can use for fine-tuning? Very little to moderate, less than 200 lines of python, QWEM FIM, HF, L…

Oh fill in the middle is definitely smart especially for codebases!!

Love unsloth btw, use it for some other stuff at work, GRPO stuff was fun :)

I know its coming but "mUlTi GpU PlZ" :pleading: <3

Re: Fine-tune Google's Gemma 3

#68

Post by onel »

smokel wrote: post #27

I'm interested to know if anyone is using fine-tuning to train a model on proprietary or in-house codebases and documentation. RAG solutions seem to have their limitations, and fine-tuning might be a more effective approach. How much effort is required to turn code into something one can use for fine-tuning?

Generally it is recommended that you fine-tune if you want to shape the output style. If you eant ro only output json, or just output jsdoc, etc. To add knowledge, RAG is a better idea and easier to keep updated on code changes. If rag does not give back good results then that's a problem with the retrieval part, which can be measured and improved. We're building documentation + rag systems for enterprise code bases and this is what we saw works best.

Re: Fine-tune Google's Gemma 3

#69

Post by Tostino »

t1amat wrote: post #61
Tostino wrote: post #40

Earlier quoted context omitted.

I've not done fine tuning on code bases but I have done other fine tuning. You will generally get better results when you fine-tune the base model on your data. Since you still want to use it with the chat template in the end, you fine-tune the base model with the chat template with your specific data. From there you'll have a lora that knows your data alright, but still doesn't really work for chatting. You take tha…

Thank you, this was a great explanation!

Very welcome, I wish you luck!

Re: Fine-tune Google's Gemma 3

#70

Post by Xmd5a »

vineyardmike wrote: post #33
smokel wrote: post #27

I'm interested to know if anyone is using fine-tuning to train a model on proprietary or in-house codebases and documentation. RAG solutions seem to have their limitations, and fine-tuning might be a more effective approach. How much effort is required to turn code into something one can use for fine-tuning?

I’ve actually found the opposite. At work, we went from a fine-tuned model to a RAG system for internal and external documentation and a generic coding-focused model for code. Fine tuning against in-house code seems like a small gain over a base model and search. It’s unlikely your code is unique and special and big enough that it’s hard to get results from a base model. You’ll be pinned to a certain version of a cer…

There are frameworks for graph-based RAG that mix both approach. One LLM encodes info as a knowledge graph, gradually building up an ontology. Another LLM is used to query this knowledge graph by emitting speculative queries. As the database grows, the second LLM is fine-tuned again and again with exemple queries using the ontology the first LLM came up with.
Post reply on HN