Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fine-tune Google's Gemma 3

unsloth.ai

Post reply on HN
Print view View original

71–80 of 80 posts

Re: Fine-tune Google's Gemma 3

#71

Post by fragmede »

317070 wrote: post #18
rockwotj wrote: post #2

is anyone outside of the research labs fine tuning models for production use cases? I have been seeing more people just using foundational models off the shelf especially in light of a new advancement that seems to come every few months

I've been finetuning these models since before chatGPT, and the one lesson I've learned is that by the time you have set up everything to fine-tune a model, you can expect a newer model to do as well with prompt-tuning. So, unless you hope to stay at the fore front (e.g. to be ahead of competitors), there has been no real reason to finetune for the last 4 years, at best you could hope to stay about 1-3 months ahead,…

Vibe coding has taken over for frontend dev, but outside that narrow band of very visible coding, most models aren't great at more esoteric programming languages. Even Swift gives Claude trouble. So the reason to fine-tune is simply that the best newest models still remain bad at things outside their comfort zone (how human).

Re: Fine-tune Google's Gemma 3

#72

Post by 317070 »

m101 wrote: post #21
317070 wrote: post #18

Earlier quoted context omitted.

I've been finetuning these models since before chatGPT, and the one lesson I've learned is that by the time you have set up everything to fine-tune a model, you can expect a newer model to do as well with prompt-tuning. So, unless you hope to stay at the fore front (e.g. to be ahead of competitors), there has been no real reason to finetune for the last 4 years, at best you could hope to stay about 1-3 months ahead,…

I feel like this is true but would be great if you could provide examples so we could get a better idea of why you think/know this.

I work for DeepMind on project Astra. Not to dwell too deep into confidentiality of what capabilities I have been looking at, but it has been the theme since the flamingo model that you only gain about 1 model-generation by fine-tuning versus prompt-tuning.

Re: Fine-tune Google's Gemma 3

#73

Post by 317070 »

fragmede wrote: post #71
317070 wrote: post #18

Earlier quoted context omitted.

I've been finetuning these models since before chatGPT, and the one lesson I've learned is that by the time you have set up everything to fine-tune a model, you can expect a newer model to do as well with prompt-tuning. So, unless you hope to stay at the fore front (e.g. to be ahead of competitors), there has been no real reason to finetune for the last 4 years, at best you could hope to stay about 1-3 months ahead,…

Vibe coding has taken over for frontend dev, but outside that narrow band of very visible coding, most models aren't great at more esoteric programming languages. Even Swift gives Claude trouble. So the reason to fine-tune is simply that the best newest models still remain bad at things outside their comfort zone (how human).

I take my quip both ways, so I would wager that even with finetuning, these models are only 1 generation ahead in esoteric language performance and therefore _still not very good_. Am I correct?

Re: Fine-tune Google's Gemma 3

#74

Post by kiratp »

simonw wrote: post #16
kiratp wrote: post #14

Earlier quoted context omitted.

We use multiple post-trained models in production, at scale at https://osmos.io

Have you published details of how you're doing that anywhere? Could be a useful marketing strategy for you, given how starved we all are of information about successful fine tuning stories.

Things have been moving so fast that it’s honestly hard for a small team to do that in parallel.

I got to present at GCP Next about a part of this last year: https://www.youtube.com/watch?v=5QsM1K9ahtw

I’m presenting in one (and maybe two) sessions with more info on the training side this year.

Re: Fine-tune Google's Gemma 3

#75

Post by fragmede »

317070 wrote: post #73
fragmede wrote: post #71

Earlier quoted context omitted.

Vibe coding has taken over for frontend dev, but outside that narrow band of very visible coding, most models aren't great at more esoteric programming languages. Even Swift gives Claude trouble. So the reason to fine-tune is simply that the best newest models still remain bad at things outside their comfort zone (how human).

I take my quip both ways, so I would wager that even with finetuning, these models are only 1 generation ahead in esoteric language performance and therefore _still not very good_. Am I correct?

Wanting it to be bad reeks of copium.

Re: Fine-tune Google's Gemma 3

#76

Post by eternityforest »

refulgentis wrote: post #10
rockwotj wrote: post #2

is anyone outside of the research labs fine tuning models for production use cases? I have been seeing more people just using foundational models off the shelf especially in light of a new advancement that seems to come every few months

IMHO the biggest factor holding that back is how rushed and distanced these model releases are, still. Both Phi-4-mini and Gemma 3 were released recently. Phi-4's damn close to a good, real, model release. Microsoft's done a great job of iterating. Gemma 3's an excellent, intelligent, model, but it's got a gaping blind spot: tool-calling / JSON output. There was a vague quick handwave about it in some PR, a PM/eng on…

Gemma3 1B seems to be able to choose which tool to use for very simple cases, if you constrain using anyOf, and narrow it down to just a few with RAG first.

It can't understand numbers very well though, "one thousand five" might become "1500".

JSON constraints seem to make them unable to figure it out even if they'd normally get it every time.

Maybe it's different with models above 4B though.

Re: Fine-tune Google's Gemma 3

#77

Post by 317070 »

fragmede wrote: post #75
317070 wrote: post #73

Earlier quoted context omitted.

I take my quip both ways, so I would wager that even with finetuning, these models are only 1 generation ahead in esoteric language performance and therefore _still not very good_. Am I correct?

Wanting it to be bad reeks of copium.

Why would I want it to be bad? I'm afraid I don't understand what you mean.

Re: Fine-tune Google's Gemma 3

#78

Post by Scipio_Afri »

Xmd5a wrote: post #70
vineyardmike wrote: post #33

Earlier quoted context omitted.

I’ve actually found the opposite. At work, we went from a fine-tuned model to a RAG system for internal and external documentation and a generic coding-focused model for code. Fine tuning against in-house code seems like a small gain over a base model and search. It’s unlikely your code is unique and special and big enough that it’s hard to get results from a base model. You’ll be pinned to a certain version of a cer…

There are frameworks for graph-based RAG that mix both approach. One LLM encodes info as a knowledge graph, gradually building up an ontology. Another LLM is used to query this knowledge graph by emitting speculative queries. As the database grows, the second LLM is fine-tuned again and again with exemple queries using the ontology the first LLM came up with.

Would you mind naming some of the frameworks?

Re: Fine-tune Google's Gemma 3

#79

Post by fragmede »

317070 wrote: post #77
fragmede wrote: post #75

Earlier quoted context omitted.

Wanting it to be bad reeks of copium.

Why would I want it to be bad? I'm afraid I don't understand what you mean.

you wrote, emphatically, that it would be "still not very good". Why do you believe that it would be still not very good after training on a specific problem? LLMs aren't able to do things outside their training data, as vast as it is, but if it's in it's training data, why are you emphatic that it's still not very good? If I ask it to make something that it just needs to copy out sample code of, it would be pretty good at that one very specific task to me.
Post reply on HN