Live data from Hacker News

Qwen3.5 Fine-Tuning Guide

unsloth.ai

21–30 of 118 posts

Re: Qwen3.5 Fine-Tuning Guide

#21
post #6

Fine tuning is a story that is nice to tell but that with modern LLMs makes less and less sense. Modern LLMs are so powerful that they are able to few shot learn complicated things, so a strong prompt and augmenting the generation (given the massive context window of Qwen3.5, too) is usually the best option available. There are models for which fine tuning is great, like image models: there with LoRa you can get good…

Fine-tuning still makes sense for cost/latency-sensitive applications. Massive context windows drastically slow down generation, and modern models' performance and instruction following ability relies heavily on a reasoning step that can consume orders of magnitude more tokens than the actual response (depending on the application), while a fine-tuned model can skip/significantly reduce that step.

Using the large model to generate synthetic data offline with the techniques you mentioned, then fine-tuning the small model on it, is an underrated technique.

Re: Qwen3.5 Fine-Tuning Guide

#22

Earlier quoted context omitted.

This is literally what I'm waiting for. I want a ~8B model that works well with OpenClaw.

I don't think you will get that anytime soon because for a model to work well with something like openclaw it needs a massive context window.

but but but but unified memory! (jk, I don't actually believe in Apple marketing words)

There might be future optimizations. Like, have your small model do COT to find where to look for memory that is relevant.

Re: Qwen3.5 Fine-Tuning Guide

#23
post #6

Fine tuning is a story that is nice to tell but that with modern LLMs makes less and less sense. Modern LLMs are so powerful that they are able to few shot learn complicated things, so a strong prompt and augmenting the generation (given the massive context window of Qwen3.5, too) is usually the best option available. There are models for which fine tuning is great, like image models: there with LoRa you can get good…

I think the biggest case for fine tuning is probably that you can take small models, fine tune them for applications that require structured output, and then run cheap inference at scale. "Frontier LLMs can do it with enough context" is not really a strong argument against fine-tuning, because they're expensive to run.

Wouldn’t it be better to use a grammar in the token sampler? Tuning is fine, but doesn’t guarantee a syntactical correct structured output. But if the sampler is grammar aware it could.

Re: Qwen3.5 Fine-Tuning Guide

#24

Earlier quoted context omitted.

I think the biggest case for fine tuning is probably that you can take small models, fine tune them for applications that require structured output, and then run cheap inference at scale. "Frontier LLMs can do it with enough context" is not really a strong argument against fine-tuning, because they're expensive to run.

This is literally what I'm waiting for. I want a ~8B model that works well with OpenClaw.

Qwen 9B doesn't?

Re: Qwen3.5 Fine-Tuning Guide

#25
post #23

Earlier quoted context omitted.

I think the biggest case for fine tuning is probably that you can take small models, fine tune them for applications that require structured output, and then run cheap inference at scale. "Frontier LLMs can do it with enough context" is not really a strong argument against fine-tuning, because they're expensive to run.

Wouldn’t it be better to use a grammar in the token sampler? Tuning is fine, but doesn’t guarantee a syntactical correct structured output. But if the sampler is grammar aware it could.

I think both should be done, they don't really serve the same purpose.

Re: Qwen3.5 Fine-Tuning Guide

#26
post #6

Fine tuning is a story that is nice to tell but that with modern LLMs makes less and less sense. Modern LLMs are so powerful that they are able to few shot learn complicated things, so a strong prompt and augmenting the generation (given the massive context window of Qwen3.5, too) is usually the best option available. There are models for which fine tuning is great, like image models: there with LoRa you can get good…

As strong as current LLMs are they are easily distracted from the task often. At production scale, fine tuning can make a lot more sense given you provide the model a very specific task.

Re: Qwen3.5 Fine-Tuning Guide

#27
post #24

Earlier quoted context omitted.

This is literally what I'm waiting for. I want a ~8B model that works well with OpenClaw.

Qwen 9B doesn't?

Nothing is really usable outside Opus.

I've tried too. Wasted a few days trying out even high end paid models.

Re: Qwen3.5 Fine-Tuning Guide

#28
post #9
post #2

What are some sample real world cases folks are using to fine tune their own small/medium models?

Only to prompt thought on this exact question, im interested in answers: I just ran a benchmark against haiku of a very simple document classification task that at the moment we farm out to haiku in parallel. very naive same prompt system via same api AWS bedrock, and can see that the a few of the 4b models are pretty good match, and could be easily run locally or just for cheap via a hosted provider. The "how much d…

if you add 2 spaces at the start of the line, you turn it into a code block

  like this

Re: Qwen3.5 Fine-Tuning Guide

#29
post #9
post #2

What are some sample real world cases folks are using to fine tune their own small/medium models?

Only to prompt thought on this exact question, im interested in answers: I just ran a benchmark against haiku of a very simple document classification task that at the moment we farm out to haiku in parallel. very naive same prompt system via same api AWS bedrock, and can see that the a few of the 4b models are pretty good match, and could be easily run locally or just for cheap via a hosted provider. The "how much d…

[deleted]

Re: Qwen3.5 Fine-Tuning Guide

#30
post #9
post #2

What are some sample real world cases folks are using to fine tune their own small/medium models?

Only to prompt thought on this exact question, im interested in answers: I just ran a benchmark against haiku of a very simple document classification task that at the moment we farm out to haiku in parallel. very naive same prompt system via same api AWS bedrock, and can see that the a few of the 4b models are pretty good match, and could be easily run locally or just for cheap via a hosted provider. The "how much d…

[deleted]
Post reply on HN