Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.5 Fine-Tuning Guide

unsloth.ai

Post reply on HN
Print view View original

61–70 of 118 posts

Re: Qwen3.5 Fine-Tuning Guide

#61

Post by lelanthran »

throwaway6977 wrote: post #10
prettyblocks wrote: post #8

Earlier quoted context omitted.

I think the biggest case for fine tuning is probably that you can take small models, fine tune them for applications that require structured output, and then run cheap inference at scale. "Frontier LLMs can do it with enough context" is not really a strong argument against fine-tuning, because they're expensive to run.

I agree- I'm currently trying to learn how I can embed a fine tuned tiny model into my c++ game so it can provide a narrative in prose of certain game-event logs. It needs to be as tiny as possible so it doesn't take resources away from the running game.

> I agree- I'm currently trying to learn how I can embed a fine tuned tiny model into my c++ game so it can provide a narrative in prose of certain game-event logs.

Unless your game states have combinatoral exlosion, would it not be better to generate all of that pre-build? If templated you can generate a few hundreds of thousands of templates to use for any circumstance, then instantiate and stitch together those templates during the game runtime.

Re: Qwen3.5 Fine-Tuning Guide

#62

Post by bugglebeetle »

Unfortunately, this looks to only cover the larger MoE models. I imagine the smaller models are what most people would target. 9B just dropped two days ago, so not surprised it’s not explicitly documented, but does use a hybrid mamba architecture that I expect needs some special consideration.

Re: Qwen3.5 Fine-Tuning Guide

#63

Post by CamouflagedKiwi »

embedding-shape wrote: post #45
someotherperson wrote: post #44

Earlier quoted context omitted.

At a very high level, think fruit sorting[0] where the conveyor belt doesn't stop rolling and you need to rapidly respond, and all the way through to monitoring for things like defects in silicon wafers and root causing it. Some of these issues aren't problematic on their own, but you can aggregate data over time to see if a particular machine, material or process within a factory is degrading over time. This might n…

But why would I want to results to be done faster but less reliable, vs slower and more reliable? Feels like the sort of thing you'd favor accuracy over speed, otherwise you're just degrading the quality control?

It's not that you want it to be faster, but you want the latency to be predictable and reliable, which is much more the case for local inference than sending it away over a network (and especially to the current set of frontier model providers who don't exactly have standout reliability numbers).

Re: Qwen3.5 Fine-Tuning Guide

#64

Post by jwatte »

embedding-shape wrote: post #56
0cf8612b2e1e wrote: post #47

Earlier quoted context omitted.

Local, offline system you control is worth a lot. Introducing an external dependency guarantees you will have downtime outside of your control.

Right, but that doesn't answer why you'd need a fast 7b LLM rather than a slightly less fast 14b LLM.

Hard real time is a thing in some systems. Also, the current approaches might have 85% accuracy -- if the LLM can deliver 90% accuracy while being "less exact" that's still a win!

Re: Qwen3.5 Fine-Tuning Guide

#65

Post by arcanemachiner »

sorenjan wrote: post #50
someotherperson wrote: post #44

Earlier quoted context omitted.

At a very high level, think fruit sorting[0] where the conveyor belt doesn't stop rolling and you need to rapidly respond, and all the way through to monitoring for things like defects in silicon wafers and root causing it. Some of these issues aren't problematic on their own, but you can aggregate data over time to see if a particular machine, material or process within a factory is degrading over time. This might n…

But that's not something you'd use an LLM for. There have been computer vision systems sorting bad peas for more than a decade[0], of course there are plenty of use cases for very fast inspection systems. But when would you use an LLM for anything like that? [0] https://www.youtube.com/watch?v=eLDxXPziztw

Nobody said you would use an LLM for that. It's an example of a process where "industrial inspection, in particular, [would] benefit from lower latency in exchange for accuracy".

The point of their comment isn't that you would use an LLM to sort fruit. It was just an illustrative example.

Re: Qwen3.5 Fine-Tuning Guide

#67

Post by sorenjan »

arcanemachiner wrote: post #65
sorenjan wrote: post #50

Earlier quoted context omitted.

But that's not something you'd use an LLM for. There have been computer vision systems sorting bad peas for more than a decade[0], of course there are plenty of use cases for very fast inspection systems. But when would you use an LLM for anything like that? [0] https://www.youtube.com/watch?v=eLDxXPziztw

Nobody said you would use an LLM for that. It's an example of a process where "industrial inspection, in particular, [would] benefit from lower latency in exchange for accuracy". The point of their comment isn't that you would use an LLM to sort fruit. It was just an illustrative example.

The discussion was about fine-tuned Qwen models, not industrial inspection in general. I would also find it interesting to learn about what kind of edge AI industrial inspection task you could do with fine-tuned llms, not some handwavy answer about how sometimes latency is important in real time systems. Of course it is, so generally you don't use models with several billion parameters unless you need to.

Re: Qwen3.5 Fine-Tuning Guide

#68

Post by mountainriver »

antirez wrote: post #6

Fine tuning is a story that is nice to tell but that with modern LLMs makes less and less sense. Modern LLMs are so powerful that they are able to few shot learn complicated things, so a strong prompt and augmenting the generation (given the massive context window of Qwen3.5, too) is usually the best option available. There are models for which fine tuning is great, like image models: there with LoRa you can get good…

If that were true, we would be able to run working agents out of the box on any domain.

We are far from that still, for reliability in most applications you need fine tuning.

For any new modality you need fine tuning

For voice, image and video models you need fine tuning

For continual learning you (often) need fine tuning.

For any domain that is somewhat OOD you need fine tuning.

To fully ground a model you need fine tuning

Re: Qwen3.5 Fine-Tuning Guide

#70

Post by kristianp »

bravura wrote: post #32
antirez wrote: post #6

Fine tuning is a story that is nice to tell but that with modern LLMs makes less and less sense. Modern LLMs are so powerful that they are able to few shot learn complicated things, so a strong prompt and augmenting the generation (given the massive context window of Qwen3.5, too) is usually the best option available. There are models for which fine tuning is great, like image models: there with LoRa you can get good…

For me, trying to fine-tune a model to write "best day" prose I would accept over 80% of the time. You are correct if we are talking about knowledge. However it is bad at hyper-idiosyncratic, gritty style transfer. I first noticed the issue when asking claude code to draft email responses. The choice of register was off. ("Register in writing refers to the level of formality and tone chosen to suit a specific audienc…

> Instruct models are already so tuned that they could not be tuned

Some models have the base model available, that is before instruction tuning. For example llama 3 comes in "pre-trained and instruction tuned variants" [1]. I'm guessing you already know that though.

[1] https://huggingface.co/meta-llama/Meta-Llama-3-8B

Post reply on HN