Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.5 Fine-Tuning Guide

unsloth.ai

Post reply on HN
Print view View original

71–80 of 118 posts

Re: Qwen3.5 Fine-Tuning Guide

#71

Post by hedgehog »

throwaway6977 wrote: post #10
prettyblocks wrote: post #8

Earlier quoted context omitted.

I think the biggest case for fine tuning is probably that you can take small models, fine tune them for applications that require structured output, and then run cheap inference at scale. "Frontier LLMs can do it with enough context" is not really a strong argument against fine-tuning, because they're expensive to run.

I agree- I'm currently trying to learn how I can embed a fine tuned tiny model into my c++ game so it can provide a narrative in prose of certain game-event logs. It needs to be as tiny as possible so it doesn't take resources away from the running game.

There are a bunch of tutorials on how to use GRPO to fine tune a small Qwen. Depending what you're doing LoRA or even just prefix tuning can give pretty good results with no special hardware.

Re: Qwen3.5 Fine-Tuning Guide

#73

Post by arcanemachiner »

sorenjan wrote: post #67
arcanemachiner wrote: post #65

Earlier quoted context omitted.

Nobody said you would use an LLM for that. It's an example of a process where "industrial inspection, in particular, [would] benefit from lower latency in exchange for accuracy". The point of their comment isn't that you would use an LLM to sort fruit. It was just an illustrative example.

The discussion was about fine-tuned Qwen models, not industrial inspection in general. I would also find it interesting to learn about what kind of edge AI industrial inspection task you could do with fine-tuned llms, not some handwavy answer about how sometimes latency is important in real time systems. Of course it is, so generally you don't use models with several billion parameters unless you need to.

The thread you're in broke away from the main discussion topic.

Again: Nobody is using LLMs to (for example) sort fruit. But there are some industrial processes that prioritize latency over reliability.

Re: Qwen3.5 Fine-Tuning Guide

#74

Post by prettyblocks »

andriy_koval wrote: post #54
prettyblocks wrote: post #8

Earlier quoted context omitted.

I think the biggest case for fine tuning is probably that you can take small models, fine tune them for applications that require structured output, and then run cheap inference at scale. "Frontier LLMs can do it with enough context" is not really a strong argument against fine-tuning, because they're expensive to run.

> "Frontier LLMs can do it with enough context" is not really a strong argument against fine-tuning, because they're expensive to run. I am not expert in this topic, but I am wondering if large cached context is actually cheap to run and frontier models would be cost efficient too in such setting?

I'd like to read more about that if anyone has any suggestions.

Re: Qwen3.5 Fine-Tuning Guide

#75

Post by thot_experiment »

dehrmann wrote: post #72
krasikra wrote: post #35

[flagged]

Naive question, but could neural networks handle these use cases?

NTA but almost certainly, the advantage is that Qwen3.5 is extremely generic already so adapting it to a specific task is way easier than training a NN from scratch. It's probably akin to how OCR is now just something I use Qwen for even though I have access to dedicated OCR tools, Qwen is good enough and it's already in my vram. Modern VLLMs are pretty great at answering basic questions about an image by default and I'm guessing finetuning takes them from "pretty good" to "good enough to use in production".

Re: Qwen3.5 Fine-Tuning Guide

#76

Post by [deleted] »

abhgh wrote: post #19
antirez wrote: post #6

Fine tuning is a story that is nice to tell but that with modern LLMs makes less and less sense. Modern LLMs are so powerful that they are able to few shot learn complicated things, so a strong prompt and augmenting the generation (given the massive context window of Qwen3.5, too) is usually the best option available. There are models for which fine tuning is great, like image models: there with LoRa you can get good…

They are great for specialized use-cases: (a) where the problem is not hard enough (you don't need reasoning), or (b) diverse enough (you don't need a world model), (c) you want cheap inference (and you can make it happen hardware-wise) and (d) you either have enough data or a workflow that accumulates data (with fine tuning with enough data you can sometimes beat a premier model while ensuring low latency - ofc, ass…

[deleted]

Re: Qwen3.5 Fine-Tuning Guide

#77

Post by NitpickLawyer »

aliljet wrote: post #33

Does fine tuning really improve anything above just pure RAG approaches for usee cases that involve tons of direct document context?

Remember how the tab-next-action model from Cursor was all the rage ~2 years ago when they launched it? That was a fine-tune of a ~70b model (they kinda alluded to this in a podcast).

Re: Qwen3.5 Fine-Tuning Guide

#78

Post by eitally »

andai wrote: post #36
krasikra wrote: post #35

[flagged]

Very interesting. Could you give examples of industrial tasks where lower accuracy is acceptable?

Industrial inspection is usually a fairly blunt task and I wouldn't be concerned about accuracy. Especially in high volume environments where training data is plentiful. Think about things like chip placement errors, alignment problems, bad solder joints, missing components.

Re: Qwen3.5 Fine-Tuning Guide

#80

Post by nl »

arcanemachiner wrote: post #73
sorenjan wrote: post #67

Earlier quoted context omitted.

The discussion was about fine-tuned Qwen models, not industrial inspection in general. I would also find it interesting to learn about what kind of edge AI industrial inspection task you could do with fine-tuned llms, not some handwavy answer about how sometimes latency is important in real time systems. Of course it is, so generally you don't use models with several billion parameters unless you need to.

The thread you're in broke away from the main discussion topic. Again: Nobody is using LLMs to (for example) sort fruit. But there are some industrial processes that prioritize latency over reliability.

No, we are literally trying to find a use case where using a lower accuracy LLM makes sense for a vision task.

But fine - what are these industrial processes where that prioritize latency over reliability and using a LLM - as mentioned by the OP - makes sense?

Post reply on HN