Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.5 Fine-Tuning Guide

unsloth.ai

Post reply on HN
Print view View original

111–118 of 118 posts

Re: Qwen3.5 Fine-Tuning Guide

#111

Post by GorbachevyChase »

therockhead wrote: post #97
Jowsey wrote: post #85

Earlier quoted context omitted.

This reply is entirely AI generated. You guys are trying to find reason in a hallucination. It's unfortunately impossible to put into words what the "LLM smell" is at this point, but I trust someone else who spends a lot of time reading LLM output can back me up on this. I've seen these agent-written fake anecdotes on Twitter, Reddit, and now here, all with the exact same formatting. They pretend to be real people wi…

The two day old account is an obvious hint but I got to be honest, the content didn't look suspicious on first read. I know you touched on it above, but what do you think triggered your AI generated thought ?

Some people don’t farm social credit. I usually drop my account after it gets too high because the evidence of hipsters approving of my words shames me.

Re: Qwen3.5 Fine-Tuning Guide

#112

Post by sorenjan »

IanCal wrote: post #109
nl wrote: post #80

Earlier quoted context omitted.

No, we are literally trying to find a use case where using a lower accuracy LLM makes sense for a vision task. But fine - what are these industrial processes where that prioritize latency over reliability and using a LLM - as mentioned by the OP - makes sense ?

> No, we are literally trying to find a use case where using a lower accuracy LLM makes sense for a vision task. They're reconfigurable on the fly with little technical expertise and without training data, that's really useful. Personally in projects for people I've found models have fewer unusual edge cases than traditional models, are less sensitive to minor changes in input and are easier to debug by asking them w…

Seems like a way to use a sledgehammer to hammer in screws, and inviting nondeterminism in important systems. Besides being way larger and more complex than what most specialized industrial processes need, they are also vulnerable to adversarial attacks.

https://www.lakera.ai/blog/visual-prompt-injections

https://www.theverge.com/2021/3/8/22319173/openai-machine-vi...

Re: Qwen3.5 Fine-Tuning Guide

#113

Post by andriy_koval »

prettyblocks wrote: post #74
andriy_koval wrote: post #54

Earlier quoted context omitted.

> "Frontier LLMs can do it with enough context" is not really a strong argument against fine-tuning, because they're expensive to run. I am not expert in this topic, but I am wondering if large cached context is actually cheap to run and frontier models would be cost efficient too in such setting?

I'd like to read more about that if anyone has any suggestions.

I am not expert in this topic, but its easy to observe that price for cached tokens is usually 10x cheaper on major providers.

Re: Qwen3.5 Fine-Tuning Guide

#114

Post by dehrmann »

thot_experiment wrote: post #75
dehrmann wrote: post #72

Earlier quoted context omitted.

Naive question, but could neural networks handle these use cases?

NTA but almost certainly, the advantage is that Qwen3.5 is extremely generic already so adapting it to a specific task is way easier than training a NN from scratch. It's probably akin to how OCR is now just something I use Qwen for even though I have access to dedicated OCR tools, Qwen is good enough and it's already in my vram. Modern VLLMs are pretty great at answering basic questions about an image by default and…

OCR as a use case for LLMs cracks me up because traditional NN OCR is probably more accurate, but significantly more efficient. I figured people would chain it with LLMs to fix misscans.

Re: Qwen3.5 Fine-Tuning Guide

#115

Post by IanCal »

sorenjan wrote: post #112
IanCal wrote: post #109

Earlier quoted context omitted.

> No, we are literally trying to find a use case where using a lower accuracy LLM makes sense for a vision task. They're reconfigurable on the fly with little technical expertise and without training data, that's really useful. Personally in projects for people I've found models have fewer unusual edge cases than traditional models, are less sensitive to minor changes in input and are easier to debug by asking them w…

Seems like a way to use a sledgehammer to hammer in screws, and inviting nondeterminism in important systems. Besides being way larger and more complex than what most specialized industrial processes need, they are also vulnerable to adversarial attacks. https://www.lakera.ai/blog/visual-prompt-injections https://www.theverge.com/2021/3/8/22319173/openai-machine-vi...

> Seems like a way to use a sledgehammer to hammer in screws

The lazy analogy the other way is that developing a custom system to do these jobs is like hiring a team of experts to spend 2 years designing the perfect crosshead screwdriver that fits exactly one screw (and doesn't work if the screw starts slightly rotated) when you have a flathead one right next to you that'll work and it'll work right now.

> and inviting nondeterminism in important systems.

Traditional ML is just as non-deterministic.

> they are also vulnerable to adversarial attacks.

Typically not relevant in these kinds of cases but also this is easily a problem in many traditional ML algos.

Have you worked on things like this?

Re: Qwen3.5 Fine-Tuning Guide

#116

Post by sorenjan »

IanCal wrote: post #115
sorenjan wrote: post #112

Earlier quoted context omitted.

Seems like a way to use a sledgehammer to hammer in screws, and inviting nondeterminism in important systems. Besides being way larger and more complex than what most specialized industrial processes need, they are also vulnerable to adversarial attacks. https://www.lakera.ai/blog/visual-prompt-injections https://www.theverge.com/2021/3/8/22319173/openai-machine-vi...

> Seems like a way to use a sledgehammer to hammer in screws The lazy analogy the other way is that developing a custom system to do these jobs is like hiring a team of experts to spend 2 years designing the perfect crosshead screwdriver that fits exactly one screw (and doesn't work if the screw starts slightly rotated) when you have a flathead one right next to you that'll work and it'll work right now. > and inviti…

A flathead screwdriver is not a valid analogy, because LLMs are big complicated and opaque machines. And while other ML methods are non-deterministic as well, gaussian process, decision trees or even CNNs are easier to try to make sense of than these huge black boxes.

And I still haven't seen a single example of anyone actually using a finetuned Qwen in industrial inspection, which leads me to believe than nobody is actually using it for that, but some people want to use it because it's their new favorite toy. You don't need a VLM to count cells in microscopy images, or find scratches in painted parts, or estimate output from a log in a saw mill. I can see the use case for things like describing a scene from a surveillance camera, finding a car of a certain model and colour, or other tasks that demand more reasoning or description. But in those cases latency is not super important compared to getting the right output, which was the tradeoff discussed from the start of this thread.

The last thing I'd want to deal with is to have a computer say something like "You're absolutely right, it was wrong of me to classify the metal debris as food".

Re: Qwen3.5 Fine-Tuning Guide

#117

Post by IanCal »

sorenjan wrote: post #116
IanCal wrote: post #115

Earlier quoted context omitted.

> Seems like a way to use a sledgehammer to hammer in screws The lazy analogy the other way is that developing a custom system to do these jobs is like hiring a team of experts to spend 2 years designing the perfect crosshead screwdriver that fits exactly one screw (and doesn't work if the screw starts slightly rotated) when you have a flathead one right next to you that'll work and it'll work right now. > and inviti…

A flathead screwdriver is not a valid analogy, because LLMs are big complicated and opaque machines. And while other ML methods are non-deterministic as well, gaussian process, decision trees or even CNNs are easier to try to make sense of than these huge black boxes. And I still haven't seen a single example of anyone actually using a finetuned Qwen in industrial inspection, which leads me to believe than nobody is…

I’ve used multimodal LLMs for this sort of task and if a fine tuned model would get reasonable performance compared to frontier models I’d use that. Running things purely locally lets you massively simplify the overall architecture and data transfer requirements of some of these tasks if nothing else and lower latency means you can report problems much faster (vs transfer images off device, batch process).

> The last thing I'd want to deal with is to have a computer say something like "You're absolutely right, it was wrong of me to classify the metal debris as food".

The cnn will do that potentially more often and it can be because it’s just not seen enough examples of the debris at that angle or something else equally irrelevant to a human.

Re: Qwen3.5 Fine-Tuning Guide

#118

Post by faxmeyourcode »

arkmm wrote: post #48
faxmeyourcode wrote: post #40

Earlier quoted context omitted.

Especially for super constrained applications. I don't care if the language model that I use for my extremely specific business domain can solve PhD math or remember the works of Shakespeare. I'd trade all of that for pure task specific accuracy.

Can you share more details about your use case? The good applications of fine tuning are usually pretty niche, which tends to make people feel like others might not be interested in hearing the details. As a result it's really hard to read about real-world use cases online. I think a lot of people would love to hear more details - at least I know I would!

If you treat LLMs as generic transformers, you can fine tune with a ton of examples of input output pairs. For messy input data with lots of examples already built, this is ideal.

At my day job we have experimented with fine tuned transformers for our receipt processing workflow. We take images of receipts, run them through OCR (this step might not even be necessary, but we do it at scale already anyways), and then take the OCR output text blobs and "transform" them into structured receipts with retailer, details like zip code, transaction timestamps, line items, sales taxes, sales, etc.

I trained a small LLM (mistral-7b) via SFT with 1000 (maybe 10,000? I don't remember) examples from receipts in our database from 2019. When I tested the model on receipts from 2020 it hit something like 98% accuracy.

The key that made this work so well is that we had a ton of data (potentially billions of example input/output pairs) and we could easily evaluate the correctness by unpacking the json output and comparing with our source tables.

Note that this isn't running in production, it was an experiment. There are edge cases I didn't consider, and there's a lot more to it in terms of accurately evaling, when to re-train, dealing with net new receipt types, retailers, new languages (we're doing global expansion RN so it's top of mind), general diversity of edge cases in your training data, etc.

Post reply on HN