Good results fine tuning a local LLM like Qwen 3:0.6B to categorize questions

Post by nl » Tue, Jun 23, 2026, 12:25 AM UTC

dev-experiments wrote: post #48

nl wrote: post #21

Earlier quoted context omitted.

Hmm maybe. Turns out the author trained a logistic-regression classifier on the embeddings too, but didn't report the results: https://github.com/thelgevold/fine-tuned-classifier/blob/mai...

Expanding on this experiment using logistic regression is an interesting continuation, detailed here: https://www.teachmecoolstuff.com/viewarticle/using-logistic-... In summary: Using logistic regression actually improves accuracy, but also performance during both runtime and during training.

Post by nextaccountic » Tue, Jun 23, 2026, 3:43 AM UTC

nl wrote: post #8

nextaccountic wrote: post #7

> The model invents new categories (e.g. apartments) and doesn’t stick to the provided list of allowed categories Can this specific failure mode be solved by providing a grammar that the output must adhere to? (Not sure if Qwen has this feature, it's used for eg. to ensure the output is parseable json)

It can. It's something that is implemented by the thing that runs the model - eg Llama.cpp - rather than the model itself. Note that it is hard to make work if you turn thinking on because the grammar gets complicated quickly (I don't recall if Qwen 0.6B can do thinking).