Qwen 3.8 27B

Compared to closed weight (especially unreleased and access-limited) and open weight/source large sparse MoE LLMs/VLMs, open weight/source small dense models benefits public the most because they just reaches more people.

Compared to Qwen 3.6, 3.8's thinking style changed drastically. With xhigh budget, it thinks a lot MORE, and longer thinking session directly translates to better performance. This tradeoff between performance and computation, memory, etc. is meaningful to me.

However, because Qwen 3.6 and 3.8 share the same architecture, with 32GB vram, llama.cpp, IQ4_XS model, MTP and FP16 mmproj, I can only get 200k context, which is not good compared to 640k context of muse glimmer. Hopefully this problem will be solved in Qwen 4.0 release.

I think the problem is that the conversation drifted from "all useful human knowledge" to "enough knowledge to do my job".

Both a human and a current large model will bring up unprompted some tax regulation that applies in your district that could matter to your task. A small model won't know the tax code of every city in the world, as it's probably impossible to fit, and it's the kind of thing that unless you already know about, it's very hard to search for, let alone come up with unless specifically prompted about something tax related.

Unless we start having different small models trained for a certain profession for a certain city, I think we'll need those trillion parameters.

#1: it does not require deep world knowledge, because that's not what local models are for.

#2: it directly attacks drive-by understanding, overly linear processing training, poor attention mechanisms, poor reasoning patterns or lazy assumptions that ignore very easy low hanging fruit.

#3: it requires solid instruction following in the face of errors. a lot of models will run into errors and then fall back into some kind of error recovery process that bypasses instruction following.

#4: does not require prompt fine tuning to tweak to each individual model. they all seem to understand.

#5: not unfair. almost every model demonstrates in their reasoning that they have the necessary information that if reasoned about appropriately, could arrive at the correct answer.

#6: not designed to add unnecessary complication that it is intended to exhaust reasoning budgets of any sort, so it is not inherently unfair to models that reason a little more or less. for example, it does not require unnecessary reasoning soaks (ie: hiding the prompt inside base-64 encoding)

#7: has real world use and is probably applicable to overall ability to generalize.

#8: can be scaled up as models get better.

#9: is a very good indicator of how bad a model is falling apart under various inference settings.

In general it's kind of a benchmaxxing/distillation (I don't mean that pejoratively here, read on) artifact where test time compute's purpose is to refine/zero-in on a particular input:output pair. Basically they're trying to "remember"/re-derive the answer rather than arrive at it deductively - it gets comical/absurd when you see it expending a huge amount of tokens trying to figure out the answer to some trivial question or response to some input, as if it were a trick question or the choice of wording was of the utmost importance. But it's not a bad thing when the output matters or the task is hard: basically the model has been trained to respond/act like a much smarter model and it's probably better for it to overgeneralize that behavior.

Also, IME Gemma and most other local models that support thinking tend to have the same problem, and it's partly only a "problem" because they give you access to the thinking tokens themselves (which you don't in many cases see when working with frontier american models) and you can actually see how they're getting spent. For the local or bulk (runnin on owned or rented hardware) use case where you are paying in time/watts rather than purely by the token, IMO it's a good idea to just look at the task success rate / time per task and not worry about the raw thinking token count.