Is this multimodal? They don’t mention it anywhere but if I go to QwenChat I can use images with it.
Nope. Best for that at the moment is probably gemma3.
Qwen3: Think deeper, act faster
81–90 of 412 posts
Re: Qwen3: Think deeper, act faster
#82>Hybrid Thinking Modes This is what gpt-5 was supposed to have right? How is this implemented under the hood? Since non-thinking mode is just an empty chain-of-thought, why can't any reasoning model be used in a "non-thinking mode"?
[deleted]
Re: Qwen3: Think deeper, act faster
#83The biggest announcement of LlamaCon week!
Re: Qwen3: Think deeper, act faster
#84Earlier quoted context omitted.
curious, why the 30b MoE over the 32b dense for local coding? I do not know much about the benchmarks but the two coding ones look similar.
The MoE version with 3b active parameters will run significantly faster (tokens/second) on the same hardware, by about an order of magnitude (i.e. ~4t/s vs ~40t/s)
> The MoE version with 3b active parameters
~34 tok/s on a Radeon RX 7900 XTX under today's Debian 13.
Re: Qwen3: Think deeper, act faster
#85These performance numbers look absolutely incredible. The MoE outperforms o1 with 3B active parameters? We're really getting close to the point where local models are good enough to handle practically every task that most people need to get done.
> We're really getting close to the point where local models are good enough to handle practically every task that most people need to get done.
After trying to implement a simple assistant/helper with GPT-4.1 and getting some dumb behavior from it, I doubt even proprietary models are good enough for every task.
Re: Qwen3: Think deeper, act faster
#86Earlier quoted context omitted.
It is also possible that this "world view tuning" may have just been the manifestation of how these models gained public attention. Whether intentional or not, seeing the Tiananmen Square reposts across all social feeds may have done more to spread awareness of these models technical merits than the technical merits themselves would have. This is certainly true for how consumers learned about free Deepseek and fit pe…
I'm curious if there's any data to come to that conclusion, its hard for me to do "They did the censor training to DeepSeek because they knew consumers would love free DeepSeek after seeing screenshots of Tiananmen censorship in screenshots of DeepSeek" (the steelman here, ofc, is "the screenshots drove buzz which drove usage!", but it's sort of steel thread in context, we'd still need to pull in a time machine and a…
> Whether intentional or not
I am not claiming it was intentional, but it certainly magnified the media attention. Maybe luck and not 4d chess.
Re: Qwen3: Think deeper, act faster
#87Gotta love how Claude is always conventiently left out of all of these benchmark lists. Anthropic really is in a league of their own right now.
I'm actually finding Claude 3.7 to be a huge step down from 3.5. I dislike it so much I actually stopped using Claude altogether...
Re: Qwen3: Think deeper, act faster
#88With all the different open-weight models appearing, is there some way of figuring out what model would work with sensible speed (> X tok/s) on a standard desktop GPU ? I.e. I have Quadro RTX 4000 with 8G vram and seeing all the models https://ollama.com/search here with all the different sizes, I am absolutely at loss which models with which sizes would be fast enough. I.e. there is no point of me downloading the la…
With 8GB VRAM, I would try this one first:
https://ollama.com/library/qwen3:8b-q4_K_M
For fast inference, you want a model that will fit in VRAM, so that none of the layers need to be offloaded to the CPU.
Re: Qwen3: Think deeper, act faster
#89Earlier quoted context omitted.
There are a lot of variables here such as your hardware's memory bandwidth, speed at which at processes tensors etc. A basic thing to remember: Any given dense model would require X GB of memory at 8-bit quantization, where X is the number of params (of course I am simplifying a little by not counting context size). Quantization is just 'precision' of the model, 8-bit generally works really well. Generally speaking,…
4 bit is absolutely fine . I know this is crazy to here because the big iron folks still debate 16 vs 32 and 8 vs 16 is near verboten in public conversation. I contribute to llama.cpp and have seen many many efforts to measure evaluation perf of various quants, and no matter which way it was sliced (ranging from subjective volunteers doing A/B voting on responses over months, to objective object perplexity loss) Q4 i…
Just for some callibration: approx. no one runs 32 bit for LLMs on any sort of iron, big or otherwise. Some models (eg DeepSeek V3, and derivatives like R1) are native FP8. FP8 was also common for llama3 405b serving.
Re: Qwen3: Think deeper, act faster
#90Something that interests me about the Qwen and DeepSeek models is that they have presumably been trained to fit the worldview enforced by the CCP, for things like avoiding talking about Tiananmen Square - but we've had access to a range of Qwen/DeepSeek models for well over a year at this point and to my knowledge this assumed bias hasn't actually resulted in any documented problems from people using the models. Asid…
The avoiding talking part is more on the Frontend level censorship I think. It doesn't censor on API
This is NOT true. At least on the 1.5B version model on my local machine. It blocks answers when using offline mode. Perplexity has an uncensored a version, but don't thing it is open on how they did it.