Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2

magazine.sebastianraschka.com

Post reply on HN
Print view View original

81–90 of 99 posts

Re: GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2

#81

Post by regularfry »

starchild3001 wrote: post #65

What stood out to me is how much of gpt-oss’s “newness” isn’t about radical architectural departures, but about a careful layering of well-understood optimizations—RoPE, SwiGLU, GQA, MoE—with some slightly unusual choices (tiny sliding-window sizes, few large experts instead of many small ones, per-head attention sinks). The MXFP4 quantization detail might be the sleeper feature here. Getting 20B running on a 16 GB c…

> will we start seeing a formal split in open-weight model development—specialized “reasoners” that rely on tool use for facts, and “knowledge bases” tuned for retrieval-heavy work?

My bet's on the former winning outright. It's very hard to outrun a good search engine, LLMs are inherently lossy so internal recall will never be perfect, and if you don't have to spend your parameter budget encoding information then you get to either spend that budget on being a much better reasoner, or you shrink the model and make it cheaper to run for the same capability. The trade-off is a more complex architecture, but that's happening anyway.

Re: GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2

#82

Post by victorbjorklund »

dzogchen wrote: post #80

Say it with me: freely downloadable model weights does not mean a model is open source. https://opensource.org/ai/open-source-ai-definition

Yeah but isn't it even doubtful if you can call a model a program or is it rather like a data set that can be used by a program.

Re: GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2

#83

Post by ethan_smith »

starchild3001 wrote: post #65

What stood out to me is how much of gpt-oss’s “newness” isn’t about radical architectural departures, but about a careful layering of well-understood optimizations—RoPE, SwiGLU, GQA, MoE—with some slightly unusual choices (tiny sliding-window sizes, few large experts instead of many small ones, per-head attention sinks). The MXFP4 quantization detail might be the sleeper feature here. Getting 20B running on a 16 GB c…

MXFP4's mixed precision approach (4-bit for weights, higher precision for KV cache) actually offers better accuracy/size tradeoffs than competing quantization methods like GPTQ or AWQ, which is why it enables these impressive resource profiles without the typical 4-bit degradation.

Re: GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2

#84

Post by eurekin »

faangguyindia wrote: post #56
7moritz7 wrote: post #3

Qwen3 is substantially better in my local testing. As in, adheres to the prompt better (pretty much exactly for the 32B parameter variant, very impressive) and is more organic sounding. In simplebench gpt-oss (120 bn) flopped hard so it doesn't appear particularly good at logical puzzles either. So presumably, this comes down to... - training technique or data - dimension - lower number of large experts vs higher num…

yesterday, i signed up for qwen3-coder-plus. It fails 4/10 "diff" edit format in various code editing tools i use. Gemini Pro 2.5 with diff fenced edit format, rarely fails. So i don't see this Qwen3 hype unless i am using wrong edit format, can anyone tell me which edit format will work better with Qwen3? https://aider.chat/docs/more/edit-formats.html

I'm running a30-a3b-instruct q6 quant on exllamav2 and checked few simple tasks in roo and cline. Prompt adherence, tool calling and file changing worked flawlessly

Re: GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2

#85

Post by LarMachinarum »

poorman wrote: post #40

This article really goes into a lot of detail which is nice. gpt-oss is just not good for agentic use in my observation. tldr; I'll save you a lot of time trying things out for yourself. If you are on a >=32 GB Mac download LMStudio and then the `qwen3-coder-30b-a3b-instruct-mlx@5bit` model. It uses ~20 GB of RAM so a 32GB machine is plenty. Set it up with opencode [1] and you're off to the races! It has great tool c…

Much as I understand how a 5 bit quantization might be a sweet spot in the tradeoff between precision and making it possible to cram more weight parameters into limited ram, and thus in that respect better than e.g. 4 bit or 8 bit,…

…I struggle to comprehend how an odd quantization like 5 bit, that doesn't align well with 8 bit boundaries, would not slow things down for inference: given that on one hand the hardware doing the multiplications doesn't support vectors of 5 bit values but needs repacking to 8 bit before multiplication, and on the other hand the weights can't be bulk-repacked to 8 bit once and for all in advance (otherwise it wouldn't fit inside the RAM, besides in that case one would use a 8 bit quantization anyways)

it would require quite a lot of instructions per multiplication (way more than for 4 bit quantization where the alignment match simplifies things) to ad-hoc repack the 5 bit values to vectors of 8 bit. So i kinda wonder how much (percentage-wise) that would impact inference performance

Re: GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2

#86

Post by throw-qqqqq »

LarMachinarum wrote: post #85
poorman wrote: post #40

This article really goes into a lot of detail which is nice. gpt-oss is just not good for agentic use in my observation. tldr; I'll save you a lot of time trying things out for yourself. If you are on a >=32 GB Mac download LMStudio and then the `qwen3-coder-30b-a3b-instruct-mlx@5bit` model. It uses ~20 GB of RAM so a 32GB machine is plenty. Set it up with opencode [1] and you're off to the races! It has great tool c…

Much as I understand how a 5 bit quantization might be a sweet spot in the tradeoff between precision and making it possible to cram more weight parameters into limited ram, and thus in that respect better than e.g. 4 bit or 8 bit,… …I struggle to comprehend how an odd quantization like 5 bit, that doesn't align well with 8 bit boundaries, would not slow things down for inference: given that on one hand the hardware…

> I struggle to comprehend how an odd quantization like 5 bit, that doesn't align well with 8 bit boundaries, would not slow things down for inference

Who says it doesn’t :)?

At least in my tests there is a big penalty to using an “odd” bit stride.

Testing 4bit quantization vs 5bit in Llama.cpp, I see quite a bit more than the “naiively expected” 25% slowdown from 4 to 5 bits.

Re: GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2

#87

Post by faangguyindia »

eurekin wrote: post #84
faangguyindia wrote: post #56

Earlier quoted context omitted.

yesterday, i signed up for qwen3-coder-plus. It fails 4/10 "diff" edit format in various code editing tools i use. Gemini Pro 2.5 with diff fenced edit format, rarely fails. So i don't see this Qwen3 hype unless i am using wrong edit format, can anyone tell me which edit format will work better with Qwen3? https://aider.chat/docs/more/edit-formats.html

I'm running a30-a3b-instruct q6 quant on exllamav2 and checked few simple tasks in roo and cline. Prompt adherence, tool calling and file changing worked flawlessly

okay turns out i was using it in aider with wrong edit format in editor mode, i switched to editor edit format it has not failed so far.

Re: GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2

#88

Post by kgeist »

littlestymaar wrote: post #74
Mars008 wrote: post #60

Earlier quoted context omitted.

You call it fair? 32 / 5.1 > 6, it's takes 6 times more to compute each token. Put it other way, Qwen3 32B is 6 times slower than GPT OSS 120B.

I've read many times that MoE models should be comparable to dense models with a number of parameters equal to the geometric mean of the MoE's total number of parameters and active ones. In the case of gpt-oss 120B that would means sqrt(5*120)=24B.

I've read that the formula is based on the early Mistral models and does not necessarily reflect what's going on nowadays.

Re: GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2

#89

Post by imtringued »

mark_l_watson wrote: post #15

Wow, Sebastian Raschk's blog articles are jewels - much appreciated. I use the get-oss and qwen3 models a lot (smaller models locally using Ollama and LM Studio) and commercial APIs for the full size models. For local model use, I get very good results with get-oss when I "over prompt," that is, I specify a larger amount of context information than I usually do. Qwen3 is simply awesome. Until about three years ago, I…

For me it is the opposite. I'm shocked by how simple transformer based models and how small the architectural differences are between the latest models. Almost nothing has changed since late 2023.

Re: GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2

#90

Post by littlestymaar »

NitpickLawyer wrote: post #75
littlestymaar wrote: post #73

Earlier quoted context omitted.

> careful layering of well-understood optimizations—RoPE, SwiGLU, GQA, MoE They basically cloned Qwen3 on that, before adding the few tweaks you mention afterwards.

> They basically cloned Qwen3 on that Oh, come on! GPT4 was rumoured to be an MoE well before Qwen even started releasing models. oAI didn't have to "clone" anything.

First, it would be great if people stopped top acting as if those billion-dollar corporations where sport teams.

Second, I don't claim OpenAI have to clone anything, and I have no reason to believe that their proprietary models are copying other people's ones. But for this particular open weight models, they clearly have an incentive to use exactly the same architectural base as another actor's, in order to avoid leaking too much information about their own secret sauce.

And finally, though GPT-4 was a MoE it was most likely what TFA calls “early MoE” with a few very big experts, not many small ones.

Post reply on HN