Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

arxiv.org

Post reply on HN
Print view View original

181–190 of 226 posts

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#181

Post by rapatel0 »

achrono wrote: post #164

Beats Opus 4.5 on reasoning you say? Prompt: If A goes to B who then goes to C, can A send something to C? Response: We need to interpret best. The phrase "If A goes to B who then goes to C, can A send something to C?" could be a puzzle about the concept of sending something (like passing a ball) and the relationships. Scenario: A gives something to B, and B passes it on to C. Question: Can A also give the same thing…

Ran the same query and there is a ton of stuff, but it looks like it's reasoning through the ambiguity of the sentence. It still gets the right answer. Moreover, if we consider the FLOPs expended to get to the answer, and compare that to opus, I think it's still a net win.

My hunch is that Opus scale models probably have shortcuts encoded into the model that handle these ambiguities cases, wheres this model has learned a program to reason through the edge case (crystalized vs fluid intelligence). Remembering that probablity (frontier) vs calculating it on the fly (vibethink)

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#183

Post by bakies »

tarruda wrote: post #169
bakies wrote: post #110

Earlier quoted context omitted.

I've just started using qwen3.6:35b a couple days ago running on my framework desktop and rather impressed. It runs really well and reminds me of probably the first Claude model I used. It's the first local model that's actually working for me in a coding agent I've tried. Very exciting!

If your framework desktop is the 128G Strix Halo, I recommend giving Qwen 3.5 122B-A10B a shot. This Q5_K_M quant should be near lossless and fit with full 256K context in about 100GB of RAM: https://huggingface.co/AesSedai/Qwen3.5-122B-A10B-GGUF

I'll keep this in mind.

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#184

Post by kristjansson »

secretslol wrote: post #23

Am I right in thinking this is a tiny model which has been trained well to reason, and that's it? Makes me think of a smart person who doesn't know anything about a given topic, but with the right tools will go and research the heck out of it. I really like the sound of this... why have models train on learning anything when you can just train them how to learn and let them get on with it from something as small as a…

other way around. it's trained to generate long CoT to reason through problems (and does it well!) but has ~no tool calling capability, and ~no ability to manage more than 1-2 messages.

see the warning at the top of https://huggingface.co/WeiboAI/VibeThinker-3B

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#185

Post by brainless »

SubiculumCode wrote: post #151
brainless wrote: post #67

I recently came across this model and I would love to try it with my coding agent soon. I really like the idea of small models that can reason but do not have too much knowledge. Also, no emphasis on tool calls. I think the agent should do the heavy lifting and reach half way. I use really small models, like Qwen 3.5 0.8B to 9B - no tool calling, no MCP, no skills, nothing. No multi-turn chat even. Models are given v…

Maybe no tool calling, but seems it could be really good at deciding which tool to use and when?

That is a good point. I do think these models would be good in the decision making. The large models are trained to use tool calling. Perhaps the small models can generate the text that would express their decision but not generate good JSON to reply with correct syntax. I do not know but this is my hunch.

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#186

Post by dominotw »

secretslol wrote: post #23

Am I right in thinking this is a tiny model which has been trained well to reason, and that's it? Makes me think of a smart person who doesn't know anything about a given topic, but with the right tools will go and research the heck out of it. I really like the sound of this... why have models train on learning anything when you can just train them how to learn and let them get on with it from something as small as a…

> Am I right in thinking this is a tiny model which has been trained well to reason, and that's it?

i remember karpathy mentioning in dwarkesh podcast. But is reasoning really possible without all the knowledge.

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#187

Post by noperator »

dummydummy1234 wrote: post #15
noperator wrote: post #8

Having some success while testing this model out as a replacement for GPT-5 nano in source code security review. Running on RTX 3090 (24 GB VRAM) via vLLM. It's not great on structured output (as noted in the model card) but I'm working around that in my harness.

Can't you just force it to do structured output via constrained generation?

Yes, I did end up figuring out a clean way to allow normal reasoning inside and then force JSON _after_ the closing . Example here: https://gist.github.com/noperator/6c711ab19027ea8056442df839...

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#188

Post by varispeed »

swiftcoder wrote: post #45
deftio wrote: post #16

There is some base level of intelligence any model needs to be useful, even in narrow tasks. Could you teach a 5 year old to drive a car? A 10 year old? A 12 year old? To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball. By the time a human in in their mid teens they have acquired the base knowledge... Small models need to have enough…

> To drive a car requires being able to read Emphatically, it does not. Passing your drivers test may require being able to read, but plenty of illiterate people around the world drive just fine. There is a reason we made all the common road signs recognisable purely by shape/colour, after all.

I don't think many drivers pay too much attention to signs apart from traffic lights.

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#189

Post by gertlabs »

ignoramous wrote: post #123
smcleod wrote: post #117

Earlier quoted context omitted.

Try 27b, it's significantly smarter than 35b-a3b (although it is slower, it's not so bad with MTP).

At least according to gertlabs, Qwen3.6 27B outperforms every SoTA (closed) model at Kotlin: https://archive.vn/RYBCL / https://gertlabs.com/rankings?mode=agentic_coding&language=k...

Qwen 3.6 27B is an anomalously strong all-around model for its size, but when we run our evaluations, we generate 10 coding submissions/language/model (110 total). So full discosure, the per-language per-model performances can be noisy (I do not think Qwen3.6 27B is better than Fable 5 in agentic workflows when writing Kotlin, given enough samples, although we do find some interesting anomalies that hold up under large sample sizes).

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#190

Post by supern0va »

dominotw wrote: post #186
secretslol wrote: post #23

Am I right in thinking this is a tiny model which has been trained well to reason, and that's it? Makes me think of a smart person who doesn't know anything about a given topic, but with the right tools will go and research the heck out of it. I really like the sound of this... why have models train on learning anything when you can just train them how to learn and let them get on with it from something as small as a…

> Am I right in thinking this is a tiny model which has been trained well to reason, and that's it? i remember karpathy mentioning in dwarkesh podcast. But is reasoning really possible without all the knowledge.

Even Karpathy acknowledged that this would require some baseline of human knowledge. The idea wasn't pure logic/reasoning, but some subset to bootstrap from.
Post reply on HN