Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

arxiv.org

Post reply on HN
Print view View original

31–40 of 226 posts

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#31

Post by universa1 »

deftio wrote: post #16

There is some base level of intelligence any model needs to be useful, even in narrow tasks. Could you teach a 5 year old to drive a car? A 10 year old? A 12 year old? To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball. By the time a human in in their mid teens they have acquired the base knowledge... Small models need to have enough…

A 10 year old definitely,and 5year old is close, but not unrealistic, To drive a car you don't need to be able to read... To drive a car on the road with other people is a whole other story :-)

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#32

Post by satvikpendem »

deftio wrote: post #16

There is some base level of intelligence any model needs to be useful, even in narrow tasks. Could you teach a 5 year old to drive a car? A 10 year old? A 12 year old? To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball. By the time a human in in their mid teens they have acquired the base knowledge... Small models need to have enough…

While I agree with your assessment, probably could've chosen a better example, as in many countries young kids even as young as 8 will learn how to drive.

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#33

Post by [deleted] »

deftio wrote: post #16

There is some base level of intelligence any model needs to be useful, even in narrow tasks. Could you teach a 5 year old to drive a car? A 10 year old? A 12 year old? To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball. By the time a human in in their mid teens they have acquired the base knowledge... Small models need to have enough…

[deleted]

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#34

Post by 3eb7988a1663 »

universa1 wrote: post #31
deftio wrote: post #16

There is some base level of intelligence any model needs to be useful, even in narrow tasks. Could you teach a 5 year old to drive a car? A 10 year old? A 12 year old? To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball. By the time a human in in their mid teens they have acquired the base knowledge... Small models need to have enough…

A 10 year old definitely,and 5year old is close, but not unrealistic, To drive a car you don't need to be able to read... To drive a car on the road with other people is a whole other story :-)

I suspect plenty of five year olds can do a respectable job in Mario Kart, Gran Turismo, etc driving games. Gaming has too low of stakes to judge them on perfectly adhering to the rules of the road, but the ability is there.

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#35

Post by pants2 »

NotSuspicious wrote: post #29

The interesting thing about models this small is they should be able to be put on a single Taalas chip (the HC1 already runs a Llama 3.1 8B model). We're already at the point where half-decent reasoning could be run on an ASIC (and at mind-boggling speeds).

Yeah, if they can fit an 8B model that's really good at improving the output by thinking, running at 16K tok/s on Taalas would be mind-blowing.

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#36

Post by hypfer »

noperator wrote: post #8

Having some success while testing this model out as a replacement for GPT-5 nano in source code security review. Running on RTX 3090 (24 GB VRAM) via vLLM. It's not great on structured output (as noted in the model card) but I'm working around that in my harness.

> but I'm working around that in my harness.

How?

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#37

Post by threatripper »

deftio wrote: post #16

There is some base level of intelligence any model needs to be useful, even in narrow tasks. Could you teach a 5 year old to drive a car? A 10 year old? A 12 year old? To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball. By the time a human in in their mid teens they have acquired the base knowledge... Small models need to have enough…

Ask people who grew up on a farm in a rural area. Sometimes you have to even if you can't and you do.

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#38

Post by altmanaltman »

secretslol wrote: post #23

Am I right in thinking this is a tiny model which has been trained well to reason, and that's it? Makes me think of a smart person who doesn't know anything about a given topic, but with the right tools will go and research the heck out of it. I really like the sound of this... why have models train on learning anything when you can just train them how to learn and let them get on with it from something as small as a…

Yeah but don't you think like that's an oversimplication with the metaphor if we assume this model can do a smart human-level analysis and distillation of knowledge, no? I mean if that were true (i.e. its just like that) then yeah there is no need for massive models but I really would doubt that.

Even recent massive models do not work anything like a smart human does at the moment so why are we assuming this can?

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#39

Post by nsingh2 »

nsingh2 wrote: post #25
gslepak wrote: post #14

Note that these are Python-only results, the model will not do as well with other languages. I'm glad to see more domain-focused SLMs, we need more of them! A programming focused MoE should work well across many languages.

Lots of confusion about what this model is actually focused on. It is a cheap specialist for closed-world, verifiable reasoning tasks like math, self-contained coding problems, and similar. "Closed-world" means the needed information is already in the context. It is not a tool-using agent that can discover missing context. "Verifiable" means answers are hard to generate but easy to check. So no open ended research, r…

To follow up on this, I had it solve a nasty ODE problem that I saw in the recent Mathematica 15 release post:

    Solve the following first-order ODE for f(x):

    ((-1 - 2*x)*f(x)*tan(1 + x - exp(-61 - 2*x)*f(x)/x)
    + exp(61 + 2*x)*x*(1 - x*tan(1 + x - exp(-61 - 2*x)*f(x)/x))
    + x*tan(1 + x - exp(-61 - 2*x)*f(x)/x)*f'(x)) = 0

    Find the general solution f(x).
And surprisingly it found a valid solution! Extra impressive because it runs 25 tok/s on my measly RTX 2070 super.

    f(x) = x*exp(61 + 2*x)*(1 + x - arccos(C/x))

    C is an arbitrary constant.
Apparently Mathematica 14.3 couldn't solve this ODE.

Re: VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO

#40

Post by kristopolous »

aero2146 wrote: post #4

I tried generating the classic pelican svg, but it failed horribly just showing me a rectangle and a black circle...

try it again but give a careful explanation of what a bicycle and a pelican is and how the pelican would sit atop the bicycle. Then give it a reference to the SVG tags you want it to use with documentation.

Here's what I got

https://9ol.es/tmp/pelican.png

with https://9ol.es/tmp/prompt_pelican.txt

using prithivMLmods/VibeThinker-3B-GGUF:Q4_K_M

Post reply on HN