Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

VibeThinker-1.5B

github.com

Post reply on HN
Print view View original

11–16 of 16 posts

Re: VibeThinker-1.5B

#11

Post by impossiblefork »

Lapel2742 wrote: post #10

I'm pretty sure that this is some kind of scientific achievement that I do not fully understand but the real world use cases for this model seem to be very limited. I gave it two tasks. "Create a new and original story in 500 words" and "Write a Python console game". Both of those resulted in an endless loop with the model repeating itself I'm honest. Given that a 1B Granite nano model has only little problems (word…

It's specifically trained on maths. I don't think they care at all about general instruction following or stories.

Re: VibeThinker-1.5B

#12

Post by zhouyi2future »

Lapel2742 wrote: post #10

I'm pretty sure that this is some kind of scientific achievement that I do not fully understand but the real world use cases for this model seem to be very limited. I gave it two tasks. "Create a new and original story in 500 words" and "Write a Python console game". Both of those resulted in an endless loop with the model repeating itself I'm honest. Given that a 1B Granite nano model has only little problems (word…

[dead]

Re: VibeThinker-1.5B

#13

Post by Lapel2742 »

impossiblefork wrote: post #11
Lapel2742 wrote: post #10

I'm pretty sure that this is some kind of scientific achievement that I do not fully understand but the real world use cases for this model seem to be very limited. I gave it two tasks. "Create a new and original story in 500 words" and "Write a Python console game". Both of those resulted in an endless loop with the model repeating itself I'm honest. Given that a 1B Granite nano model has only little problems (word…

It's specifically trained on maths. I don't think they care at all about general instruction following or stories.

>> [...] achieving state-of-the-art performance in mathematical and *coding tasks* [...]

And it fails at one of the simplest coding tasks where a Granite model at nearly half the size has no problems.

It's probably an important discovery but seemingly only usable in an academic context.

Re: VibeThinker-1.5B

#14

Post by impossiblefork »

Lapel2742 wrote: post #13
impossiblefork wrote: post #11

Earlier quoted context omitted.

It's specifically trained on maths. I don't think they care at all about general instruction following or stories.

>> [...] achieving state-of-the-art performance in mathematical and *coding tasks* [...] And it fails at one of the simplest coding tasks where a Granite model at nearly half the size has no problems. It's probably an important discovery but seemingly only usable in an academic context.

The way I read the paper is that they've only tuned it on that maths dataset, so it's not made to have any coding ability.

Re: VibeThinker-1.5B

#15

Post by impossiblefork »

I don't quite understand the MRPO.

So during the final they try to ensure the model doesn't get the right answer every time, but only 50% of time, so as to avoid killing all variability-- very sensible, and then they compute a measure of this, take the negative exponential of this measure and then they scale the advantage by this.

So a question matters in proportion to the variability of the answers. Isn't this more curriculum learning stuff than actually suppressing things that don't vary enough?

Basically focusing on questions that are still hard instead of trying to push the probability of problem it's often able to solve to 99.99%?

Also very reasonable, but this isn't how they describe it. Instead, from their description I would think they're sort of forcing entropy to be high somehow.

I think the way I'd have titled it would be something like "Dynamic curricula to preserve model entropy".

Re: VibeThinker-1.5B

#16

Post by Balinares »

Alifatisk wrote: post #2

Does benchmarks look incredible. Like almost too good to be true, what am I missing? Is this hosted online somewhere so I can try it out?

I don't know how the coding benchmarks are computed but this model, on its own and outside of agentic loops, definitely doesn't compare to e.g. Qwen3 Coder. I might still try that for fun, just to see how it performs given a feedback loop.

On math questions, though, beside a marked tendency towards rambling thinking, it's just plain implausibly good for a 1.5B model. This is probably just rote learning, though. Otherwise this might well be a breakthrough.

Post reply on HN