Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

S1: A $6 R1 competitor?

timkellogg.me

Post reply on HN
Print view View original

71–80 of 430 posts

Re: S1: A $6 R1 competitor?

#71

Post by jebarker »

pas wrote: post #70
jebarker wrote: post #66

S1 (and R1 tbh) has a bad smell to me or at least points towards an inefficiency. It's incredible that a tiny number of samples and some inserted tokens can have such a huge effect on model behavior. I bet that we'll see a way to have the network learn and "emerge" these capabilities during pre-training. We probably just need to look beyond the GPT objective.

can you please elaborate on the wait tokens? what's that? how do they work? is that also from the R1 paper?

The same idea is in both the R1 and S1 papers ( tokens are used similarly). Basically they're using special tokens to mark in the prompt where the LLM should think more/revise the previous response. This can be repeated many times until some stop criteria occurs. S1 manually inserts these with heuristics, R1 learns the placement through RL I think.

Re: S1: A $6 R1 competitor?

#72

Post by naasking »

yapyap wrote: post #8

> If you believe that AI development is a prime national security advantage, then you absolutely should want even more money poured into AI development, to make it go even faster. This, this is the problem for me with people deep in AI. They think it’s the end all be all for everything. They have the vision of the ‘AI’ they’ve seen in movies in mind, see the current ‘AI’ being used and to them it’s basically almost t…

> This is machine learning except the machines are bigger than ever before. There is no intellect.

Define "intellect".

Re: S1: A $6 R1 competitor?

#73

Post by baxtr »

jxdxbx wrote: post #69
syntaxing wrote: post #61

Earlier quoted context omitted.

Matt Levine tangentially talked about this during his podcast this past Friday (or was it the one before?). It was a good way to value these companies according to their compute size since those chips are very valuable. At a minimum, the chips are an asset that acts as a collateral.

I hear this a lot, but what the hell. It's still computer chips. They depreciate. Short supply won't last forever. Hell, GPUs burn out. It seems like using ice sculptures as collateral, and then spring comes.

If so wouldn’t it be the first time in history when more processing power is not used?

In my experience CPU/GPU power is used up as much as possible. Increased efficiency just leads to more demand.

Re: S1: A $6 R1 competitor?

#76

Post by dleslie »

bloomingkales wrote: post #75

This thing that people are calling “reasoning” is more like rendering to me really, or multi pass rendering. We’re just refining the render, there’s no reasoning involved.

That was succinct and beautifully stated. Thank-you for the "Aha!" moment.

Re: S1: A $6 R1 competitor?

#77

Post by SecretDreams »

syntaxing wrote: post #61
gessha wrote: post #56

Earlier quoted context omitted.

This is pure speculation on my part but I think at some point a company's valuation became tied to how big their compute is so everybody jumped on the bandwagon.

Matt Levine tangentially talked about this during his podcast this past Friday (or was it the one before?). It was a good way to value these companies according to their compute size since those chips are very valuable. At a minimum, the chips are an asset that acts as a collateral.

> It was a good way to value these companies according to their compute size since those chips are very valuable.

Are they actually, though? Presently yes, but are they actually driving ROI? Or just an asset nobody really is meaningfully utilizing, but helps juice the stocks?

Re: S1: A $6 R1 competitor?

#79

Post by amelius »

ninetyninenine wrote: post #62
amelius wrote: post #51

Earlier quoted context omitted.

I'm not convinced. This is using the tooling and paradigms invented by humans.

Science is a paradigm invented by humans. If a human uses this paradigm to invent something he is considered intelligent but if an ai uses it it’s not? All humans use human paradigms and they are intelligent. If a human uses such a paradigm to success it is the same.

Biology is a field where you can find new things relatively easily, if you follow existing protocols and do the hard lab work (which costs time and money).

Re: S1: A $6 R1 competitor?

#80

Post by mistermann »

bloomingkales wrote: post #75

This thing that people are calling “reasoning” is more like rendering to me really, or multi pass rendering. We’re just refining the render, there’s no reasoning involved.

"...there’s no reasoning involved...wait, could I just be succumbing to my heuristic intuitions of what is (seems to be) true....let's reconsider using System 2 thinking..."
Post reply on HN