Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

S1: A $6 R1 competitor?

timkellogg.me

Post reply on HN
Print view View original

61–70 of 430 posts

Re: S1: A $6 R1 competitor?

#61

Post by syntaxing »

gessha wrote: post #56
swiftcoder wrote: post #22

> having 10,000 H100s just means that you can do 625 times more experiments than s1 did I think the ball is very much in their court to demonstrate they actually are using their massive compute in such a productive fashion. My BigTech experience would tend to suggest that frugality went out the window the day the valuation took off, and they are in fact just burning compute for little gain, because why not...

This is pure speculation on my part but I think at some point a company's valuation became tied to how big their compute is so everybody jumped on the bandwagon.

Matt Levine tangentially talked about this during his podcast this past Friday (or was it the one before?). It was a good way to value these companies according to their compute size since those chips are very valuable. At a minimum, the chips are an asset that acts as a collateral.

Re: S1: A $6 R1 competitor?

#62

Post by ninetyninenine »

amelius wrote: post #51
mrshadowgoose wrote: post #32

Earlier quoted context omitted.

Here you go: https://www.biorxiv.org/content/10.1101/2024.11.11.623004v1

I'm not convinced. This is using the tooling and paradigms invented by humans.

Science is a paradigm invented by humans. If a human uses this paradigm to invent something he is considered intelligent but if an ai uses it it’s not? All humans use human paradigms and they are intelligent.

If a human uses such a paradigm to success it is the same.

Re: S1: A $6 R1 competitor?

#63

Post by robwwilliams »

sidewndr46 wrote: post #34
yapyap wrote: post #8

> If you believe that AI development is a prime national security advantage, then you absolutely should want even more money poured into AI development, to make it go even faster. This, this is the problem for me with people deep in AI. They think it’s the end all be all for everything. They have the vision of the ‘AI’ they’ve seen in movies in mind, see the current ‘AI’ being used and to them it’s basically almost t…

What is even the possible usage of AI for national security? Generating pictures of kittens riding nuclear weapons to the very end like in Dr Strangelove?

Lol: Where I live (Memphis) both “one” and “two” are considered two syllable words. Seriously. Our kids were taught this in the best public elementary school.

Re: S1: A $6 R1 competitor?

#65

Post by TehCorwiz »

sambull wrote: post #55

That sovereign wealth fund with tik tok might set a good precedent; when we have to 'pour money' into these companies we can do so with stake in them held in our sovereign wealth fund.

Extra-legal financial instruments meant to suck money from other federal departments don't strike me as a good precedent in any sense. I don't disagree though that nationalizing the value of enormous public investments is something we should be considering, looking at you oil industry. But until congress appropriates the money under law it's a pipe dream or theft.

Re: S1: A $6 R1 competitor?

#66

Post by jebarker »

S1 (and R1 tbh) has a bad smell to me or at least points towards an inefficiency. It's incredible that a tiny number of samples and some inserted tokens can have such a huge effect on model behavior. I bet that we'll see a way to have the network learn and "emerge" these capabilities during pre-training. We probably just need to look beyond the GPT objective.

Re: S1: A $6 R1 competitor?

#67

Post by davely »

mettamage wrote: post #35
anentropic wrote: post #26

Earlier quoted context omitted.

and: https://github.com/simplescaling/s1

When you're only used to ollama, how do I go about using this model?

I think we need to wait for someone to convert it into a GGUF file format.

However, once that happens, you can run it (and any GGUF model) from Hugging Face![0]

[0] https://huggingface.co/docs/hub/en/ollama

Re: S1: A $6 R1 competitor?

#68

Post by light_hue_1 »

S1 has no relationship to R1. It's a marketing campaign for an objectively terrible and unrelated paper.

S1 is fully supervised by distilling Gemini. R1 works by reinforcement learning with a much weaker judge LLM.

They don't follow the same scaling laws. They don't give you the same results. They don't have the same robustness. You can use R1 for your own problems. You can't use S1 unless Gemini works already.

We know that distillation works and is very cheap. This has been true for a decade; there's nothing here.

S1 is a rushed hack job (they didn't even run most of their evaluations with an excuse that the Gemini API is too hard to use!) that probably existed before R1 was released and then pivoted into this mess.

Re: S1: A $6 R1 competitor?

#69

Post by jxdxbx »

syntaxing wrote: post #61
gessha wrote: post #56

Earlier quoted context omitted.

This is pure speculation on my part but I think at some point a company's valuation became tied to how big their compute is so everybody jumped on the bandwagon.

Matt Levine tangentially talked about this during his podcast this past Friday (or was it the one before?). It was a good way to value these companies according to their compute size since those chips are very valuable. At a minimum, the chips are an asset that acts as a collateral.

I hear this a lot, but what the hell. It's still computer chips. They depreciate. Short supply won't last forever. Hell, GPUs burn out. It seems like using ice sculptures as collateral, and then spring comes.

Re: S1: A $6 R1 competitor?

#70

Post by pas »

jebarker wrote: post #66

S1 (and R1 tbh) has a bad smell to me or at least points towards an inefficiency. It's incredible that a tiny number of samples and some inserted tokens can have such a huge effect on model behavior. I bet that we'll see a way to have the network learn and "emerge" these capabilities during pre-training. We probably just need to look beyond the GPT objective.

can you please elaborate on the wait tokens? what's that? how do they work? is that also from the R1 paper?
Post reply on HN