Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

S1: A $6 R1 competitor?

timkellogg.me

Post reply on HN
Print view View original

321–330 of 430 posts

Re: S1: A $6 R1 competitor?

#321

Post by bloomingkales »

Valgrim wrote: post #253
spiorf wrote: post #203

Earlier quoted context omitted.

We know how the next token is selected, but not why doing that repeatedly brings all the capabilities it does. We really don't understand how the emergent behaviours emerge.

It feels less like a word prediction algorithm and more like a world model compression algorithm. Maybe we tried to create one and accidentaly created the other?

Why would asking a question about ice cream trigger a consideration about all possible topics? As in, to formulate the answer, the LLM will consider the origin of Elephants even. It won’t be significant, but it will be factored in.

Why? In the spiritual realm, many postulated that even the Elephant you never met is part of your life.

None of this is a coincidence.

Re: S1: A $6 R1 competitor?

#322

Post by mig1 »

This argument that the data centers and all the GPUs will be useful even in the context of Deepseek doesn't add up... basically they showed that it's diminishing returns after a certain amount. And so far it didn't make OpenAI or Anthropic go faster, did it?

Re: S1: A $6 R1 competitor?

#323

Post by rayboy1995 »

mig1 wrote: post #322

This argument that the data centers and all the GPUs will be useful even in the context of Deepseek doesn't add up... basically they showed that it's diminishing returns after a certain amount. And so far it didn't make OpenAI or Anthropic go faster, did it?

What is the source for the diminishing returns? I would like to read about it as I have only seen papers referring to the scaling law still applying.

Re: S1: A $6 R1 competitor?

#324

Post by rayboy1995 »

gorgoiler wrote: post #270

This feels just like telling a constraint satisfaction engine to backtrack and find a more optimal route through the graph. We saw this 25 years ago with engines like PROVERB doing directed backtracking, and with adversarial planning when automating competitive games. Why would you control the inference at the token level? Wouldn’t the more obvious (and technically superior) place to control repeat analysis of the op…

This is the difference between science and engineering. What they have done is engineering. If the result is 90% of the way there with barely any effort, its best to move on to something else that may be low hanging fruit than to spend time chasing that 10%.

Re: S1: A $6 R1 competitor?

#325

Post by adamc »

I found it interesting but the "Wait" vs. "Hmm" bit just made me think we don't really understand our own models here. I mean, sure, it's great that they measured and found something better, but it's kind of disturbing that you have to guess.

Re: S1: A $6 R1 competitor?

#326

Post by adamc »

teruakohatu wrote: post #193
xg15 wrote: post #175

Earlier quoted context omitted.

I think the fact alone that distillation and quantization are techniques that can produce substantial improvements is a strong sign that we still have no real comprehensive understanding how the models work. If we had, there would be no reason to train a model with more parameters than are strictly necessary to represent the space's semantic structure. But then it should be impossible for distilled models with less p…

> still have no real comprehensive understanding how the models work. We do understand how they work, we just have not optimised their usage. For example someone who has a good general understanding of how an ICE or EV car works. Even if the user interface is very unfamiliar, they can figure out how to drive any car within a couple of minutes. But that does not mean they can race a car, drift a car or drive a car on…

The "Wait" vs. "Hmm" discussion in the paper does not suggest we know how they work. If we knew, we wouldn't have to try things and measure to figure out the best prompt.

Re: S1: A $6 R1 competitor?

#327

Post by [deleted] »

int_19h wrote: post #244
zoogeny wrote: post #214

Earlier quoted context omitted.

For the "looping" I'm talking about you don't need to update the weights. It is simply, old context in, new context out, new context in, new-new context out, etc. Of course, keeping that coherent over numerous loops isn't going to be easy. No doubt there is a chance it goes off the rails. So you might have a section of context that is constantly stable, a section of context that updates each loop, etc. In the other r…

Then I'm not quite sure what benefit you expect to derive from it? Making e.g. QwQ-32 loop isn't hard - it often does it all by itself, even. But it doesn't translate to improvements on every iteration; it just goes in circles.

[deleted]

Re: S1: A $6 R1 competitor?

#328

Post by leopoldj »

>it can run on my laptop

Has anyone run it on a laptop (unquantized)? Disk size of the 32B model appears to be 80GB. Update: I'm using a 40GB A100 GPU. Loading the model took 30GB vRAM. I asked a simple question "How many r in raspberry". After 5 minutes nothing got generated beyond the prompt. I'm not sure how the author ran this on a laptop.

Re: S1: A $6 R1 competitor?

#329

Post by mvc »

baxtr wrote: post #170
littlestymaar wrote: post #138

Earlier quoted context omitted.

I think you're missing the point: H100 isn't going to remain useful for a long time, would you consider Tesla or Pascal graphic cards a collateral? That's what those H100 will look like in just a few years.

Not sure I do tbh. Any asset depreciates over time. But they usually get replaced. My 286 was replaced by a faster 386 and that by an even faster 468. I’m sure you see a naming pattern there.

> My 286 was replaced by a faster 386 and that by an even faster 468.

How much was your 286 chip worth when you bought your 486?

Re: S1: A $6 R1 competitor?

#330

Post by luc4sdreyer »

codeulike wrote: post #290
mtrovo wrote: post #133

I found the discussion around inference scaling with the 'Wait' hack so surreal. The fact such an ingeniously simple method can impact performance makes me wonder how many low-hanging fruit we're still missing. So weird to think that improvements on a branch of computer science is boiling down to conjuring the right incantation words, how you even change your mindset to start thinking this way?

Wait, so the trick is they reach into the context and basically switch ' ' with 'wait' and that makes it carry on thinking?

Yes, that's one of the tricks.
Post reply on HN