Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An analysis of DeepSeek's R1-Zero and R1

arcprize.org

Post reply on HN
Print view View original

221–230 of 280 posts

Re: An analysis of DeepSeek's R1-Zero and R1

#221

Post by boroboro4 »

vidarh wrote: post #219
boroboro4 wrote: post #210

Earlier quoted context omitted.

I didn’t say the model gonna run on one chip of course. 70B needs ~300 chips (only for weights, fp8, just like they do, key value cache not included), 670B would need ~3000 chips, and in racks or not it’s very hard to set up such cluster for one model. There are reasons they still don’t have Llama 405B model.

They deliver pre-built full racks. The "reasons" are most likely because it's not cost-effective as what is effective at this point a tech demo, that first becomes cheap to run if you're actually going to use a decent portion of the capacity for a single model.

How many servers in one rack? Let's say 42. How many chips in one server? Let's say 8. It's 336 cards per rack - enough for fp8 70B model weights (and, maybe, kv cache if your requests aren't too long, but probably not really). You need 10 (!) racks to serve one (!) DeepSeek model weights. There is also massive amount complexity arises from operating so many nodes.

During short time when Groq hardware appeared on the market it was costing 20K per card. It's 60 mln (!) per 1 Deepseek model. You need absolutely crazy amount of load to justify those costs, and, most likely, you will need massive amount additional nodes to handle KV cache of those requests.

Re: An analysis of DeepSeek's R1-Zero and R1

#222

Post by deegles »

dematz wrote: post #99
deegles wrote: post #93

Earlier quoted context omitted.

not being snarky, but what is the point of using the model if you already know enough to correct it into giving the right answer? an example that just occurred to me - if you asked it to generate an image of a mushroom that is safe to eat in your area, how would you tell it it was wrong? "oh, they never got back to me, I'll generate this image for others as well!"

A common use of these models is asking for code, and maybe you don't know the answer or would take a while to figure it out. For example, here's some html, make it blue and centered. You could give the model feedback on if its answer worked or not, without knowing the correct answer yourself ahead of time.

I was using llama3 and deepseek-r1 literally to center an element in a div and they were not able to despite many prompts and variations. I guess I figured it out in the end but I'm not convinced I saved any time vs just carefully reading flexbox docs.

Re: An analysis of DeepSeek's R1-Zero and R1

#223

Post by wildermuthn »

anothermathbozo wrote: post #3

The claim is that this removes the human bottleneck (aka SFT or supervised fine tuning) on domains with a verifiable reward. Critically, this verifiable reward is extremely hard to pin down in nearly all domains besides mathematics and computer science.

This feels quite close to the definition of the singularity; if an LLM can become both the Generator and the Discriminator (to use a GAN analogy), then we have takeoff.

Re: An analysis of DeepSeek's R1-Zero and R1

#224

Post by sirsinsalot »

merrywhether wrote: post #216
mohsen1 wrote: post #31

Earlier quoted context omitted.

every time you respond to an AI model "no, you got that wrong, do it this way" you provide a very valuable piece of data to train on. With reasoning tokens there is just a lot more of that data to train on now

Users can be adversarial to the “truth” (to the extent it exists) without being adversarial in intent. Dinosaur bones are either 65 million year old remnants of ancient creatures or decoys planted by a God during a 7 day creation, and a large proportion of humans earnestly believe either take. Choosing which of these to believe involves a higher level decision about fundamental worldviews. This is an extreme example,…

This is a good take. What models seem to be poor at is undoing their own thinking down a path even when they can test.

If you let a model write code, test it, identify bugs and fix them, you get an increasingly obtuse and complex code base where errors happen more. The more it iterate the worse it gets.

At the end of the day, written human language is a poor way of describing software. Even to a model. The code is the description.

At the moment we describe solutions we want to see to the models and they aren't that smart about translating that to an unambiguous form.

We are a long was off describing the problems and asking for a solution. Even when the model can test and iterate.

Re: An analysis of DeepSeek's R1-Zero and R1

#226

Post by gbasin »

spyckie2 wrote: post #22

> But now with reasoning systems and verifiers, we can create brand new legitimate data to train on. This can either be done offline where the developer pays to create the data or at inference time where the end user pays! > This is a fascinating shift in economics and suggests there could be a runaway power concentrating moment for AI system developers who have the largest number of paying customers. Those customers…

the main bottleneck will be model depth... you can only do so much with N layers, and recurrence has proven to be way less efficient (for now)

Re: An analysis of DeepSeek's R1-Zero and R1

#227

Post by verisimi »

spyckie2 wrote: post #22

> But now with reasoning systems and verifiers, we can create brand new legitimate data to train on. This can either be done offline where the developer pays to create the data or at inference time where the end user pays! > This is a fascinating shift in economics and suggests there could be a runaway power concentrating moment for AI system developers who have the largest number of paying customers. Those customers…

*SOTA state of the art

Re: An analysis of DeepSeek's R1-Zero and R1

#228

Post by hammock »

halfadot wrote: post #171
hammock wrote: post #54

Earlier quoted context omitted.

The AI models to begin with assume that a significant majority of the training material is honest/in good faith. So that is not new?

AI models don't assume anything. AI models are just statistical tools. Their data is prepared by humans, who aren't morons. What is it with these super-ignorant AI critiques popping up everywhere?

Try a little benefit of the doubt, nuance or colloquialism. Or a bit of all three.

Re: An analysis of DeepSeek's R1-Zero and R1

#229

Post by cootsnuck »

nogridbag wrote: post #176
octacat wrote: post #168

I think deepseek accidentally also killed google for me, not just chatgpt. Because of the visible reasoning part.

From what I read elsewhere (random reddit comment), the visible reasoning is just "for show" and isn't the process deepseek used to arrive at the result. But if the reasoning has value, I guess it doesn't matter even if it's fake.

That doesn't really make sense with how LLMs work. I think this is exactly why it's risky to use words like "thinking" and "reasoning".

If by the "visible reasoning" is just for show they meant these models don't actually think and reason, then yes that is correct.

But if they meant that the visible reasoning is not quite literally a part of inference process...that's entirely incorrect.

R1 is open source. We don't have to make guesses about its functioning.

Post reply on HN