Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An analysis of DeepSeek's R1-Zero and R1

arcprize.org

Post reply on HN
Print view View original

51–60 of 280 posts

Re: An analysis of DeepSeek's R1-Zero and R1

#51

Post by fizx »

spyckie2 wrote: post #22

> But now with reasoning systems and verifiers, we can create brand new legitimate data to train on. This can either be done offline where the developer pays to create the data or at inference time where the end user pays! > This is a fascinating shift in economics and suggests there could be a runaway power concentrating moment for AI system developers who have the largest number of paying customers. Those customers…

You're not getting new high-quality textual data for pre-training from your chat service. But you are potentially getting a lot of RL feedback on ambiguous problems.

Re: An analysis of DeepSeek's R1-Zero and R1

#52

Post by WhitneyLand »

mikejulietbravo wrote: post #16

Mike from Baseten here We're super proud to support this work. If you're thinking of running deepseek in production, give us a shout!

Can you share at a high level how you run this model?

We know it’s 671B params with each MOE node at 37B…

If the GPUs have say, 140GB for an H200, then do you just load up as many nodes as will fit into a GPU?

How much do interconnects hurt performance vs being able to load the model into a single GPU?

Re: An analysis of DeepSeek's R1-Zero and R1

#53

Post by kristjansson »

polishdude20 wrote: post #6

I predict that the future of LLM's when it comes to coding and software creation is in "custom individually tailored apps". Imagine telling an AI agent what app you want, the requirements and all that and it just builds everything needed from backend to frontend, asks for your input on how things should work, clarifying questions etc. It tests the software by compiling and running it reading errors and failed tests a…

Most people really do not know what they want at any level of detail.

Re: An analysis of DeepSeek's R1-Zero and R1

#54

Post by hammock »

nine_k wrote: post #46
mohsen1 wrote: post #31

Earlier quoted context omitted.

every time you respond to an AI model "no, you got that wrong, do it this way" you provide a very valuable piece of data to train on. With reasoning tokens there is just a lot more of that data to train on now

This assumes that you give honest feedback. Efforts to feed deployed AI models various epistemic poisons abound in the wild.

The AI models to begin with assume that a significant majority of the training material is honest/in good faith. So that is not new?

Re: An analysis of DeepSeek's R1-Zero and R1

#55

Post by Exoristos »

aprilthird2021 wrote: post #40
mohsen1 wrote: post #31

Earlier quoted context omitted.

every time you respond to an AI model "no, you got that wrong, do it this way" you provide a very valuable piece of data to train on. With reasoning tokens there is just a lot more of that data to train on now

So if I just pay OpenAI $200/mo, and randomly tell the AI, no that's wrong. I can stop the AI takeover?

You can have our thank-you cards forwarded to your cell at Guantanamo Bay.

Re: An analysis of DeepSeek's R1-Zero and R1

#56

Post by repelsteeltje »

jacobsenscott wrote: post #48
polishdude20 wrote: post #6

I predict that the future of LLM's when it comes to coding and software creation is in "custom individually tailored apps". Imagine telling an AI agent what app you want, the requirements and all that and it just builds everything needed from backend to frontend, asks for your input on how things should work, clarifying questions etc. It tests the software by compiling and running it reading errors and failed tests a…

What's it called when you describe an app with sufficient detail that a computer can carry out the processes you want? Where will the record of those clarifying questions and updates be kept? What if one developer asks the AI to surreptitiously round off pennies and put those pennies into their bank account? Where will that change be recorded, will humans be able to recognize it? What if two developers give it confli…

That.

Plus coding (producing a working program that fits some requirement) is the least interesting part of software development. It adds complexity, bugs and maintenance.

Re: An analysis of DeepSeek's R1-Zero and R1

#57

Post by rpastuszak »

fragmede wrote: post #43
optimalsolver wrote: post #25

Earlier quoted context omitted.

But can o3 write a symphony? Seriously though, I'd like to hear suggestions on how to automatically evaluate an AI model's creativity, no humans in the loop.

we'd have to create a numerical scale for creativity, from boring to Dali, with milliEschers and MegaGeigers somewhere in there as well

It's essential that we quantify everything so that we can put a price on it. I'd go with Kahlograms though.

Re: An analysis of DeepSeek's R1-Zero and R1

#58

Post by philipkiely »

WhitneyLand wrote: post #52
mikejulietbravo wrote: post #16

Mike from Baseten here We're super proud to support this work. If you're thinking of running deepseek in production, give us a shout!

Can you share at a high level how you run this model? We know it’s 671B params with each MOE node at 37B… If the GPUs have say, 140GB for an H200, then do you just load up as many nodes as will fit into a GPU? How much do interconnects hurt performance vs being able to load the model into a single GPU?

Yeah so MoE doesn't really come into play for production serving -- once you are batching your requests you hit every expert at a large enough batch size so you have to think about running the models as a whole.

There are two ways we can run it:

- 8xH200 GPU == 8x141GB == 1128 GB VRAM

- 16xH100 GPU == 8x80GB == 1280 GB VRAM

Within a single node (up to 8 GPUs) you don't see any meaningful hit from GPU-to-GPU communication.

More than that (e.g. 16xH100) requires multi-node inference which very few places have solved at a production-ready level, but it's massive because there are way more H100s out there than H200s.

Re: An analysis of DeepSeek's R1-Zero and R1

#59

Post by [deleted] »

spyckie2 wrote: post #22

> But now with reasoning systems and verifiers, we can create brand new legitimate data to train on. This can either be done offline where the developer pays to create the data or at inference time where the end user pays! > This is a fascinating shift in economics and suggests there could be a runaway power concentrating moment for AI system developers who have the largest number of paying customers. Those customers…

[deleted]

Re: An analysis of DeepSeek's R1-Zero and R1

#60

Post by BorisMelnik »

nine_k wrote: post #46
mohsen1 wrote: post #31

Earlier quoted context omitted.

every time you respond to an AI model "no, you got that wrong, do it this way" you provide a very valuable piece of data to train on. With reasoning tokens there is just a lot more of that data to train on now

This assumes that you give honest feedback. Efforts to feed deployed AI models various epistemic poisons abound in the wild.

I am not in this space, question: are there "bad actors" that are known to feed AI models with poisonous information?
Post reply on HN