Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An analysis of DeepSeek's R1-Zero and R1

arcprize.org

Post reply on HN
Print view View original

131–140 of 280 posts

Re: An analysis of DeepSeek's R1-Zero and R1

#131

Post by Levitz »

deegles wrote: post #93
mohsen1 wrote: post #31

Earlier quoted context omitted.

every time you respond to an AI model "no, you got that wrong, do it this way" you provide a very valuable piece of data to train on. With reasoning tokens there is just a lot more of that data to train on now

not being snarky, but what is the point of using the model if you already know enough to correct it into giving the right answer? an example that just occurred to me - if you asked it to generate an image of a mushroom that is safe to eat in your area, how would you tell it it was wrong? "oh, they never got back to me, I'll generate this image for others as well!"

>not being snarky, but what is the point of using the model if you already know enough to correct it into giving the right answer?

For your example, what if you want to show what such a mushroom looks like to a friend? What if you want to use it on a website?

Re: An analysis of DeepSeek's R1-Zero and R1

#132

Post by buyucu »

panabee wrote: post #122
vidarh wrote: post #26

Earlier quoted context omitted.

For inference Nvidia has more significant competition than for training. See Groq, Google's TPU's etc.

Nvidia (NVDA) generates revenue with hardware, but digs moats with software. The CUDA moat is widely unappreciated and misunderstood. Dethroning Nvidia demands more than SOTA hardware. OpenAI, Meta, Google, AWS, AMD, and others have long failed to eliminate the Nvidia tax. Without diving into the gory details, the simple proof is that billions were spent on inference last year by some of the most sophisticated techno…

My company recently switched from A100s to MI300s. I can confidently say that in my line of work, there is no CUDA moat. Onboarding took about month, but afterwards everything was fine.

Re: An analysis of DeepSeek's R1-Zero and R1

#133

Post by panabee »

onlyrealcuzzo wrote: post #127
panabee wrote: post #122

Earlier quoted context omitted.

Nvidia (NVDA) generates revenue with hardware, but digs moats with software. The CUDA moat is widely unappreciated and misunderstood. Dethroning Nvidia demands more than SOTA hardware. OpenAI, Meta, Google, AWS, AMD, and others have long failed to eliminate the Nvidia tax. Without diving into the gory details, the simple proof is that billions were spent on inference last year by some of the most sophisticated techno…

> OpenAI, Meta, AWS, AMD, and others have long attempted to eliminate the Nvidia tax, yet failed. Gemini / Google runs and trains on TPUs. You have no incentive to infer on AMD if you need to buy a massive Nvidia cluster to train.

Google was omitted because they own the hardware and the models, but in retrospect, they represent a proof point nearly as compelling as OpenAI. Thanks for the comment.

Google has leading models operating on leading hardware, backed by sophisticated tech talent who could facilitate migrations, yet Google still cannot leap over the CUDA moat and capture meaningful inference market share.

Yes, training plays a crucial role. This is where companies get shoehorned into the CUDA ecosystem, but if CUDA were not so intertwined with performance and reliability, customers could theoretically switch after training.

Re: An analysis of DeepSeek's R1-Zero and R1

#134

Post by vagabund »

spyckie2 wrote: post #22

> But now with reasoning systems and verifiers, we can create brand new legitimate data to train on. This can either be done offline where the developer pays to create the data or at inference time where the end user pays! > This is a fascinating shift in economics and suggests there could be a runaway power concentrating moment for AI system developers who have the largest number of paying customers. Those customers…

> I highly doubt you are getting novel, high quality data.

Why wouldn't you? Presumably the end user would try their use case on the existing model, and if it performs well, wouldn't bother with the expense of setting up an RL environment specific to their task.

If it doesn't perform well, they do bother, and they have all the incentive in the world to get the verifier right -- which is not an extraordinarily sophisticated task if you're only using rules-based outcome rewards (as R1 and R1-Zero do)

Re: An analysis of DeepSeek's R1-Zero and R1

#135

Post by panabee »

buyucu wrote: post #132
panabee wrote: post #122

Earlier quoted context omitted.

Nvidia (NVDA) generates revenue with hardware, but digs moats with software. The CUDA moat is widely unappreciated and misunderstood. Dethroning Nvidia demands more than SOTA hardware. OpenAI, Meta, Google, AWS, AMD, and others have long failed to eliminate the Nvidia tax. Without diving into the gory details, the simple proof is that billions were spent on inference last year by some of the most sophisticated techno…

My company recently switched from A100s to MI300s. I can confidently say that in my line of work, there is no CUDA moat. Onboarding took about month, but afterwards everything was fine.

Alternatives exist, especially for mature and simple models. The point isn't that Nvidia has 100% market share, but rather that they command the most lucrative segment and none of these big spenders have found a way to quit their Nvidia addiction, despite concerted efforts to do so.

For instance, we experimented with AWS Inferentia briefly, but the value prop wasn't sufficient even for ~2022 computer vision models.

The calculus is even worse for SOTA LLMs.

The more you need to eke out performance gains and ship quickly, the more you depend on CUDA and the deeper the moat becomes.

Re: An analysis of DeepSeek's R1-Zero and R1

#136

Post by fsndz »

spyckie2 wrote: post #22

> But now with reasoning systems and verifiers, we can create brand new legitimate data to train on. This can either be done offline where the developer pays to create the data or at inference time where the end user pays! > This is a fascinating shift in economics and suggests there could be a runaway power concentrating moment for AI system developers who have the largest number of paying customers. Those customers…

I think we will have to move with pre-training and post-training efforts in parallel. What DeepSeek showed is that you first need to have a strong enough pretrained model. For that, we have to continue the acquisition of high quality, multilingual datasets. Then, when we have a stronger pretrained model, we can apply pure RL to get a reasoning model that we use only to generate synthetic reasoning data. We then use those synthetic reasoning data to fine-tune the original pretrained model and make it even stronger. https://transitions.substack.com/p/the-laymans-introduction-...

Re: An analysis of DeepSeek's R1-Zero and R1

#137

Post by snovv_crash »

ClumsyPilot wrote: post #47
talldayo wrote: post #14

Earlier quoted context omitted.

Nvidia can actually charge larger margins if inference compute goes down. It would enable them to manufacture more units of smaller GPUs using inferior and cheaper silicon, all of which would increase the profits per unit sold as well as the number of units they can manufacture. The industry has to find a way to separate itself from Nvidia's GPGPU technology if they want to stop being gouged. The issue is that nobody…

I think future of inference is on the client side You can do inference on almost any hardware, I do not see any edge for NVIDIA here I can download DeepSeek 30b model and run inference at good speed on AMD GPU ms and even on CPU. Apple silicon works fine too. I get >50 tokens/s on £300 AMD GPUs. The main bottleneck appears to be memory, not processing power.

Which AMD GPU gives you 50 tok/s on a 30b model? My 3090 does 30 tok/s with a 4 bit quant.

Re: An analysis of DeepSeek's R1-Zero and R1

#138

Post by hnburnsy »

>There are two major shifts happening in AI, economically speaking:

  You can now spend more $ to get higher accuracy and reliability
  Training $ is moving to inference $
>Both are going to drive a massive amount of demand for inference and neither will curtail the demand for more compute. In fact, they will increase the demand for compute.

Is this Nvidia compute or something else?

Re: An analysis of DeepSeek's R1-Zero and R1

#139

Post by throw310822 »

jacobsenscott wrote: post #48
polishdude20 wrote: post #6

I predict that the future of LLM's when it comes to coding and software creation is in "custom individually tailored apps". Imagine telling an AI agent what app you want, the requirements and all that and it just builds everything needed from backend to frontend, asks for your input on how things should work, clarifying questions etc. It tests the software by compiling and running it reading errors and failed tests a…

What's it called when you describe an app with sufficient detail that a computer can carry out the processes you want? Where will the record of those clarifying questions and updates be kept? What if one developer asks the AI to surreptitiously round off pennies and put those pennies into their bank account? Where will that change be recorded, will humans be able to recognize it? What if two developers give it confli…

> What's it called when you describe an app with sufficient detail that a computer can carry out the processes you want?

You're wrong here. The entire point is that these are not computers as we used to think of them. These things have common sense; they can analyse a problem including all the implicit aspects, suggest and evaluate different implementation methods, architectures, interfaces.

So the right question is: "what's it called when you describe an app to a development team and they ask back questions and come back with designs and discuss them with you, and finally present you with an mvp, and then you iterate on that?"

Re: An analysis of DeepSeek's R1-Zero and R1

#140

Post by amelius »

levocardia wrote: post #102
dagelf wrote: post #2

Well o3 scored 75% on AGI-1, R1 and o1 only 25%.... watch this space though....

What's interesting is that you can already see the "AI race" dynamics in play -- OpenAI must be under immense market pressure to push o3 out to the public to reclaim "king of the hill" status.

Yeah, but they can use DeepSeek's new algorithm too.
Post reply on HN