Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An analysis of DeepSeek's R1-Zero and R1

arcprize.org

Post reply on HN
Print view View original

121–130 of 280 posts

Re: An analysis of DeepSeek's R1-Zero and R1

#121

Post by IAmGraydon »

acchow wrote: post #88
polishdude20 wrote: post #6

I predict that the future of LLM's when it comes to coding and software creation is in "custom individually tailored apps". Imagine telling an AI agent what app you want, the requirements and all that and it just builds everything needed from backend to frontend, asks for your input on how things should work, clarifying questions etc. It tests the software by compiling and running it reading errors and failed tests a…

Have you tried https://bolt.diy ? It does what you describe

It claims to do what he describes.

Re: An analysis of DeepSeek's R1-Zero and R1

#122

Post by panabee »

vidarh wrote: post #26
talldayo wrote: post #14

Earlier quoted context omitted.

Nvidia can actually charge larger margins if inference compute goes down. It would enable them to manufacture more units of smaller GPUs using inferior and cheaper silicon, all of which would increase the profits per unit sold as well as the number of units they can manufacture. The industry has to find a way to separate itself from Nvidia's GPGPU technology if they want to stop being gouged. The issue is that nobody…

For inference Nvidia has more significant competition than for training. See Groq, Google's TPU's etc.

Nvidia (NVDA) generates revenue with hardware, but digs moats with software.

The CUDA moat is widely unappreciated and misunderstood. Dethroning Nvidia demands more than SOTA hardware.

OpenAI, Meta, Google, AWS, AMD, and others have long failed to eliminate the Nvidia tax.

Without diving into the gory details, the simple proof is that billions were spent on inference last year by some of the most sophisticated technology companies in the world.

They had the talent and the incentive to migrate, but didn't.

In particular, OpenAI spent $4 billion, 33% more than on training, yet still ran on NVDA. Google owns leading chips and leading models, and could offer the tech talent to facilitate migrations, yet still cannot cross the CUDA moat and convince many inference customers to switch.

People are desperate to quit their NVDA-tine addiction, but they can't for now.

[Edited to include Google, even though Google owns the chips and the models; h/t @onlyrealcuzzo]

Re: An analysis of DeepSeek's R1-Zero and R1

#123

Post by sheepscreek »

spyckie2 wrote: post #22

> But now with reasoning systems and verifiers, we can create brand new legitimate data to train on. This can either be done offline where the developer pays to create the data or at inference time where the end user pays! > This is a fascinating shift in economics and suggests there could be a runaway power concentrating moment for AI system developers who have the largest number of paying customers. Those customers…

> The most promising idea is to use reasoning models to generate data, and then train our non-reasoning models with the reasoning-embedded data.

DeepSeek did precisely this with their LLama fine-tunes. You can try the 70B one here (might have to sign up): https://groq.com/groqcloud-makes-deepseek-r1-distill-llama-7...

Re: An analysis of DeepSeek's R1-Zero and R1

#124

Post by vidarh »

pants2 wrote: post #34
vidarh wrote: post #26

Earlier quoted context omitted.

For inference Nvidia has more significant competition than for training. See Groq, Google's TPU's etc.

People talk about Groq and Cerberus as competitors but it seems to me their manufacturing process makes the availability of those chips extremely limited. You can call up Nvidia and order $10B worth of GPUs and have them delivered the next week. Can't say the same for these specialty competitors.

> You can call up Nvidia and order $10B worth of GPUs and have them delivered the next week

Nvidia sold $14.5 billion of datacenter hardware in the third quarter of their fiscal 2024 and that led to severe supply constraints, with estimate lead times for H100's up to 52 weeks some places, so no you can't, as that $14.5 billion was clearly capped by their ability to supply, not demand.

You're right, though, that Groq etc. can't deliver anywhere near the same volume now, but there's little reason to believe that will continue. There's no need for full GPU's for inference only workloads, so competitors can enter the space with a tiny proportion of functionality.

Re: An analysis of DeepSeek's R1-Zero and R1

#125

Post by baq »

ant6n wrote: post #86
jl6 wrote: post #67

Earlier quoted context omitted.

$3.4K is about what you might pay a magic circle lawyer for an opinion on a matter. Not saying o3 is an efficient use of resources, just saying that it’s not outlandish that a sufficiently good AI could be worth that kind of money.

What’s the liability insurance of the AI like

Refer to IBM’s 1979 slide for details on that

Re: An analysis of DeepSeek's R1-Zero and R1

#126

Post by vidarh »

panabee wrote: post #122
vidarh wrote: post #26

Earlier quoted context omitted.

For inference Nvidia has more significant competition than for training. See Groq, Google's TPU's etc.

Nvidia (NVDA) generates revenue with hardware, but digs moats with software. The CUDA moat is widely unappreciated and misunderstood. Dethroning Nvidia demands more than SOTA hardware. OpenAI, Meta, Google, AWS, AMD, and others have long failed to eliminate the Nvidia tax. Without diving into the gory details, the simple proof is that billions were spent on inference last year by some of the most sophisticated techno…

The CUDA moat is largely irrelevant for inference. The code needed for inference is small enough that there are e.g. bare-metal CPU only implementations. That isn't what's limiting people from moving fully off Nvidia for inference. And you'll note almost "everyone" in this game are in the process of developing their own chips.

Re: An analysis of DeepSeek's R1-Zero and R1

#127

Post by onlyrealcuzzo »

panabee wrote: post #122
vidarh wrote: post #26

Earlier quoted context omitted.

For inference Nvidia has more significant competition than for training. See Groq, Google's TPU's etc.

Nvidia (NVDA) generates revenue with hardware, but digs moats with software. The CUDA moat is widely unappreciated and misunderstood. Dethroning Nvidia demands more than SOTA hardware. OpenAI, Meta, Google, AWS, AMD, and others have long failed to eliminate the Nvidia tax. Without diving into the gory details, the simple proof is that billions were spent on inference last year by some of the most sophisticated techno…

> OpenAI, Meta, AWS, AMD, and others have long attempted to eliminate the Nvidia tax, yet failed.

Gemini / Google runs and trains on TPUs.

You have no incentive to infer on AMD if you need to buy a massive Nvidia cluster to train.

Re: An analysis of DeepSeek's R1-Zero and R1

#130

Post by theendisney »

mrandish wrote: post #73
BorisMelnik wrote: post #60

Earlier quoted context omitted.

I am not in this space, question: are there "bad actors" that are known to feed AI models with poisonous information?

I'm not in the space either but I think the answer is an emphatic yes. Three categories come to mind: 1. Online trolls and pranksters (who already taught several different AIs to be racist in a matter of hours - just for the LOLs). 2. Nation states like China who already require models to conform to state narratives. 3. More broadly, when training on "the internet" as a whole there is a huge amount of wrong, confused…

The part where people disagree seems fun.

Im looking forwards to protoscience/unconventional science and perhaps even that what is worthy of the fringe or pseudoscience labels. The debunking there usually fails to adress the topic as it is incredibly hard to spend even a single day reading about something you "know" to be nonsense. Who has time for that?

If you take a hundred thousand such topics the odds they should all be dismissed without looking arent very good.

Post reply on HN