DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL • Hacker News Board

Re: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL

#21

Post by leetharris » Sat, Jan 25, 2025, 7:33 PM UTC

siliconc0w wrote: post #3

The US Economy is pretty vulnerable here. If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived value. I wonder if this was a deliberate move by PRC or really our own fault in falling for the fallacy that more is always better.

CEO of Scale said Deepseek is lying and actually has a 50k GPU cluster. He said they lied in the paper because technically they aren't supposed to have them due to export laws.

I feel like this is very likely. They obvious did some great breakthroughs, but I doubt they were able to train on so much less hardware.

Re: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL

#22

Post by logicchains » Sat, Jan 25, 2025, 7:33 PM UTC

m_a_g wrote: post #16

refulgentis wrote: post #6

Earlier quoted context omitted.

I've been confused over this. I've seen a $5.5M # for training, and commensurate commentary along the lines of what you said, but it elides the cost of the base model AFAICT.

With $5.5M, you can buy around 150 H100s. Experts correct me if I’m wrong but it’s practically impossible to train a model like that with that measly amount. So I doubt that figure includes all the cost of training.

The cost, as expressed in the DeepSeek V3 paper, was expressed in terms of training hours based on the market rate per hour if they'd rented the 2k GPUs they used.

Re: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL

#23

Post by Leary » Sat, Jan 25, 2025, 7:35 PM UTC

siliconc0w wrote: post #3

The US Economy is pretty vulnerable here. If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived value. I wonder if this was a deliberate move by PRC or really our own fault in falling for the fallacy that more is always better.

or maybe the US economy will do even better because more people will be able to use AI at a low cost.

OpenAI will be also be able to serve o3 at a lower cost if Deepseek had some marginal breakthrough OpenAI did not already think of.

Re: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL

#24

Post by rikafurude21 » Sat, Jan 25, 2025, 7:35 PM UTC

siliconc0w wrote: post #3

The US Economy is pretty vulnerable here. If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived value. I wonder if this was a deliberate move by PRC or really our own fault in falling for the fallacy that more is always better.

Why do americans think china is like a hivemind controlled by an omnisicient Xi, making strategic moves to undermine them? Is it really that unlikely that a lab of genius engineers found a way to improve efficiency 10x?

Re: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL

#25

Post by etc-hosts » Sat, Jan 25, 2025, 7:35 PM UTC

m_a_g wrote: post #16

refulgentis wrote: post #6

Earlier quoted context omitted.

I've been confused over this. I've seen a $5.5M # for training, and commensurate commentary along the lines of what you said, but it elides the cost of the base model AFAICT.

With $5.5M, you can buy around 150 H100s. Experts correct me if I’m wrong but it’s practically impossible to train a model like that with that measly amount. So I doubt that figure includes all the cost of training.

It's even more. You also need to fund power and maintain infrastructure to run the GPUs. You need to build fast networks between the GPUs for RDMA. Ethernet is going to be too slow. Infiniband is unreliable and expensive.

Re: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL

#26

Post by pdntspa » Sat, Jan 25, 2025, 7:35 PM UTC

leetharris wrote: post #21

siliconc0w wrote: post #3

The US Economy is pretty vulnerable here. If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived value. I wonder if this was a deliberate move by PRC or really our own fault in falling for the fallacy that more is always better.

CEO of Scale said Deepseek is lying and actually has a 50k GPU cluster. He said they lied in the paper because technically they aren't supposed to have them due to export laws. I feel like this is very likely. They obvious did some great breakthroughs, but I doubt they were able to train on so much less hardware.

I would think the CEO of an American AI company has every reason to neg and downplay foreign competition...

And since it's a businessperson they're going to make it sound as cute and innocuous as possible

Re: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL

#27

Post by robertclaus » Sat, Jan 25, 2025, 7:36 PM UTC

siliconc0w wrote: post #3

The US Economy is pretty vulnerable here. If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived value. I wonder if this was a deliberate move by PRC or really our own fault in falling for the fallacy that more is always better.

Doesn't this just mean throwing a gazillion GPUs at the new architecture and defining a new SOTA?

Re: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL

#28

Post by latchkey » Sat, Jan 25, 2025, 7:36 PM UTC

leetharris wrote: post #21

siliconc0w wrote: post #3

The US Economy is pretty vulnerable here. If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived value. I wonder if this was a deliberate move by PRC or really our own fault in falling for the fallacy that more is always better.

CEO of Scale said Deepseek is lying and actually has a 50k GPU cluster. He said they lied in the paper because technically they aren't supposed to have them due to export laws. I feel like this is very likely. They obvious did some great breakthroughs, but I doubt they were able to train on so much less hardware.

Thanks to SMCI that let them out...

https://wccftech.com/nvidia-asks-super-micro-computer-smci-t...

Chinese guy in a warehouse full of SMCI servers bragging about how he has them...

https://www.youtube.com/watch?v=27zlUSqpVn8

Re: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL

#29

Post by [deleted] » Sat, Jan 25, 2025, 7:38 PM UTC

rikafurude21 wrote: post #24

siliconc0w wrote: post #3

The US Economy is pretty vulnerable here. If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived value. I wonder if this was a deliberate move by PRC or really our own fault in falling for the fallacy that more is always better.

Why do americans think china is like a hivemind controlled by an omnisicient Xi, making strategic moves to undermine them? Is it really that unlikely that a lab of genius engineers found a way to improve efficiency 10x?

[deleted]

Re: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL

#30

Post by logicchains » Sat, Jan 25, 2025, 7:38 PM UTC

jvanderbot wrote: post #11

siliconc0w wrote: post #3

The US Economy is pretty vulnerable here. If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived value. I wonder if this was a deliberate move by PRC or really our own fault in falling for the fallacy that more is always better.

How likely is this? Just a cursory probing of deepseek yields all kinds of censoring of topics. Isn't it just as likely Chinese sponsors of this have incentivized and sponsored an undercutting of prices so that a more favorable LLM is preferred on the market? Think about it, this is something they are willing to do with other industries. And, if LLMs are going to be engineering accelerators as the world believes, the…

>Isn't it just as likely Chinese sponsors of this have incentivized and sponsored an undercutting of prices so that a more favorable LLM is preferred on the market?

Since the model is open weights, it's easy to estimate the cost of serving it. If the cost was significantly higher than DeepSeek charges on their API, we'd expect other LLM hosting providers to charge significantly more for DeepSeek (since they aren't subsidised, so need to cover their costs), but that isn't the case.

This isn't possible with OpenAI because we don't know the size or architecture of their models.

Regarding censorship, most of it is done at the API level, not the model level, so running locally (or with another hosting provider) is much less expensive.