Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

huggingface.co

Post reply on HN
Print view View original

21–30 of 485 posts

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#21

Post by rhdunn »

htrp wrote: post #17

what is the ballpark vram / gpu requirement to run this ?

For just the model itself: 4 x params at F32, 2 x params at F16/BF16, or 1 x params at F8, e.g. 685GB at F8. It will be smaller for quantizations, but I'm not sure how to estimate those.

For a Mixture of Experts (MoE) model you only need to have the memory size of a given expert. There will be some swapping out as it figures out which expert to use, or to change expert, but once that expert is loaded it won't be swapping memory to perform the calculations.

You'll also need space for the context window; I'm not sure how to calculate that either.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#22

Post by espadrine »

jodleif wrote: post #5

I genuinely do not understand the evaluations of the US AI industry. The chinese models are so close and far cheaper

Two aspects to consider:

1. Chinese models typically focus on text. US and EU models also bear the cross of handling image, often voice and video. Supporting all those is additional training costs not spent on further reasoning, tying one hand in your back to be more generally useful.

2. The gap seems small, because so many benchmarks get saturated so fast. But towards the top, every 1% increase in benchmarks is significantly better.

On the second point, I worked on a leaderboard that both normalizes scores, and predicts unknown scores to help improve comparisons between models on various criteria: https://metabench.organisons.com/

You can notice that, while Chinese models are quite good, the gap to the top is still significant.

However, the US models are typically much more expensive for inference, and Chinese models do have a niche on the Pareto frontier on cheaper but serviceable models (even though US models also eat up the frontier there).

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#23

Post by jodleif »

jasonsb wrote: post #10
jodleif wrote: post #5

I genuinely do not understand the evaluations of the US AI industry. The chinese models are so close and far cheaper

It's all about the hardware and infrastructure. If you check OpenRouter, no provider offers a SOTA chinese model matching the speed of Claude, GPT or Gemini. The chinese models may benchmark close on paper, but real-world deployment is different. So you either buy your own hardware in order to run a chinese model at 150-200tps or give up an use one of the Big 3. The US labs aren't just selling models, they're selling…

Assuming your hardware premise is right (and lets be honest, nobody really wants to send their data to chinese providers) You can use a provider like Cerebras, Groq?

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#24

Post by jodleif »

espadrine wrote: post #22
jodleif wrote: post #5

I genuinely do not understand the evaluations of the US AI industry. The chinese models are so close and far cheaper

Two aspects to consider: 1. Chinese models typically focus on text. US and EU models also bear the cross of handling image, often voice and video. Supporting all those is additional training costs not spent on further reasoning, tying one hand in your back to be more generally useful. 2. The gap seems small, because so many benchmarks get saturated so fast. But towards the top, every 1% increase in benchmarks is sign…

1. Have you seen the Qwen offerings? They have great multi-modality, some even SOTA.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#25

Post by cyberlimerence »

newyankee wrote: post #19
isamuel wrote: post #18

Earlier quoted context omitted.

There is a great deal of orientalism --- it is genuinely unthinkable to a lot of American tech dullards that the Chinese could be better at anything requiring what they think of as "intelligence." Aren't they Communist? Backward? Don't they eat weird stuff at wet markets? It reminds me, in an encouraging way, of the way that German military planners regarded the Soviet Union in the lead-up to Operation Barbarossa. Th…

but didn't Chinese already surpass the rest of the world in Solar, batteries, EVs among other things ?

They did, but the goalposts keep moving, so to speak. We're approximately here : advanced semiconductors, artificial intelligence, reusable rockets, quantum computing, etc. Chinese will never catch up. /s

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#28

Post by mosselman »

isamuel wrote: post #18
jodleif wrote: post #5

I genuinely do not understand the evaluations of the US AI industry. The chinese models are so close and far cheaper

There is a great deal of orientalism --- it is genuinely unthinkable to a lot of American tech dullards that the Chinese could be better at anything requiring what they think of as "intelligence." Aren't they Communist? Backward? Don't they eat weird stuff at wet markets? It reminds me, in an encouraging way, of the way that German military planners regarded the Soviet Union in the lead-up to Operation Barbarossa. Th…

Back when deepseek came out and people were tripping over themselves shouting it was so much better than what was out there, it just wasn’t good.

It might be this model is super good, I haven’t tried it, but to say the Chinese models are better is just not true.

What I really love though is that I can run them (open models) on my own machine. The other day I categorised images locally using Qwen, what a time to be alive.

Further even than local hardware, open models make it possible to run on providers of choice, such as European ones. Which is great!

So I love everything about the competitive nature of this.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#29

Post by ksynwa »

jodleif wrote: post #20
jazzyjackson wrote: post #8

Earlier quoted context omitted.

Valuation is not based on what they have done but what they might do, I agree tho it's investment made with very little insight into Chinese research. I guess it's counting on deepseek being banned and all computers in America refusing to run open software by the year 2030 /snark

> Valuation is not based on what they have done but what they might do Exactly what I’m thinking. Chinese models catching rapidly. Soon to be on-par with the big dogs.

Even if they do continue to lag behind they are a good bet against monopolisation by proprietary vendors.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#30

Post by noosphr »

TIPSIO wrote: post #9

It's awesome that stuff like this is open source, but even if you have a basement rig with 4 NVIDIA GeForce RTX 5090 graphic cards ($15-20k machine), can it even run with any reasonable context window that isn't like a crawling 10/tps? Frontier models are far exceeding even the most hardcore consumer hobbyist requirements. This is even further

Home rigs like that are no longer cost effective. You're better off buying an rtx pro 6000 outright. This holds both for the sticker price, the supporting hardware price, the electricity cost to run it and cooling the room that you use it in.
Post reply on HN