Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

huggingface.co

Post reply on HN
Print view View original

391–400 of 485 posts

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#391

Post by ptsneves »

deaux wrote: post #261
ptsneves wrote: post #107

Earlier quoted context omitted.

This is the rare earth minerals dumping all over again. Devalue to such a price as to make the market participants quit, so they can later have a strategic stranglehold on the supply. This is using open source in a bit of different spirit than the hacker ethos, and I am not sure how I feel about it. It is a kind of cheat on the fair market but at the same time it is also costly to China and its capital costs may beco…

Ah, so exactly like Uber, Netflix, Microsoft, Amazon, Facebook and so on have done to the rest of the world over the last few decades then? Where do you think they learnt this trick? Years lurking on HN and this post's comment section wins #1 on the American Hypocrisy chart. Unbelievable that even in the current US people can't recognize when they're looking in the mirror. But I guess you're disincentivized to do so…

Not American and I also agree that the current big techs should be broken up by force of the state, there is a very big difference between a company becoming monopolistic due to market forces, and a company becoming monopolistic due to state strategy, intervention, backing.

Things can be bad in a spectrum and I believe it is much easier for society/state to break up a capitalistic monopoly than a state backed monopoly. To illustrate, the state has sued some of those companies and they were seriously threatened, because of competition ills. That is not the case with a state company.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#392

Post by chronogram »

nickandbro wrote: post #289

For anyone that is interested "create me a svg of a pelican riding on a bicycle" https://www.svgviewer.dev/s/FhqYdli5

It created a whole webpage to showcase the SVG with animation for me: https://output.jsbin.com/qeyubehate

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#393

Post by eru »

timschmidt wrote: post #390
eru wrote: post #386

Earlier quoted context omitted.

Yes. Increase in transistor count is what the original Moore's law was about. But during the golden age of Dennard scaling it was easy to get confused.

Agreed. And specifically Moore's law is about transistors per constant dollar. Because even in his time, spending enough could get you scaling beyond what was readily commercially available. Even if transistor count had stagnated, there is still a massive improvement from the $4,000 386sx Dad somehow convinced Mom to greenlight in the late 80s compared to a $45 Raspberry Pi today. And that factors into the equation a…

The way I remember it, it was about the transistor count in the commercially available chip with the lowest per transistor cost. Not transistor count per constant dollar.

Wikipedia quotes it as:

> The complexity for minimum component costs has increased at a rate of roughly a factor of two per year. Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for at least 10 years.

But I'm fairly sure, if you graph how many transistors you can buy per inflation adjusted dollar, you get a very similar graph.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#394

Post by KvanteKat »

moffkalast wrote: post #380
ericskiff wrote: post #53

Earlier quoted context omitted.

I believe this was a statement on cost per token to us as consumers of the service

Training cost-effectiveness doesn't matter for open models since someone else ate the cost. In this case, Chinese taxpayers.

Deepseek is a private corporation funded by a hedge fund (High-Flyer). I doubt much public money was spent by the Chinese state on this. Like with LLMs in the US, the people paying for it so far are mainly investors who are betting on a return in the long to medium term.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#395

Post by wickedsight »

potsandpans wrote: post #111
TIPSIO wrote: post #9

It's awesome that stuff like this is open source, but even if you have a basement rig with 4 NVIDIA GeForce RTX 5090 graphic cards ($15-20k machine), can it even run with any reasonable context window that isn't like a crawling 10/tps? Frontier models are far exceeding even the most hardcore consumer hobbyist requirements. This is even further

I run a bunch of smaller models on a 12gb vram 3060 and it's quite good. For larger open models ill use open router. I'm looking into on- demand instances with cloud/vps providers, but haven't explored the space too much. I feel like private cloud instances that run on demand is still in the spirit of consumer hobbyist. It's not as good as having it all local, but the bootstrapping cost plus electricity to run seems…

Which ones are your favorites that fit on the 3060?

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#396

Post by throwdbaaway »

embedding-shape wrote: post #74

> DeepSeek-V3.2 introduces significant updates to its chat template compared to prior versions. The primary changes involve a revised format for tool calling and the introduction of a "thinking with tools" capability. At first, I thought they had gone the route of implementing yet another chat format that can handle more dynamic conversations like that, instead of just using Harmony, but looking at the syntax, doesn'…

That DSML in the encoding directory looks quite a bit different from the Harmony chat template.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#397

Post by embedding-shape »

storus wrote: post #120
embedding-shape wrote: post #113

Earlier quoted context omitted.

> I run it all the time, token generation is pretty good. I feel like because you didn't actually talk about prompt processing speed or token/s, you aren't really giving the whole picture here. What is the prompt processing tok/s and the generation tok/s actually like?

I addressed both points - I mentioned you can offload token prefill (the slow part, 9t/s) to DGX Spark. Token generation is at 6t/s which is acceptable.

6 tok/sec might be acceptable for a dense model that doesn't do thinking, but for something like DeepSeek 3.2 that does do reasoning, 6 tok/sec isn't acceptable for anything else but async/batched stuff, sadly. Even for a response with just 100 tokens we're talking a minute for it to just write the response, for anything except the smallest of prompts you'll easily be hitting 1000 tokens (600 seconds!).

Maybe my 6000 Pro spoiled me, but for actual usage, 6 or even 9 tok/sec is too slow for a reasoning/thinking model. To be honest, kind of expected on CPU though. I guess it's cool that it can run on Apple hardware, but it isn't exactly a pleasant experience at least today.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#398

Post by timschmidt »

eru wrote: post #393
timschmidt wrote: post #390

Earlier quoted context omitted.

Agreed. And specifically Moore's law is about transistors per constant dollar. Because even in his time, spending enough could get you scaling beyond what was readily commercially available. Even if transistor count had stagnated, there is still a massive improvement from the $4,000 386sx Dad somehow convinced Mom to greenlight in the late 80s compared to a $45 Raspberry Pi today. And that factors into the equation a…

The way I remember it, it was about the transistor count in the commercially available chip with the lowest per transistor cost. Not transistor count per constant dollar. Wikipedia quotes it as: > The complexity for minimum component costs has increased at a rate of roughly a factor of two per year. Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the rate…

Yes. I think you're probably right about phrasing. And transistor count per inflation adjusted dollar is the unit most commonly used to graph it. Similar ways to say the same thing.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#399

Post by Aeolun »

nylonstrung wrote: post #344
chazeon wrote: post #292

Earlier quoted context omitted.

Well the seemingly cheap comes with significantly degraded performance, particular for agentic use. Have you tried replacing Claude Code with some locally deployed model, say, on 4090 or 5090? I have. It is not usable.

Deepseek and Kimi both have great agentic performance When used with crush/opencode they are close to Claude performance. Nothing that runs on a 4090 would compete but Deepseek on openrouter is still 25x cheaper than claude

> Deepseek on openrouter is still 25x cheaper than claude

Is it? Or only when you don’t factor in Claude cached context? I’ve consistently found it pointless to use open models because the price of the good ones is so close to cached context on Claude that I don’t need them.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#400

Post by joefourier »

Aeolun wrote: post #399
nylonstrung wrote: post #344

Earlier quoted context omitted.

Deepseek and Kimi both have great agentic performance When used with crush/opencode they are close to Claude performance. Nothing that runs on a 4090 would compete but Deepseek on openrouter is still 25x cheaper than claude

> Deepseek on openrouter is still 25x cheaper than claude Is it? Or only when you don’t factor in Claude cached context? I’ve consistently found it pointless to use open models because the price of the good ones is so close to cached context on Claude that I don’t need them.

Deepseek via their API also has cached context, although the tokens/s was much lower than Claude when I tried it. But for background agents the price difference makes it absolutely worth it.
Post reply on HN