Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

huggingface.co

Post reply on HN
Print view View original

381–390 of 485 posts

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#381

Post by a96 »

linkage wrote: post #327
WhyOhWhyQ wrote: post #303

Earlier quoted context omitted.

I don't see what OpenAI's niche is supposed to be, other than role playing? Google seems like they'll be the AI utility company, and Anthropic seems like the go-to for the AI developer platform of the future.

Anthropic has RLed the shit out of their models to the extent that they give sub-par answers to general purpose questions. Google has great models but is institutionally incapable of building a cohesive product experience. They are literally shipping their org chart with Gemini (mediocre product), AI Overview (trash), AI Mode (outstanding but limited modality), Gemini for Google Workspace (steaming pile), Gemini on A…

RLed?

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#382

Post by a96 »

storus wrote: post #120
embedding-shape wrote: post #113

Earlier quoted context omitted.

> I run it all the time, token generation is pretty good. I feel like because you didn't actually talk about prompt processing speed or token/s, you aren't really giving the whole picture here. What is the prompt processing tok/s and the generation tok/s actually like?

I addressed both points - I mentioned you can offload token prefill (the slow part, 9t/s) to DGX Spark. Token generation is at 6t/s which is acceptable.

So, quarter stroll.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#383

Post by cherioo »

spullara wrote: post #62

I hate that their model ids don't change as they change the underlying model. I'm not sure how you can build on that. % curl https://api.deepseek.com/models \ -H "Authorization: Bearer ${DEEPSEEK_API_KEY}" {"object":"list","data":[{"id":"deepseek-chat","object":"model","owned_by":"deepseek"},{"id":"deepseek-reasoner","object":"model","owned_by":"deepseek"}]}

Allegedly deepseek is doing this because they don’t have enough gpu to serve two models concurrently.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#384

Post by Aachen »

bigyabai wrote: post #11
TIPSIO wrote: post #9

It's awesome that stuff like this is open source, but even if you have a basement rig with 4 NVIDIA GeForce RTX 5090 graphic cards ($15-20k machine), can it even run with any reasonable context window that isn't like a crawling 10/tps? Frontier models are far exceeding even the most hardcore consumer hobbyist requirements. This is even further

People with basement rigs generally aren't the target audience for these gigantic models. You'd get much better results out of an MoE model like Qwen3's A3B/A22B weights, if you're running a homelab setup.

Who is the target audience of these free releases? I don't mind free and open information sharing but I have wondered what's in it for the people that spent unholy amounts of energy on scraping, developing, and training

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#386

Post by eru »

timschmidt wrote: post #376
eru wrote: post #359

Earlier quoted context omitted.

During the 1990s (and for some years before and after) we got 'Dennard scaling'. The frequency of processors tended to increase exponentially, too, and featured prominently in advertising and branding. I suspect many people conflated Dennard scaling with Moore's law and the demise of Dennard scaling is what contributes to the popular imagination that Moore's law is dead: frequencies of processors have essentially sta…

Yup. Since then we've seen scaling primarily in transistor count, though clock speed has increased slowly as well. Increased transistor count has led to increasingly complex and capable instruction decode, branch prediction, out of order execution, larger caches, and wider execution pipelines in attempt to increase single-threaded performance. We've also seen the rise of embarrassingly parallel architectures like GPU…

Yes. Increase in transistor count is what the original Moore's law was about. But during the golden age of Dennard scaling it was easy to get confused.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#387

Post by bogdan »

corimaith wrote: post #302
deaux wrote: post #261

Earlier quoted context omitted.

Ah, so exactly like Uber, Netflix, Microsoft, Amazon, Facebook and so on have done to the rest of the world over the last few decades then? Where do you think they learnt this trick? Years lurking on HN and this post's comment section wins #1 on the American Hypocrisy chart. Unbelievable that even in the current US people can't recognize when they're looking in the mirror. But I guess you're disincentivized to do so…

Except domestic alternatives to the tech companies you listed were not driven out by them, they still exist today with substantial market share. American tech dominance elsewhere has more to do a lack of competition, and when competition does exist they're more often than not held at a disadvantage by domestic governments. So your counter narrative is false here.

> American tech dominance elsewhere has more to do a lack of competition

If that's true, why doesn't America compete on this front against China?

> they're more often than not held at a disadvantage by domestic governments

So when the US had the policy advantage over the EU it was just the market working, but when China has the policy advantage over the US it suddenly becomes unfair?

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#388

Post by js8 »

blonder wrote: post #294
observationist wrote: post #202

Earlier quoted context omitted.

Because Moore's law marches on. We're around 35-40 orders of magnitude from computers now to computronium. We'll need 10-15 years before handheld devices can run a couple terabytes of ram, 64-128 terabytes of storage, and 80+ TFLOPS. That's enough to run any current state of the art AI at around 50 tokens per second, but in 10 years, we're probably going to have seen lots of improvements, so I'd guess conservatively…

I appreciate your rabid optimism, but considering that Moores Law has ceased to be true for multiple years now I am not sure a handwave about being able to scale to infinity is a reasonable way to look at things. Plenty of things have slowed down in progress in our current age, for example airplanes.

You could put 64TBs of storage into your pocket with current technology. There are 4TB microSD cards available.

Not sure about the stated GFlops.. but I suspect we find that AI doesn't need that much compute to begin with.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#389

Post by fragmede »

js8 wrote: post #388
blonder wrote: post #294

Earlier quoted context omitted.

I appreciate your rabid optimism, but considering that Moores Law has ceased to be true for multiple years now I am not sure a handwave about being able to scale to infinity is a reasonable way to look at things. Plenty of things have slowed down in progress in our current age, for example airplanes.

You could put 64TBs of storage into your pocket with current technology. There are 4TB microSD cards available. Not sure about the stated GFlops.. but I suspect we find that AI doesn't need that much compute to begin with.

You can run models locally on high end smartphones today with apps like PocketPal or Local LLM.

Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]

#390

Post by timschmidt »

eru wrote: post #386
timschmidt wrote: post #376

Earlier quoted context omitted.

Yup. Since then we've seen scaling primarily in transistor count, though clock speed has increased slowly as well. Increased transistor count has led to increasingly complex and capable instruction decode, branch prediction, out of order execution, larger caches, and wider execution pipelines in attempt to increase single-threaded performance. We've also seen the rise of embarrassingly parallel architectures like GPU…

Yes. Increase in transistor count is what the original Moore's law was about. But during the golden age of Dennard scaling it was easy to get confused.

Agreed. And specifically Moore's law is about transistors per constant dollar. Because even in his time, spending enough could get you scaling beyond what was readily commercially available. Even if transistor count had stagnated, there is still a massive improvement from the $4,000 386sx Dad somehow convinced Mom to greenlight in the late 80s compared to a $45 Raspberry Pi today. And that factors into the equation as well.

Of course, feature size (and thus chip size) and cost are intimately related (wafers are a relatively fixed cost). And related as well to production quantity and yield (equipment and labor costs divide across all chips produced). That the whole thing continues scaling is non-obvious, a real insight, and tantamount to a modern miracle. Thanks to the hard work and effort of many talented people.

Post reply on HN