Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek open source DeepEP – library for MoE training and Inference

github.com

Post reply on HN
Print view View original

31–40 of 77 posts

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#31

Post by ammo1662 »

k_sze wrote: post #29
mohsen1 wrote: post #22

> For extreme performance, we discover and use an out-of-doc PTX instruction: ld.global.nc.L1::no_allocate.L2::256B. This instruction will lead to an undefined behavior: accessing volatile GPU memory with non-coherent read-only PTX modifiers .nc. But the correctness is tested to be guaranteed with .L1::no_allocate on Hopper architectures, and performance will be much better.

Practically speaking, is it possible for NVIDIA to "pull the rug" later, intentionally or otherwise, by subtly changing the behaviour of this out-of-doc instruction on new architectures?

They could. That's why there is a switch to disable it.

> If you find kernels not working on some other platforms, you may add DISABLE_AGGRESSIVE_PTX_INSTRS=1 to setup.py and disable this, or file an issue.

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#32

Post by larodi »

gertop wrote: post #28
azinman2 wrote: post #20

Earlier quoted context omitted.

Now. It’s amazing to me that everyone is like fuck OpenAI deepseek is the savior, when OpenAI’s papers and code jump started an AI revolution just a few years ago. Let’s wait the same number of years and see what deepseek does.

I thought the papers that jump started the revolution came from Google?

Indeed. And the papers were about doing better translation of char sequences, essentially the tech emerged as linguistics improvement for language. Then someone realised the parrot learns enough ZIP and JPEG alongside and can spit back hazy memories of it all.

the one still super useful thing OpenAI ever released must’ve been Whisper. But they could’ve been much more open for sure.

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#33

Post by rfoo »

ta988 wrote: post #26
swyx wrote: post #15

Earlier quoted context omitted.

> the PTX that everyone was looking forward to explanation for the rest of us why this is so important?

Parallel Thread Execution. Think of them as opcodes for the Nvidia GPUs. They are a bit more complex that your traditional opcodes (the lowest level of abstraction accessible to users) in CPUs, as you can specify cache parameters, memory barriers etc. There are documented combinations of parameters for those instructions but if you fuzz (search new combinations in a random or organized way because you hope some will…

Gonna check what SASS it get translated to and whether it makes any sense.

I wonder if they had SASS assembler for Hopper (either by reverse engineering nvdisasm or by fuzzing instructions + nvdisasm + stare hard) and don't want to say it out :p

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#35

Post by antirez »

pama wrote: post #7

I feel like a kid in a candy shop. Some of these tricks would take way too long to reverse engineer correctly based on the papers. I hope that the releases this week start a renaissance of the use of MoE as baseline academic models.

From this point of view I don't understand what's happening between the actual SOTA models practice and the academic models. The former at this point are all MoEs, starting with GPT4. But then the open models, if not for DeepSeek V3 and Mixtral, are always dense models.

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#36

Post by WiSaGaN »

gostsamo wrote: post #10
hackit2 wrote: post #8

Earlier quoted context omitted.

Kind of ironic that DeepSeek is more Open than ChatGPT

They do it for their own reasons, but OpenAI are straight up liars and they are neither open nor give a fuck about humanity.

It would be hilarious if this scenario played out.

OpenAI starts as a nonprofit, aiming to benefit all humanity. Eventually, they discover a path to AGI and engage in intense internal debates: Should they abandon their original mission and chase profit, knowing it could bring generational wealth? They ultimately decide, "To hell with humanity—let’s go for the money."

As they pivot to prioritizing profit, DeepSeek emerges. Staying true to OpenAI’s original vision, DeepSeek open-sources everything, benefiting humanity and earning global admiration. Unintentionally, this move tanks OpenAI’s valuation. In the end, OpenAI fails to become the hero or secure the massive profits they chased. Instead, they leave behind a legacy rebranded as "ClosedAI"

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#39

Post by amelius »

gostsamo wrote: post #10
hackit2 wrote: post #8

Earlier quoted context omitted.

Kind of ironic that DeepSeek is more Open than ChatGPT

They do it for their own reasons, but OpenAI are straight up liars and they are neither open nor give a fuck about humanity.

Well, they do give us a great free tool to use, but that's where it ends and probably has some agenda behind it.

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#40

Post by saagarjha »

ta988 wrote: post #26
swyx wrote: post #15

Earlier quoted context omitted.

> the PTX that everyone was looking forward to explanation for the rest of us why this is so important?

Parallel Thread Execution. Think of them as opcodes for the Nvidia GPUs. They are a bit more complex that your traditional opcodes (the lowest level of abstraction accessible to users) in CPUs, as you can specify cache parameters, memory barriers etc. There are documented combinations of parameters for those instructions but if you fuzz (search new combinations in a random or organized way because you hope some will…

CPUs have instructions with similar semantics.
Post reply on HN