Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek open source DeepEP – library for MoE training and Inference

github.com

Post reply on HN
Print view View original

41–50 of 77 posts

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#41

Post by woctordho »

antirez wrote: post #35
pama wrote: post #7

I feel like a kid in a candy shop. Some of these tricks would take way too long to reverse engineer correctly based on the papers. I hope that the releases this week start a renaissance of the use of MoE as baseline academic models.

From this point of view I don't understand what's happening between the actual SOTA models practice and the academic models. The former at this point are all MoEs, starting with GPT4. But then the open models, if not for DeepSeek V3 and Mixtral, are always dense models.

MoEs require less computation and more memory, so they're harder to setup in small labs

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#42

Post by anticensor »

alpb wrote: post #25
echelon wrote: post #11

Earlier quoted context omitted.

I hope you're reading this Sam Altman: Make Open AI open . Or else you'll lose to the ecosystem.

That’s an impossible ask. Sam is the pinnacle of capitalist ruling class, he’s a pure businessman. He has no interest in giving anything for free unless there’s a business plan. He doesn’t care about humanity. He’ll pretend to change the world and tell you that they’re inventing AGI, Q*, strawberry or whatever they’re branding it, but the reality is he knows it’s all over and unless there’s a major breakthrough this…

Their state-of-the-art speech to text model, Whisper, is available as open weights for free.

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#43

Post by ghfhghg »

WiSaGaN wrote: post #36
gostsamo wrote: post #10

Earlier quoted context omitted.

They do it for their own reasons, but OpenAI are straight up liars and they are neither open nor give a fuck about humanity.

It would be hilarious if this scenario played out. OpenAI starts as a nonprofit, aiming to benefit all humanity. Eventually, they discover a path to AGI and engage in intense internal debates: Should they abandon their original mission and chase profit, knowing it could bring generational wealth? They ultimately decide, "To hell with humanity—let’s go for the money." As they pivot to prioritizing profit, DeepSeek eme…

Admittedly I'm a sideline observer but it feels like the first half of your scenario is already happening (sans the agi).

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#44

Post by saagarjha »

rfoo wrote: post #33
ta988 wrote: post #26

Earlier quoted context omitted.

Parallel Thread Execution. Think of them as opcodes for the Nvidia GPUs. They are a bit more complex that your traditional opcodes (the lowest level of abstraction accessible to users) in CPUs, as you can specify cache parameters, memory barriers etc. There are documented combinations of parameters for those instructions but if you fuzz (search new combinations in a random or organized way because you hope some will…

Gonna check what SASS it get translated to and whether it makes any sense. I wonder if they had SASS assembler for Hopper (either by reverse engineering nvdisasm or by fuzzing instructions + nvdisasm + stare hard) and don't want to say it out :p

You'd be looking at ptxas here. FWIW, it looks like it generates LDG.E.NA.LTC256B.U8.CONSTANT on my machine.

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#45

Post by saagarjha »

magicalhippo wrote: post #23
zardinality wrote: post #14

Earlier quoted context omitted.

"For extreme performance, we discover and use a behavior-out-of-doc PTX instruction: ld.global.nc.L1::no_allocate.L2::256B. This instruction will lead to an undefined behavior: accessing volatile GPU memory with non-coherent read-only PTX modifiers .nc. But the correctness is tested to be guaranteed with .L1::no_allocate on Hopper architectures, and performance will be much better. If you find kernels not working on…

So non-coherent refers to bypassing cache coherency, ie don't care about what other units might have written to that address? And the L1/L2 modifiers are to avoid L1 thrashing, keeping the value in L2 only? Or did I get that wrong?

Yeah that's about right

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#46

Post by yieldcrv »

WiSaGaN wrote: post #36
gostsamo wrote: post #10

Earlier quoted context omitted.

They do it for their own reasons, but OpenAI are straight up liars and they are neither open nor give a fuck about humanity.

It would be hilarious if this scenario played out. OpenAI starts as a nonprofit, aiming to benefit all humanity. Eventually, they discover a path to AGI and engage in intense internal debates: Should they abandon their original mission and chase profit, knowing it could bring generational wealth? They ultimately decide, "To hell with humanity—let’s go for the money." As they pivot to prioritizing profit, DeepSeek eme…

"I don't want to live in a world where someone else is making the world a better place better than we are"

- Silicon Valley Season 2

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#47

Post by yieldcrv »

so while the US is chasing GPU receipts in Singapore just to ensure DeepSeek was using H800s only, the rest of the world can run these optimizations on the full H100s?

while we also pretend that H100s were difficult to get or access because of the US sanctions and their hubris to believe their edicts blanket the globe?

am I understanding this correctly?

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#48

Post by ur-whale »

hackit2 wrote: post #8
grg0 wrote: post #6

Earlier quoted context omitted.

Open AI™ (with a space)

Kind of ironic that DeepSeek is more Open than ChatGPT

> Kind of ironic that DeepSeek is more Open than ChatGPT

Not ironic at all.

You've simply be lied to by OpenAI.

Nothing ironic about being naive.

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#49

Post by ur-whale »

echelon wrote: post #11
grg0 wrote: post #6

Earlier quoted context omitted.

Open AI™ (with a space)

I hope you're reading this Sam Altman: Make Open AI open . Or else you'll lose to the ecosystem.

> I hope you're reading this Sam Altman

I hope he's not.

All he deserves at this point is to go down as hard as possible.

Re: DeepSeek open source DeepEP – library for MoE training and Inference

#50

Post by ur-whale »

The incentive behind the work of DeepSeek might very well be wrong (something along the lines of a state-sponsored attempt at shrinking the US first mover advantage in AI to nil) but the net result for everyone on the planet is simply fantastic.

So even in the worst case (doing this for the wrong reasons): thank you DeepSeek, you are actually doing what OpenAI lied through their teeth to the whole world about doing for years.

You rock.

Post reply on HN