> Meta scrambling 'war rooms' of engineers to figure out DeepSeek's AI
"Gentlemen, you can't fight in the war room."
Meta scrambling 'war rooms' of engineers to figure out DeepSeek's AI
11–20 of 21 posts
Re: Meta scrambling 'war rooms' of engineers to figure out DeepSeek's AI
#12Can anyone explain why Meta's share price was untouched by the deep seek announcement ? They have spent billions on AI infra? According to this article they are rattled in some way...
Expensive models are AI companies core business.
Meta can use cheap models to enhance core business.
Re: Meta scrambling 'war rooms' of engineers to figure out DeepSeek's AI
#13What a time to be alive. Chinese companies were copying everything from the west, now it seems the opposite.
(annoying voice) "Hold on to your papers...!"
Re: Meta scrambling 'war rooms' of engineers to figure out DeepSeek's AI
#14Can anyone explain why Meta's share price was untouched by the deep seek announcement ? They have spent billions on AI infra? According to this article they are rattled in some way...
They make all their money on ads in FB and IG. It's how their stock barely budged despite losing $30b on a VR ghost town.
Re: Meta scrambling 'war rooms' of engineers to figure out DeepSeek's AI
#15Meta has been aware of DeepSeek for a long time (as Zuckerberg mentioned the company by name in his podcast with Joe Rogan) and a “war room” is just a meeting room.
Re: Meta scrambling 'war rooms' of engineers to figure out DeepSeek's AI
#16Meta has been aware of DeepSeek for a long time (as Zuckerberg mentioned the company by name in his podcast with Joe Rogan) and a “war room” is just a meeting room.
My experience is that a "War Room" is just a meeting room, but one where 1. engineers are rounded up to work in (because as we all know, developers type code faster when co-located in a single room under pressure), and 2. where panicked executives occasionally wander in to say things like "How are things going?" and "What's the current status?" and "Do you have an ETA for when we can stop panicking?"
Re: Meta scrambling 'war rooms' of engineers to figure out DeepSeek's AI
#17I might be missing something, but DeepSeek’s recipe is right there in plain sight. Most of the cost efficiency of DeepSeek v3 seem to be attributable to MoE and FP8 training. DeepSeek R1s improvements are from GRPO-based RL. Interesting to note - we have no idea how much R1 cost to train. To speculate - maybe DeepSeek’s release made an upcoming Llama release moot in comparison.
What is different about Deepseek's use of MoE vs all the other MoE models that makes training more efficient?
FP8 training and GRPO make sense to me, but that only gets you a 4x improvement total, right?
Re: Meta scrambling 'war rooms' of engineers to figure out DeepSeek's AI
#18I might be missing something, but DeepSeek’s recipe is right there in plain sight. Most of the cost efficiency of DeepSeek v3 seem to be attributable to MoE and FP8 training. DeepSeek R1s improvements are from GRPO-based RL. Interesting to note - we have no idea how much R1 cost to train. To speculate - maybe DeepSeek’s release made an upcoming Llama release moot in comparison.
What is different about Deepseek's use of MoE vs all the other MoE models that makes training more efficient? FP8 training and GRPO make sense to me, but that only gets you a 4x improvement total, right?
They slightly restructure their MoE [1], but I think the main difference is that other big models (e.g Llama 504B) are dense and have higher FLOP requirements. MoE should represent a ~5x improvement. FP8 should be about a ~2x improvement.
We don’t know how much of a speed improvement GRPO represents. They didn’t say how many GPU hours went into to RLing DeepSeek-r1 and we don’t have a o1 numbers to compare.
There’s definitely lots of misinformation spreading though. The $5.5m number refers to Deepseek-v3, not Deepseek-r1. I don't want to take away from HighFlyer's accomplishment, though. I think a lot of these innovations were forced to work around H800 networking limitations, and it's impressive what they've done.
Re: Meta scrambling 'war rooms' of engineers to figure out DeepSeek's AI
#19Earlier quoted context omitted.
What is different about Deepseek's use of MoE vs all the other MoE models that makes training more efficient? FP8 training and GRPO make sense to me, but that only gets you a 4x improvement total, right?
They slightly restructure their MoE [1], but I think the main difference is that other big models (e.g Llama 504B) are dense and have higher FLOP requirements. MoE should represent a ~5x improvement. FP8 should be about a ~2x improvement. We don’t know how much of a speed improvement GRPO represents. They didn’t say how many GPU hours went into to RLing DeepSeek-r1 and we don’t have a o1 numbers to compare. There’s d…
It's interesting that only having access to less powerful hardware motivated/necessitated more efficient training--like how tariffs can backfire if left in place too long.
Re: Meta scrambling 'war rooms' of engineers to figure out DeepSeek's AI
#20Can anyone explain why Meta's share price was untouched by the deep seek announcement ? They have spent billions on AI infra? According to this article they are rattled in some way...
i think it's because openai makes a bunch of money off "AI stuff" by being regarded the best at this game... and guess what, there's a new player that makes "AI stuff" as good as them (or possibly better) and maybe even cheaper. this could be a threat to their source of revenue.
Meta on the other hand makes money off whatsapp, facebook, instagram and threads. for meta an additional provider of "AI stuff" is not a threat to their source of revenue.