Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The impact of competition and DeepSeek on Nvidia

youtubetranscriptoptimizer.com

Post reply on HN
Print view View original

291–300 of 500 posts

Re: The impact of competition and DeepSeek on Nvidia

#291

Post by aurareturn »

meiraleal wrote: post #286
aurareturn wrote: post #253

Earlier quoted context omitted.

Companies in the mobile era took a decade or more to become profitable. For example, Uber and Airbnb. Why do you expect OpenAI to become profitable after 3 years of chatgpt?

Nobody expects it but what we know for sure is that they have burnt billions of dollars. If other startups can get there spending millions, the fact is that openai won't ever be profitable. And more important (for us), let the hiring frenzy start again :)

They have a ton of revenue and high gross margins. They burn billions because they need to keep training ever better models until the market slows and competition consolidates.

Re: The impact of competition and DeepSeek on Nvidia

#292

Post by thijson »

vFunct wrote: post #264
pjdesno wrote: post #176

The description of DeepSeek reminds me of my experience in networking in the late 80s - early 90s. Back then a really big motivator for Asynchronous Transfer Mode (ATM) and fiber-to-the-home was the promise of video on demand, which was a huge market in comparison to the Internet of the day. Just about all the work in this area ignored the potential of advanced video coding algorithms, and assumed that broadcast TV-q…

I worked on a network that used a protocol very similar to ATM (actually it was the first Iridium satellite network). An internet based on ATM would have been amazing. You’re basically guaranteeing a virtual switched circuit, instead of the packets we have today. The horror of packet switching is all the buffering it needs, since it doesn’t guarantee circuits. Bandwidth is one thing, but the real benefit is that ATM…

I remember my professor saying how the fixed packet size in ATM (53 bytes) was a committee compromise. North America wanted 64 bytes, Europe wanted 32 bytes. The committee chose around the midway point.

Re: The impact of competition and DeepSeek on Nvidia

#293

Post by lxgr »

rfoo wrote: post #285
lxgr wrote: post #275

The most important part for me is: > DeepSeek is a tiny Chinese company that reportedly has under 200 employees. The story goes that they started out as a quant trading hedge fund similar to TwoSigma or RenTec, but after Xi Jinping cracked down on that space, they used their math and engineering chops to pivot into AI research. I guess now we have the answer to the question that countless people have already asked: W…

This is completely fake though. It was more like their founder decided to start a branch to do AI research. It was well planned, they bought significantly more GPUs than they can use for quant research even before they start to do anything AI. There was a crack down on algorithmic trading, but it didn't had much impact and IMO someone higher up definitely does not want to kill these trading firms.

The optimal amount of algorithmic trading is definitely more than none (I appreciate liquidity and price quality as much as the next guy), but arguably there's a case here that we've overshot a bit.

Re: The impact of competition and DeepSeek on Nvidia

#294

Post by liuliu »

andrewgross wrote: post #237
liuliu wrote: post #164

Earlier quoted context omitted.

It does. They have 256 experts per MLP layer, and some shared ones. The minimal deployment for decoding (aka. token generation) they recommend is 320 GPUs (H800). It is all in the DeepSeek v3 paper that everyone should read rather than speculating.

Got it. I’ll review the paper again for that portion. However, it still sounds like the end result is not VRAM savings but efficiently and speed improvements.

Yeah, if you look DeepSeek v3 paper deeper, each saving on each axis is understandable. Combined, they reach some magic number people can talk about (10x!): FP8: ~1.6 to 2x faster than BF16 / FP16; MLA: cut KV cache size by 4x (I think); MTP: converges 2x to 3x faster; DualPipe: maybe ~1.2 to 1.5x faster.

If you look deeper, many of these are only applicable to training (we already do FP8 for inference, MTP is to improve training convergence, and DualPipe is to overlapping communication / compute mostly for training purpose too). The efficiency improvement on inference IMHO is overblown.

Re: The impact of competition and DeepSeek on Nvidia

#295

Post by hn_throwaway_99 »

I'm curious if someone more informed than me can comment on this part:

> Besides things like the rise of humanoid robots, which I suspect is going to take most people by surprise when they are rapidly able to perform a huge number of tasks that currently require an unskilled (or even skilled) human worker (e.g., doing laundry ...

I've always said that the real test for humanoid AI is folding laundry, because it's an incredibly difficult problem. And I'm not talking about giving a machine clothing piece-by-piece flattened so it just has to fold, I'm talking about saying to a robot "There's a dryer full of clothes. Go fold it into separate piles (e.g. underwear, tops, bottoms) and don't mix the husband's clothes with the wife's". That is, something most humans in the developed world have to do a couple times a week.

I've been following some of the big advances in humanoid robot AI, but the above task still seems miles away given current tech. So is the author's quote just more unsubstantiated hype that I'm constantly bombarded with in the AI space, or have there been advancements recently in robot AI that I'm unaware of?

Re: The impact of competition and DeepSeek on Nvidia

#296

Post by [deleted] »

mmiliauskas wrote: post #218
startupsfail wrote: post #201

Earlier quoted context omitted.

- Fail at the above. I don’t think this is what happened with DeepSeek. It seems that they’ve genuinely optimized their model for efficiency and used GPUs properly (tiled FP8 trick and FP8 training). And came out on top. The impact on the NVIDIA stock is ridiculous. DeepSeek took the advantage of flexible GPU architecture (unlike inflexible hardware acceleration).

This is what I still don't understand, how much of what they claim has been actually replicated? From what I understand the "50x cheaper" inference is coming from their pricing page, but is it actually 50x cheaper than the best open source models?

[deleted]

Re: The impact of competition and DeepSeek on Nvidia

#297

Post by auntienomen »

lxgr wrote: post #275

The most important part for me is: > DeepSeek is a tiny Chinese company that reportedly has under 200 employees. The story goes that they started out as a quant trading hedge fund similar to TwoSigma or RenTec, but after Xi Jinping cracked down on that space, they used their math and engineering chops to pivot into AI research. I guess now we have the answer to the question that countless people have already asked: W…

DeepSeek is a subsidiary of a relatively successful Chinese quant trading firm. It was the boss' weird passion project, after he made a few billion yuan from his other passion, trading. The whole thing was funded by quant trading profits, which kind of undermines your argument. Maybe we should just let extremely smart people work on the things that catch their interest?

Re: The impact of competition and DeepSeek on Nvidia

#298

Post by creato »

ghostzilla wrote: post #252
rightbyte wrote: post #215

Earlier quoted context omitted.

Unique, ye, but isn't their method open? I read something about a group replicating a smaller variant of their main model.

Which brings the question, if LLMs are an asset of such strategic value, why did China allow the DeepSeek to be released? I see two possibilities here, either that the CCP is not that all-reaching as we think, or that the value of the technology isn't critical, and that the release was further cleared with the CCP and maybe even timed to come right after Trump's announcement of American AI supremacy.

I really doubt there was any intention behind it at all. I bet deepseek themselves are surprised at the impact this is having, and probably regret releasing so much information into the open.

Re: The impact of competition and DeepSeek on Nvidia

#299

Post by fairity »

aurareturn wrote: post #291
meiraleal wrote: post #286

Earlier quoted context omitted.

Nobody expects it but what we know for sure is that they have burnt billions of dollars. If other startups can get there spending millions, the fact is that openai won't ever be profitable. And more important (for us), let the hiring frenzy start again :)

They have a ton of revenue and high gross margins. They burn billions because they need to keep training ever better models until the market slows and competition consolidates.

The counter argument is that they won't be able to sustain those gross margins when the market matures because they don't have an effective moat.

In this world, R&D costs and gross margin/revenue are inextricably correlated.

Re: The impact of competition and DeepSeek on Nvidia

#300

Post by tomrod »

simpaticoder wrote: post #287
tomrod wrote: post #158

Earlier quoted context omitted.

Conversely, how much larger can you scale if frontier models only currently need 3 consumer computers? Imagine having 300. Could you build even better models? Is DeepSeek the right team to deliver that, or can OpenAI, Meta, HF, etc. adapt? Going to be an interesting few months on the market. I think OpenAI lost a LOT in the board fiasco. I am bullish on HF. I anticipate Meta will lose folks to brain drain in response…

>Imagine having 300. Would it not be useful to have multiple independent AIs observing and interacting to build a model of the world? I'm thinking something roughly like the "councelors" in the Civilization games, giving defense/economic/cultural advice, but generalized over any goal-oriented scenario (and including one to take the "user" role). A group of AIs with specific roles interacting with each other seems lik…

Yes; to my understanding that is MoE.
Post reply on HN