Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia's Risky Business

stratechery.com

Post reply on HN
Print view View original

61–70 of 185 posts

Re: Nvidia's Risky Business

#61

Post by leoc »

buildbot wrote: post #55
pletnes wrote: post #40

Earlier quoted context omitted.

Nvidia sell iot boards with unified architecture. Would not be shocked if they launch pc/laptop/server boards at some point.

Unified memory DGX Spark and RTX Spark laptops are already a thing :)

Right, it's clear that nVidia is taking care and trying to position itself so that it can continue making sales if and when inference goes local. And it's in a much better intrinsic position to do that than the LLM SaaS vendors are: nVidia sells shovels to the army, but it also knows how to sell shovels to Walmart. Whether the financial relationships that Huang's got his company into will cause it problems if the market shifts is a different question, though.

Re: Nvidia's Risky Business

#62

Post by amelius »

onlyrealcuzzo wrote: post #21
jcfrei wrote: post #17

In many investment theses - like Nvidia's bet that demand for compute will keep growing - the first order assumption is usually correct. Yes, demand for more compute, chips, infrastructure is huge and each year some additional data centers will be built. Where such investment bets usually fail is in the second-order assumptions: Ie. the expectation of the growth of demand. This is where there's a high chance that the…

What makes this insanely hard to predict is that the compute needed for the same quality output has roughly gone down 90% every 18 months for ~5 years. 1) We don't know how long that trend will continue, but you do know where to look for when it may end (if smaller sized models continue to compress the knowledge effectively of larger models). 2) We don't know when the appetite for higher cost models might go down and…

Nvidia's great superpower is flexibility. You can easily run models of very different types on the same card; and their hardware is great for R&D.

However, at some point AI may be good enough for most people and then it makes sense to make an ASIC for the model (or group of models); and at that point you don't need Nvidia.

I suppose this scenario will happen in various moments at different levels.

Re: Nvidia's Risky Business

#63

Post by petesergeant »

> After the departure of DeepMind CEO Demis Hassabis (technically promoted to chairman, but no longer in charge of day-to-day operations) and Gemini co-lead and former Chief Scientist Jeff Dean, along with a host of other prominent researchers, SemiAnalysis declared that Gemini is Cooked: "For all intents and purposes, we believe DeepMind is no longer a frontier lab"

Counterpoint: xAI pooped out a frontier model based on nothing but capital and one man's desire to push a right-wing political narrative. Google has the talent, and the money, and the experience, they just need some leadership.

Re: Nvidia's Risky Business

#64

Post by officeplant »

bigyabai wrote: post #51
synergy20 wrote: post #42

Earlier quoted context omitted.

that undercuts their core business, so it will be a defensive play at most to fend off mac and amd's local inference offerings

I don't know how people can say this with a straight face. Nvidia was selling desktop-grade ARM SOCs before Apple Silicon was ever announced, specifically for edge robotics, computer vision and ML. The absolute fastest desktop Mac GPUs cannot beat an Nvidia laptop GPU in prefill or inference speeds. Apple Silicon is a non-entity for professional datacenter deployment and arguably unusable for frontier models at agent…

>Nvidia was selling desktop-grade ARM SOCs before Apple Silicon was ever announced

You can believe all you want that the dinky little jetson boards were desktop grade when historically the ARM SoC portion of a jetson board couldn't even keep up with broadcom/rockchip SoCs. It's taken until recently for the actual arm compute portion of Nvidia SoC's to be worth a damn at all, and they still fall far behind Apple let alone the rest of the pack like Qualcomm/Samsung.

Re: Nvidia's Risky Business

#65

Post by officeplant »

ekianjo wrote: post #12
wongarsu wrote: post #3

Earlier quoted context omitted.

Even in the west, Nvidia's dominance is bound to weaken. There is a notable uptick of articles on HN about people running large models on AMD hardware. And while I don't know official sales figures, I know we have trouble getting our AMD system delivered AMD's software story is still a lot worse than Nvidia's. But patching up vllm to run one or two models you care about on AMD hardware is a much easier proposition th…

amd is not putting nearly enough effort to improve their software its almost suspicious

They've struggled with software even in the ATi days. It's just normal AMD behavior. Drivers will always be half baked.

Re: Nvidia's Risky Business

#66

Post by wslh »

pletnes wrote: post #40
dzonga wrote: post #36

Nvidia has been playing a dangerous but profitable game since the Crypto boom. but now I think they probably have bitten more than they can chew. Apple already proved with their unified memory - that as long you have the capacity you can run capable models locally - thereby goes demand for inference if everyone is running some model locally. For training - Chinese models have proved that you don't need the latest & g…

Nvidia sell iot boards with unified architecture. Would not be shocked if they launch pc/laptop/server boards at some point.

Yes. The more capable SoC PCs become, the weaker the "everything has to run in the datacenter" argument gets. As more powerful SoC PCs from multiple vendors appear over the next few years, that advantage may shrink for many workloads.

Re: Nvidia's Risky Business

#67

Post by znnajdla »

There's another factor which Ben failed to consider. Which is that NVIDIA doesn't need to rely on demand for their proprietary CUDA stack or their GPUs growing -- they are already selling directly to the consumer, and likely capturing much higher margins. They are moving up stack, not down, where demand for raw compute matters less. With the DGX Spark and Jensen’s statement about “open models”, their next product is likely a strong hint: consumer devices to fulfill the Mac Mini demand craze. They are probably going to start burning LLMs durectly onto sillicon and then selling DeepSeek-in-your-home to individual developers. I bet that would sell even better than Anthropic Max coding plans and is not dependent on hyperscaler funded boom-bust cycles. So Ben’s analysis highlights the risk of their existing business not growing but they are likely planning new businesses.

Re: Nvidia's Risky Business

#68

Post by YuechenLi »

Nvidia's biggest advantage in AI has never been only their hardware performance but how entrenched their software is in ML research that flowed down stream. However, if you've actually used CUDA C/C++, it's pretty one of the worst software development ecosystem imaginable: you get all the footgun of regular C++, plus GPU compute pretending to be C++ and but doesn't actually behave like C++ because CPU and GPU compute are fundamentally different, and the only reason people put up with it is because Vulkan and HIP C/C++ are even worse.

Google's limitation is that they still don't offer TPUs in a PCI-E card/dev board that people can plug in to their PC for local development and sane low level API to develop against, instead you have to go through their cloud and their full software stack which greatly limits ecosystem growth. The minute that Google figures that out, that's when Nvidia's dominance would be challenged.

Re: Nvidia's Risky Business

#69

Post by adrianN »

mattnewton wrote: post #33
onlyrealcuzzo wrote: post #21

Earlier quoted context omitted.

What makes this insanely hard to predict is that the compute needed for the same quality output has roughly gone down 90% every 18 months for ~5 years. 1) We don't know how long that trend will continue, but you do know where to look for when it may end (if smaller sized models continue to compress the knowledge effectively of larger models). 2) We don't know when the appetite for higher cost models might go down and…

I think efficiency is unlikely to result in lower demand for compute, instead more useful compute per watt increases the value of that compute; and we are not going to run out of economically useful things to do with it anytime soon on the demand side. The harder thing to forecast for me is if we hit a wall on increasing efficiency, either on the model weights side or silicon side, with current approaches. If we have…

When efficiency reaches the point where local models on consumer hardware are good enough, demand for cloud tokens could rapidly shrink.

Re: Nvidia's Risky Business

#70

Post by dnnehgf »

so short them. if you think that the demand for skilled-labor-substitutive capital is saturable in the medium term or that improvements at the model level eat those at the hardware/cuda level or that nvidia just has the timing wrong, short them.
Post reply on HN