Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia's Risky Business

stratechery.com

Post reply on HN
Print view View original

71–80 of 185 posts

Re: Nvidia's Risky Business

#71

Post by bigyabai »

coredog64 wrote: post #56
acdha wrote: post #26

Earlier quoted context omitted.

Nobody is saying they’re outright failing, but that they’re not going to be printing money the way they have been recently. Think about Intel circa 2010: most of their competitors like POWER or MIPS were marginalized, they owned the desktop and server markets with a bit of competition from AMD well contained, and their biggest desktop competitor (Apple) had just switched. A lot of analyst predictions … did not match…

> The more money Nvidia makes, the more motivated their competitors will be to get a piece of that market and the more customers will be looking for alternatives like the push into TPUs which the article discussed. Your margin is my opportunity - Jeff Bezos

But people have been saying this about CUDA for twenty years, and we are not any closer to a replacement GPGPU paradigm today.

The root comment in this thread was about Nvidia hedging their bet on lost AI market share. They recognize that a reduced pace in training and inference competition will undercut their business, but CUDA isn't a one-trick pony for LLMs alone. TPUs are - you can't even reuse the same architecture for training and inference, they're separate ASICs unlike CUDA cores/ALUs. Veterans of crypto mining will tell you that the ASICs lost in the end, as Nvidia was evolving their hardware faster than the ASIC manufacturers could iterate. When the crypto acceleration landscape diversified away from ETH/BTC into altcoins, Nvidia was still there making money hand-over-fist from mining hardware.

I guess you could argue that robotics, world models or computer vision won't be a trillion-dollar market. But Nvidia is positioned to be the first mover in all of these markets, and none of their competitors are even coming close to the integrated stack that they sell consumers.

Re: Nvidia's Risky Business

#72

Post by HeWhoLurksLate »

YuechenLi wrote: post #68

Nvidia's biggest advantage in AI has never been only their hardware performance but how entrenched their software is in ML research that flowed down stream. However, if you've actually used CUDA C/C++, it's pretty one of the worst software development ecosystem imaginable: you get all the footgun of regular C++, plus GPU compute pretending to be C++ and but doesn't actually behave like C++ because CPU and GPU compute…

I mean they had/have the Coral but that's in an entirely different market segment

Re: Nvidia's Risky Business

#73

Post by bigyabai »

officeplant wrote: post #64
bigyabai wrote: post #51

Earlier quoted context omitted.

I don't know how people can say this with a straight face. Nvidia was selling desktop-grade ARM SOCs before Apple Silicon was ever announced, specifically for edge robotics, computer vision and ML. The absolute fastest desktop Mac GPUs cannot beat an Nvidia laptop GPU in prefill or inference speeds. Apple Silicon is a non-entity for professional datacenter deployment and arguably unusable for frontier models at agent…

>Nvidia was selling desktop-grade ARM SOCs before Apple Silicon was ever announced You can believe all you want that the dinky little jetson boards were desktop grade when historically the ARM SoC portion of a jetson board couldn't even keep up with broadcom/rockchip SoCs. It's taken until recently for the actual arm compute portion of Nvidia SoC's to be worth a damn at all, and they still fall far behind Apple let a…

I don't have to believe. I've run KDE and GNOME on the Tegra boards, you get full-fat CUDA support without sacrificing Vulkan drivers. It's incredible.

You can believe all you want that good single-core performance will corner the edge compute market. It hasn't, Graviton has more buy-in than any Apple Silicon chip ever got.

Re: Nvidia's Risky Business

#74

Post by rcr-anti »

For awhile I've found two things hard to square, that the hardware and software making up current gen AI will bring us to a socioeconomic singularity, and the reality the thing they're mostly trying to emulate is a few pounds of meat and fat running on tens of watts equivalent. On one hand the current AIs are obviously super human in some tasks, get completely dunked on in others by far simpler organisms. My cat can catch a bug out of the air, Fable 5 in Cowork can lack the dexterity to make a slideshow because I had LibreOffice instead of Microsoft Office. Not even close to analogous, but point being they appear to have pretty fundamental differences in how they can interface with the world that the economic thesis seems to gloss over.

Re: Nvidia's Risky Business

#75

Post by hylaride »

onlyrealcuzzo wrote: post #21
jcfrei wrote: post #17

In many investment theses - like Nvidia's bet that demand for compute will keep growing - the first order assumption is usually correct. Yes, demand for more compute, chips, infrastructure is huge and each year some additional data centers will be built. Where such investment bets usually fail is in the second-order assumptions: Ie. the expectation of the growth of demand. This is where there's a high chance that the…

What makes this insanely hard to predict is that the compute needed for the same quality output has roughly gone down 90% every 18 months for ~5 years. 1) We don't know how long that trend will continue, but you do know where to look for when it may end (if smaller sized models continue to compress the knowledge effectively of larger models). 2) We don't know when the appetite for higher cost models might go down and…

It's also hard to predict how much money will be burned going down wrong avenues. The internet was the future, but it took a lot of failed companies to eventually land on a sustainable model that brought us the giants we have today.

Railways were also the future, but that didn't stop a rush to build out (often subsidized) lines that were ultimately uneconomical (either because they were corrupt or the planned settlements never arrived).

If AI is similar, then there's going to be a long slowdown on compute spend until the surplus is worked through. A good historical analogy could be the fiber optic buildouts of the late 1990s. The demand for data never really went down much, but the industry eventually commodified and took down some large companies (Nortel, especially)

Re: Nvidia's Risky Business

#76

Post by gizajob »

Laughable to cite and reiterate the idea that Google is cooked where it comes to SoTA and AI in general when they operate, reliably and successfully for decades, one of the largest computing infrastructures on Earth and will likely continue to usefully serve the 90% of AI requests that don’t involve managing large codebases. Also seems strange to suggest that Google would need to Aquihire a company like Thinking Machines when it could spin up their AI model in a couple of weeks on its own TPUs if it felt like it. Demis likely wants to focus on his specific interest at the junction of biochemistry, neurobiology and computation which is more specific and unique to Demis than building a general purpose Q&A search model.

Re: Nvidia's Risky Business

#77

Post by KaseyKim »

Ekaros wrote: post #50
georgemcbay wrote: post #49

Earlier quoted context omitted.

> What's the danger? They slide back down to being just a gaming graphics card company with a $10 share price? Nvidia dropping from being a $5 trillion company to a $242 billion company would be 1929 levels of bad. Global economy end of days stuff, especially since Nvidia can't crash that hard without a lot of other stuff crashing with it.

How much in value could Nvidia safely drop and over what period of time for it be fine for greater economy? What level of correction would be manageable? And I am pretty sure that their value will drop in 5 to 10 years.

Nvida's action may affect the semi- marketing

Re: Nvidia's Risky Business

#78

Post by bigyabai »

HeWhoLurksLate wrote: post #72
YuechenLi wrote: post #68

Nvidia's biggest advantage in AI has never been only their hardware performance but how entrenched their software is in ML research that flowed down stream. However, if you've actually used CUDA C/C++, it's pretty one of the worst software development ecosystem imaginable: you get all the footgun of regular C++, plus GPU compute pretending to be C++ and but doesn't actually behave like C++ because CPU and GPU compute…

I mean they had/have the Coral but that's in an entirely different market segment

Coral and the TPUs are ASICs, and therefore are barely reprogrammable. It doesn't really compare to the complexity and flexibility of CUDA ALUs.

Re: Nvidia's Risky Business

#79

Post by FuriouslyAdrift »

jcfrei wrote: post #17

In many investment theses - like Nvidia's bet that demand for compute will keep growing - the first order assumption is usually correct. Yes, demand for more compute, chips, infrastructure is huge and each year some additional data centers will be built. Where such investment bets usually fail is in the second-order assumptions: Ie. the expectation of the growth of demand. This is where there's a high chance that the…

They also have to be feeling the heat of the ASIC vendors. AMD just acquired Taalas and they work with Cerebras all the time on special projects. ASICs outgun nVidia's chips by an order of magnitude.

Re: Nvidia's Risky Business

#80

Post by musebox35 »

YuechenLi wrote: post #68

Nvidia's biggest advantage in AI has never been only their hardware performance but how entrenched their software is in ML research that flowed down stream. However, if you've actually used CUDA C/C++, it's pretty one of the worst software development ecosystem imaginable: you get all the footgun of regular C++, plus GPU compute pretending to be C++ and but doesn't actually behave like C++ because CPU and GPU compute…

The biggest advantage of tpus is the high bandwidth fiber optic interconnect between them that allows distributed computing on pods with thousands of tpus and the co-design of cooling systems that go with their racks. I do not think that we will see personal tpus any time soon.
Post reply on HN