Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia's Risky Business

stratechery.com

Post reply on HN
Print view View original

121–130 of 185 posts

Re: Nvidia's Risky Business

#121

Post by ijidak »

YuechenLi wrote: post #68

Nvidia's biggest advantage in AI has never been only their hardware performance but how entrenched their software is in ML research that flowed down stream. However, if you've actually used CUDA C/C++, it's pretty one of the worst software development ecosystem imaginable: you get all the footgun of regular C++, plus GPU compute pretending to be C++ and but doesn't actually behave like C++ because CPU and GPU compute…

Genuine question. Given that LLMs are supposed to allow us to rewrite anything, and I am an LLM believer, what I don't understand is: how does CUDA continue to be a moat in a world where LLMs can rewrite entire software development stacks? If NVIDIA is right about AI, isn't this same technology going to erode the software side of this same software moat?

Re: Nvidia's Risky Business

#123

Post by dragonwriter »

mandeepj wrote: post #110
elictronic wrote: post #103

Earlier quoted context omitted.

Retraining a large high paid user base is often a non-starter. To put this in perspective, Boeing’s eventual retraining costs for all the pilots for the 737Max was around 5 billion dollars. Looking at software more specifically the Linux foundation reported based on software dev salaries in 2008 it would be 1.4 billion to only write the Linux kernel. Up until about 2023 there wasn’t enough money involved to have any…

Yet, Microsoft pulled it off with a new .NET Framework, and Apple with its new iOS SDK. There are many more examples besides those two.

Neither of those are a competitor replacing another, they are a same-vendor replacement. This is easier, because the company whose established product you are trying to displace is cooperating (because its you!) not actively resisting.

Re: Nvidia's Risky Business

#124

Post by Melatonic »

rcr-anti wrote: post #74

For awhile I've found two things hard to square, that the hardware and software making up current gen AI will bring us to a socioeconomic singularity, and the reality the thing they're mostly trying to emulate is a few pounds of meat and fat running on tens of watts equivalent. On one hand the current AIs are obviously super human in some tasks, get completely dunked on in others by far simpler organisms. My cat can…

You make a good point. Maybe the AI apocalypse will actually be when someone hooks up a cat brain to a super advanced AI.

Re: Nvidia's Risky Business

#125

Post by kevstev »

ravenstine wrote: post #84
YuechenLi wrote: post #68

Nvidia's biggest advantage in AI has never been only their hardware performance but how entrenched their software is in ML research that flowed down stream. However, if you've actually used CUDA C/C++, it's pretty one of the worst software development ecosystem imaginable: you get all the footgun of regular C++, plus GPU compute pretending to be C++ and but doesn't actually behave like C++ because CPU and GPU compute…

That's really interesting. I have no experience writing anything that involves GPUs/TPUs, but over the years I've consistently read that CUDA is the "real moat" of Nvidia, which I never totally believed, but the way you describe makes it seem like it's not actually a moat in the slightest. It just happens to be an ecosystem associated with hardware that is not only considered the gold standard but happens to be more…

I think OP is overstating it a bit tbh. So Nvidia has the market for the hardware, which helps, but there is literally no alternative to CUDA. Nvidia keeps it a scalpel for skilled users, its not super easy to use, but unlocks orders of more magnitudes of power for the use cases it excels at vs CPUs. I don't have access to anything like it in the Apple ecosystem.

AMD has had years to try and counter it, but just has not. Google is kinda trying to do an end run around it with TPUs but they are still niche high end stuff with limited availability.

Its really just CUDA, and CUDA can be seen as somewhat akin to C for assembly used by Nvidia's gpus- In many ways a wrapper around the low level hardware that often has those details bleed through.

Re: Nvidia's Risky Business

#126

Post by eterm »

mandeepj wrote: post #110
elictronic wrote: post #103

Earlier quoted context omitted.

Retraining a large high paid user base is often a non-starter. To put this in perspective, Boeing’s eventual retraining costs for all the pilots for the 737Max was around 5 billion dollars. Looking at software more specifically the Linux foundation reported based on software dev salaries in 2008 it would be 1.4 billion to only write the Linux kernel. Up until about 2023 there wasn’t enough money involved to have any…

Yet, Microsoft pulled it off with a new .NET Framework, and Apple with its new iOS SDK. There are many more examples besides those two.

Microsoft pulled it off with dotnet, sort of, because they approached it like a completely new language, sold people on the benefits of it.

And the people they were selling that to, ( It's free and open souce now! ), were a very different group to the market they left behind on .NET Framework, who are often still struggling to make the transition now.

Had they actually killed off .NET Framework, it would have been a different story, much more like the VB6/VBA to VB.NET transition, which so few people bothered with that VB.NET died out, because if you had to retrain that much, you figured you might as well go to C# or a instead, or indeed a completely different language entirely.

I briefly worked professionally on a VB.NET project, but outside that job I've never met anyone else who can say the same. I've met a few who went straight from VB6 to C# though.

Re: Nvidia's Risky Business

#127

Post by mrandish »

Dardalus wrote: post #57

Tend to agree with Ben's thesis RE Demis and DeepMind not really being focused on the agentic coding race. That being said, it remains to be seen whether Sergey and Koray can inspire the foot soldiers in the same way that Sama and Dario do. I'm not too optimistic, and that's to say nothing of the fact that Google cannot possibly hope to compete with these other companies on potential employee upside.

> Google cannot possibly hope to compete with these other companies...

It's possible Google has intentionally decided to take a more conservative blended approach than purely competing at the bleeding edge of the frontier. If so, they obviously have no incentive to state it publicly but the recent departures and financials are consistent with the idea. It also makes sense that a company so much bigger, longer-term and (somewhat) more diversified than pure-play frontier labs would play the game to align with their strengths (capital, balance sheet, breadth, etc).

In their position, why not take an 'arms supplier' strategy in the near-term while drafting behind the frontier labs as a fast follower in AI, essentially betting the AI race is more akin to the Indy 500 than a quarter-mile drag race. If they're wrong and it IS more like a drag race, Google is in a better position to absorb and adjust than a frontier lab, for whom current valuations and capex spending requirements nearly require this to be a relatively short, winner-take-all race.

Re: Nvidia's Risky Business

#128

Post by TheBicPen »

atom_arranger wrote: post #83
rcr-anti wrote: post #74

For awhile I've found two things hard to square, that the hardware and software making up current gen AI will bring us to a socioeconomic singularity, and the reality the thing they're mostly trying to emulate is a few pounds of meat and fat running on tens of watts equivalent. On one hand the current AIs are obviously super human in some tasks, get completely dunked on in others by far simpler organisms. My cat can…

Another interesting discrepancy is that people think current GPUs are maybe capable of running AGI but they still can barely manage photorealistic rendering of a single room in realtime, or simulate something like a shirt thrown into a pile of laundry. They can generate a video of it based on millions of existing videos, but not do a real simulation of light and physics in realtime.

An AGI doesn't need that level of detail to do most tasks effectively. To use the OP's example, a cat catching a bug out of the air does not need to run a fluid dynamics simulation of airflow over the bug's wings to be able to catch it. A cheap approximation of the flight path is sufficient. Perhaps some physical tasks will need that level of detail but many will not.

Re: Nvidia's Risky Business

#129

Post by mrandish »

hermitShell wrote: post #97
dzonga wrote: post #36

Nvidia has been playing a dangerous but profitable game since the Crypto boom. but now I think they probably have bitten more than they can chew. Apple already proved with their unified memory - that as long you have the capacity you can run capable models locally - thereby goes demand for inference if everyone is running some model locally. For training - Chinese models have proved that you don't need the latest & g…

It seems they had a head start but are now facing stiff competition on all fronts. Software moat, GPU's for gaming, and its distant cousin datacenter compute. They rightfully invested their insane profits into many ventures, and how many of those have turned around into profit? They are also a robotics AI company with Omniverse. They are also an AI company with Nemotron. They are also a bleeding edge network equipmen…

> They rightfully invested their insane profits into many ventures

This is the inherent downside of such a rapid rise to being the world's most valuable company AND still being considered a growth stock. At their massive scale, the number of new adjacent businesses that have both sufficient size and potential growth is limited.

> I hope they succeed in chewing even 50% of what they bit off.

Anything approaching that is vanishingly unlikely. They're being forced to play the game more like a VC. The question is if a few unicorn winners can offset dozens of losers. The challenge is that, unlike a VC, their bets are much more correlated around AI.

Re: Nvidia's Risky Business

#130

Post by HDBaseT »

bryanlarsen wrote: post #99
adrianN wrote: post #69

Earlier quoted context omitted.

When efficiency reaches the point where local models on consumer hardware are good enough, demand for cloud tokens could rapidly shrink.

Very few consumers are going to spend multiple thousands of dollars to save $10 per month. Companies absolutely will to save hundreds per month per employee, but that's not consumer hardware.

Many gamers already spend $1000+ on a GPU.

If you can integrate AI accelerators into consumer cards (you can), you can have local AI for "reasonably" cheap. This is Nvidia's long term goal if you listen to what Jensen has to say.

The limitation is entirely on memory right now. Just a few years ago we could of been strapping 80-100GB to cards for under $200 (BoM).

Post reply on HN