Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia's Risky Business

stratechery.com

Post reply on HN
Print view View original

161–170 of 185 posts

Re: Nvidia's Risky Business

#161

Post by galaxyLogic »

bri3d wrote: post #152
YuechenLi wrote: post #68

Nvidia's biggest advantage in AI has never been only their hardware performance but how entrenched their software is in ML research that flowed down stream. However, if you've actually used CUDA C/C++, it's pretty one of the worst software development ecosystem imaginable: you get all the footgun of regular C++, plus GPU compute pretending to be C++ and but doesn't actually behave like C++ because CPU and GPU compute…

The CUDA runtime coming with a gazillion reasonably decent kernels (DNN, BLAS, CUTLASS) and a concurrency system (NCCL) is a big deal; especially in the “early days” very few researchers or development runtimes were even writing their own kernels or dealing with CUDA C++ extensively, they were wrapping the ones NVidia gave them. I do agree that it’s really not great, and I also have never been a strong believer in th…

What I don't quite get is why can't they use AI to translate CUDA programs into more open architectures like AMD ROCm?

AI is supposed have solved the "coding problem". But shouldn't translating a program from one platform to another be an even easier, more mechanical, task for the AI?

Re: Nvidia's Risky Business

#162

Post by tomhow »

dh2022 wrote: post #159
tomhow wrote: post #145

Earlier quoted context omitted.

Please don't post grouchy comments like this on HN. The guidelines make it clear we're trying for something better here. https://news.ycombinator.com/newsguidelines.html

I did not mean to be snarky. But still, the proposition that a railroad bankruptcy “led to world war” (direct quote from the article) 40 years later is ridiculous.

We just don’t want denunciatory rhetoric like “is where I stopped reading” or “this guy should re-read what he puts out” or “ridiculous”. If someone is wrong just point out where they’re wrong. Educate us, don’t fulminate.

Re: Nvidia's Risky Business

#163

Post by galaxyLogic »

0xDEAFBEAD wrote: post #156
elictronic wrote: post #103

Earlier quoted context omitted.

Retraining a large high paid user base is often a non-starter. To put this in perspective, Boeing’s eventual retraining costs for all the pilots for the 737Max was around 5 billion dollars. Looking at software more specifically the Linux foundation reported based on software dev salaries in 2008 it would be 1.4 billion to only write the Linux kernel. Up until about 2023 there wasn’t enough money involved to have any…

As agentic coding continues to improve, won't it get easier for devs to retrain for new languages/frameworks/etc.?

And couldn't we just ask AI to translate our program in one language/framework into another?

I was under the impression that AI was supposed to remove software-moats, let us all ask it to write our custom MS Word for us for instance?

Re: Nvidia's Risky Business

#164

Post by galaxyLogic »

mandeepj wrote: post #110
elictronic wrote: post #103

Earlier quoted context omitted.

Retraining a large high paid user base is often a non-starter. To put this in perspective, Boeing’s eventual retraining costs for all the pilots for the 737Max was around 5 billion dollars. Looking at software more specifically the Linux foundation reported based on software dev salaries in 2008 it would be 1.4 billion to only write the Linux kernel. Up until about 2023 there wasn’t enough money involved to have any…

Yet, Microsoft pulled it off with a new .NET Framework, and Apple with its new iOS SDK. There are many more examples besides those two.

Anthropic rewrote Bun in Rust, with much help from AI of course

Re: Nvidia's Risky Business

#165

Post by galaxyLogic »

hylaride wrote: post #75
onlyrealcuzzo wrote: post #21

Earlier quoted context omitted.

What makes this insanely hard to predict is that the compute needed for the same quality output has roughly gone down 90% every 18 months for ~5 years. 1) We don't know how long that trend will continue, but you do know where to look for when it may end (if smaller sized models continue to compress the knowledge effectively of larger models). 2) We don't know when the appetite for higher cost models might go down and…

It's also hard to predict how much money will be burned going down wrong avenues. The internet was the future, but it took a lot of failed companies to eventually land on a sustainable model that brought us the giants we have today. Railways were also the future, but that didn't stop a rush to build out (often subsidized) lines that were ultimately uneconomical (either because they were corrupt or the planned settlem…

I rhink there's a difference with AI because -- it brings true value because you pay for the tokens, you only pay for what you use. That is true value.

Compare to just paying for an internet connection, you have bandwidth but not sure what you can do with it that is valuable.

Let's say you use AI to produce software. There;s no limit as to how high the quality you want your software to have. And how fast you want your project to be complete. There's plenty of room for higher quality, and more performant AI. As AI becomes chepaer people will use more of it, they're not going to say "We have enough AI".

Compare to railroads. Yes you pay for the distance travelled but there's a limit to how much people wwill want to travel, how it will benefit them.

Re: Nvidia's Risky Business

#166

Post by Someone »

YuechenLi wrote: post #68

Nvidia's biggest advantage in AI has never been only their hardware performance but how entrenched their software is in ML research that flowed down stream. However, if you've actually used CUDA C/C++, it's pretty one of the worst software development ecosystem imaginable: you get all the footgun of regular C++, plus GPU compute pretending to be C++ and but doesn't actually behave like C++ because CPU and GPU compute…

> Google's limitation is that they still don't offer TPUs in a PCI-E card/dev board that people can plug in to their PC for local development

I’m not familiar with this field, but to my brain, https://www.amazon.com/s?k=Google+Coral seem to show me several such options.

Re: Nvidia's Risky Business

#167

Post by spwa4 »

Someone wrote: post #166
YuechenLi wrote: post #68

Nvidia's biggest advantage in AI has never been only their hardware performance but how entrenched their software is in ML research that flowed down stream. However, if you've actually used CUDA C/C++, it's pretty one of the worst software development ecosystem imaginable: you get all the footgun of regular C++, plus GPU compute pretending to be C++ and but doesn't actually behave like C++ because CPU and GPU compute…

> Google's limitation is that they still don't offer TPUs in a PCI-E card/dev board that people can plug in to their PC for local development I’m not familiar with this field, but to my brain, https://www.amazon.com/s?k=Google+Coral seem to show me several such options.

... with performance greater or comparable to even the lowest performance Nvidia card?

Re: Nvidia's Risky Business

#168

Post by radicalbyte »

eterm wrote: post #126
mandeepj wrote: post #110

Earlier quoted context omitted.

Yet, Microsoft pulled it off with a new .NET Framework, and Apple with its new iOS SDK. There are many more examples besides those two.

Microsoft pulled it off with dotnet, sort of, because they approached it like a completely new language, sold people on the benefits of it. And the people they were selling that to, ( It's free and open souce now! ), were a very different group to the market they left behind on .NET Framework, who are often still struggling to make the transition now. Had they actually killed off .NET Framework, it would have been a…

The whole push behind .Net Core was to get .Net running on Linux well and natively and was being lead largely by those of us using .Net for web. We were being murdered by other languages at the time; mono was an option but Microsoft shops usually needed to target something blessed/backed by MS.

Largely the same market (Enterprise) but not the different segment (web as opposed to Windows/WinForms).

I ported about 15 years of projects from various versions of .Net to .Net Core whilst they were developing (and sent feedback to the team - they were asking us to do that) and the process was pretty reasonable. You were only really stuck if you were using something very very Windows specific (certain image processing libraries iirc) and even then it was largely manageable.

The old full-fat framework is, AFAIK, still supported, as there's a whole lot of legacy code which is Windows specific which is still expensive / hard to port over.

Re: Nvidia's Risky Business

#169

Post by mdp2021 »

galaxyLogic wrote: post #161
bri3d wrote: post #152

Earlier quoted context omitted.

The CUDA runtime coming with a gazillion reasonably decent kernels (DNN, BLAS, CUTLASS) and a concurrency system (NCCL) is a big deal; especially in the “early days” very few researchers or development runtimes were even writing their own kernels or dealing with CUDA C++ extensively, they were wrapping the ones NVidia gave them. I do agree that it’s really not great, and I also have never been a strong believer in th…

What I don't quite get is why can't they use AI to translate CUDA programs into more open architectures like AMD ROCm? AI is supposed have solved the "coding problem". But shouldn't translating a program from one platform to another be an even easier, more mechanical, task for the AI?

> AI is supposed have solved the

Which AI? LLMs are coding facilitators and code producers.

A problem is solved when the solution is reliable. Non-deterministic Neural Networks are not reliable. In fact,

> more mechanical[] task

that suggests an expectation of process and procedure, which is still not a capability of current architectures.

Sure, you can ask a brains-deficient operator to perform a huge task, but then you'll have to check the whole product, and that remains not cheap.

Re: Nvidia's Risky Business

#170

Post by eterm »

radicalbyte wrote: post #168
eterm wrote: post #126

Earlier quoted context omitted.

Microsoft pulled it off with dotnet, sort of, because they approached it like a completely new language, sold people on the benefits of it. And the people they were selling that to, ( It's free and open souce now! ), were a very different group to the market they left behind on .NET Framework, who are often still struggling to make the transition now. Had they actually killed off .NET Framework, it would have been a…

The whole push behind .Net Core was to get .Net running on Linux well and natively and was being lead largely by those of us using .Net for web. We were being murdered by other languages at the time; mono was an option but Microsoft shops usually needed to target something blessed/backed by MS. Largely the same market (Enterprise) but not the different segment (web as opposed to Windows/WinForms). I ported about 15 y…

Unfortunately "something windows specific" was pretty broad.

Between MSMQ, WCF over named pipes, MSDTC, and MSI installers, there's a lot to replace that is hard to provide the same guarantees or performance with straight replacements, if they even exist.

The end goal, being on modern dotnet, is better, but it's difficult to get there with a phased approach without accepting a temporary worsening, which is often hard to sell.

Especially while Framework is still supported.

Post reply on HN