Earlier quoted context omitted.
I don't understand AMD in this. Isn't it insanity that they're not throwing all they've got at their software stack?
Hardware people don't get along very well with software people.
Why's that?
Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
211–220 of 340 posts
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#212Earlier quoted context omitted.
The fundamental 'unit' of NN computation is not an individual vector element but rather an entire vector. One of the first results you often learn about in linear algebra is that some axes are more important than others (principal components, singular value decomposition). Thus, it totally stands to reason that the underlying field of the vector is inconsequential but rather the entire vector machinery. All you have…
I see, so the idea is that enough of the quantization errors are sort of averaged out across the dimensions of the vector space to still be useful?
The way I think about it is finally it will end in a binary feature vector similar to 20Questions (male or female, alive or dead ...) just with 100s of dimensions
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#213Earlier quoted context omitted.
They are priced as if they are the only ones who are capable of creating chips that can crunch LLM algos. But AMD, Google, Intel, and even Apple are also capable. Apple is in talks with Google to bring Gemini to the iPhone, and it will obviously also be on android phones. So almost every phone on earth is poised to be using Gemini in the near future, and Gemini runs entirely on Google's own custom hardware (which is…
This seems as good a place as any to be Corrected by the Internet, so... correct me if I'm wrong. Making a graphics chip that is as good as Nvidia: Very difficult. Huge moat, huge effort, lots of barriers, lots of APIs, lot of experience, lots of decades of experience to overcome. Making something that can run a NN: Much, much easier. I'd guess, start-up level feasible. The math is much simpler. There's a lot of it,…
Anecdata... one of the folks sitting in front of me at a session at GTC claimed the be an AMD employee who also claimed to previously work on cuda. He seemed skeptical that AMD would pull this off. This is the sort of fun stuff that you hear at a conference and aren't sure how much of it is just technical bragging/oneupmanship.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#214Double digit peta flop mass produced. "The computing power needed to replicate the human brain’s relevant activities has been estimated by various authors, with answers ranging from 10^12 to 10^28 FLOPS." Petaflop is 10^15 Crazy times.
I’ll be happy with this if we use it to design viable fusion power plants. And I’ll be severely disappointed if it’s mostly used for ad targeting.
You are about to be severely disappointed.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#215Earlier quoted context omitted.
> My prediction is eventually there will be anti-trust ligitation, they will be required to open the CUDA standard, after which AMD will become a competitor. If AMD isn't a competitor before government intervention, I don't the government forcing nvidia to open up CUDA changes much. CUDA's moat isn't due to some secret sauce - nvidia put in the developer hours; and if AMDs CUDA implementation is still broken, people…
I don't understand AMD in this. Isn't it insanity that they're not throwing all they've got at their software stack?
You know what happens to companies that panic and throw all their resources into knee-jerk software projects? I don't, but I'd predict it is ugly. Adding more people to a bad project generally makes it worse.
The issue that AMD has is they had a long period where they clearly had no idea what they were doing. You could tell just from looking at websites, CUDA pretty much immediately gets to "here is a library for FFT", "here is a library for sparse matricies". AMD would explain that ROCM is an abbreviation of the ROCm Software platform or something unspeakably stupid. And that your graphics card wasn't supported.
That changed a few months ago; so it looks like they have put some competent PMs in the chair now or something. But it'll take months for the flow on effects to reach the market. They have to figure out what the problems are which takes months to do properly; then fix the software (1-3 months more minimum); then get it into the open and the foundational libraries like PyTorch pick it up (might take another year). You can speed that up, but more cooks in the kitchen is not the way. Bandwidth use needs to be optimised.
It isn't like ROCm seems lacks key features; it can technically do inference and training. My card crashes regularly though (might be a VRAM issue) so it is useless in practice. AMD can check boxes but the software doesn't really work and grappling with that organisationally is hard. Unless you have the right people in the right places, which AMD didn't have up to at least mid 2023.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#216My take from being at the keynote and the content I've seen so far at the conference is that Nvidia's is moving up the stack (like all good hardware vendors are prone to do). Obviously they are going to keep doing bigger. But the takeaway for me is that they are building "docker for llms" - NIM. They are building a container system where you can download/buy(?) NIMs and easily deploy them on their hardware. Going to…
[deleted]
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#217Earlier quoted context omitted.
I'd prefer they stick to GPUs, but I think you're over simplifying. Dell proves that selling complete units is very profitable. Apple shows that owning the entire stack is immensely profitable. Nvidia already has significant hardware and software investment. They very well could fully integrate and grab larger slices of the pie. In fact, Nvidia already has complete appliance like fully integrated machines. But enterp…
>> Apple shows that owning the entire stack is immensely profitable. Apple shows no such thing. Apple, sells pretty, reliable and safe. A car is a car, but apple is a sports car, or a saloon. Vertical integration is the way they chose to deliver that, and pretty and reliable are all normal people care about. Nvidia is gonna have to think long and hard about the "whole stack". 20 years ago they might have been able to…
Nvidia is moving up the stack. They announced NIMs today. I liken it to Docker for AI.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#218Earlier quoted context omitted.
no it means rent seeking. imagine aws if they also sold all computers in the world, now you can only rent from them
"For only 100$ a month, you'll be able to turn on the gpu you already paid for" --Nvidia, pretty soon
This is sort of already a reality. Their vGPU functionality (partitioning a single physical GPU into multiple virtual GPUs) is already separately licensed - https://www.nvidia.com/en-us/data-center/buy-grid/
And that's once you've bought an expensive Tesla/Quadro GPU too.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#219My take from being at the keynote and the content I've seen so far at the conference is that Nvidia's is moving up the stack (like all good hardware vendors are prone to do). Obviously they are going to keep doing bigger. But the takeaway for me is that they are building "docker for llms" - NIM. They are building a container system where you can download/buy(?) NIMs and easily deploy them on their hardware. Going to…
I'm not that abreast of all the developments in the AI space.
What specific class of AI startups do you have in mind here? AI-aaS startups who provide the "infra"?
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#220My take from being at the keynote and the content I've seen so far at the conference is that Nvidia's is moving up the stack (like all good hardware vendors are prone to do). Obviously they are going to keep doing bigger. But the takeaway for me is that they are building "docker for llms" - NIM. They are building a container system where you can download/buy(?) NIMs and easily deploy them on their hardware. Going to…
Won't do anything to most consumer facing AI, the UI & convenience is already a major selling point. A bigger threat is that the feature the business is built around makes it into mainline software... there is no demand for (paid) background removal anymore as every iPhone can do it nowadays.
Generally if whatever AI product you have can easily just be a feature in whatever application businesses already use, then you are running a business on borrowed time.