The issue with bigger LLM or even MMMs is that the bigger they are the more they are cramming and regurgitating the training data, and that opens up to lawsuits. Making NNs generalize the way humans do it is still a hard problem.
Is this indeed established ? Could you provide a link or three ?
Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
321–330 of 340 posts
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#322No, we need less AI, less robots, more (alive) humans.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#323Earlier quoted context omitted.
It changes alot. It is not legal to make a 'CUDA' driver for an AMD GPU as Nvidia own cuda. You can see there was a open implementation of this that AMD sponsored until they got threatened with a lawsuit by Nvidia
ZLUDA ate the dust not because they implemented CUDA but because they were misusing complied NVIDIA libraries. If it was a clean room implementation of the API NVIDIA wouldn’t care. Heck that’s exactly what AMD did with HIP. But what you cannot do is essentially intercept calls to and reverse engineer NVIDIA binaries in real time because you can’t be arsed to build your own.
> But what you cannot do is essentially intercept calls to and reverse engineer NVIDIA binaries in real time because you can’t be arsed to build your own.
And this is precisely what anti-trust ligitation would allow them to do.
Preventing someone from reverse-engineering a product with the sole intention of maintaining monopoly status may be seen as anti-competitive.
AMD makes really, really good CPUs now, but only after ligitation against Intel allowed them to keep up with evolving x86 standards.
It's not about being "arsed" to build your own, the problem is NVIDIA controls the ecosystem-wide standard. NVIDIA can add to CUDA at any point in time and launch a GPU at the same time, the ecosystem would be forced to buy it if they want to stay on the cutting edge, and AMD would never be able to compete or reverse engineer these new standards in time.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#324Earlier quoted context omitted.
I don't understand AMD in this. Isn't it insanity that they're not throwing all they've got at their software stack?
You have to remember that this only applies to cheap consumer GPUs, they tend to support their datacenter GPUs better. When you consider that Ryzen AI already eats the AI inference lunch, having better GPUs with better software only threatens to cannibalize their data center GPU offering. Given enough time nobody will care about using AMD GPUs for AI.
Yet we've heard about nobody doing training on AMD.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#325Earlier quoted context omitted.
I don't understand AMD in this. Isn't it insanity that they're not throwing all they've got at their software stack?
It's a political problem. Good software engineers are paid more than good hardware engineers, but AMD management is unwilling to pay up to bring on good software engineers because then they'd also need to pay their hardware engineers more, otherwise the hardware engineers would be unsatisfied. If you check NVidia salaries online you'll see NVidia pays significantly more than AMD for both hardware and software enginee…
This is quite possibly also why Boeing is having issues.
If they paid everyone $1M/year salaries maybe more people would consider going into aerospace engineering.
Right now though Boeing's starting salaries aren't that much higher than what an Uber driver in the bay area makes.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#326Earlier quoted context omitted.
Wasteful software development is easy and keeps momentum for development. As long as growth is king, quick and dirty will always beat well optimized and smaller systems. This is not a problem with AI only, but with every software we use. Only two groups try to optimize things and try to fit into smaller systems. Passionate programmers and people who is paid to do this (e.g.: phone manufacturers' software teams, etc.)…
Not sure it's fair to characterize modern LLMs as 'wasteful software development" or unoptimized. The implementations do quite an impressive level of optimization with what hardware is available. New theoretical methods w.r.t quantization represent most of our software optimization techniques and we're probably hitting the limit of that shortly with ternary or binary gates. To your point enthusiasts and developers wi…
LLMs are efficient for programs in their class but they are substantially more power intensive than less generalized software.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#327Earlier quoted context omitted.
I’ll be happy with this if we use it to design viable fusion power plants. And I’ll be severely disappointed if it’s mostly used for ad targeting.
You are about to be severely disappointed.
Why? More compute helps with fluid dynamics, particle physics, plasma control, and material science. Can't imagine it hurts.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#328Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#329They are claiming a 25x reduction in power consumption. That can't be right. Anyone understand where this number is coming from?
Comes from here [1]. Basically 100 racks of H vs 8 racks of B.
I think there may be a typo though, I assume this also includes liquid-cooled vs air-cooled.
[1] https://nvdam.widen.net/s/xqt56dflgh/nvidia-blackwell-archit...
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#330Earlier quoted context omitted.
what would your top suggestions be for server or desktop, instead of ubuntu? Arch (too unstable for server?), Silverblue?
Debian for server, Fedora or openSUSE Tumbleweed for desktop.
Thanks, I've started going that way too (back to Debian for server based)