431–440 of 581 posts
The M1 Max from 2021 has better memory bandwidth. The M3 Max can be specced to 128GB. Nothing new here, apart from being able to use CUDA on a less power hungry system.
> Nothing new here, apart from being able to use CUDA on a less power hungry system.
CUDA has been running on ARM SOCs since the Tegra K1, 12 years ago. Nvidia is not new to ARM, nor is CUDA.
Earlier quoted context omitted.
How much of that difficulty comes from the chosen game engine? I assume the engine is the primary factor in how resources are allocated.
I think much of the difficulty is just that, for example, the 1.8 TB/s of an RTX 5090 is a lot of bandwidth for a game to use. That's over 50,000 4k textures per second at 32bpp.
Earlier quoted context omitted.
Every AMD APU since introduction of HSA did it, which is how AMD ended up doing SoCs for PS4, PS5, and Xbox
Ok, so which one of these contemporary or previous chipsets could compete with the M1 for inference? Perhaps I'm missing some major detail.
I don't think the M1 specifically focused on inference. Their goal was to replace Intel/AMD/Nvidia with their own chips, and since the previous Macs shipped discrete GPUs, they had to match or beat those so they don't ship something slower.
The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…
I'm honestly a little confused by what you mean here. Why would we want to maximize those things? Games are about consistent output under the frame deadline, not full saturation of the hardware.
Why would anyone try to saturate a 5090 with their game? The addressable market is tiny and you'd have to hope their full spec runs as well as or better than your test rig or they'll still not hit framerate.
Earlier quoted context omitted.
I'm not sure everyone uses the terms consistently, but the difference is that the old "shared" memory was reserving a section to act as VRAM under the control of the GPU, ignored by the OS. The CPU ran the same kind of code pretending there is a "bus transfer" between host memory and graphics memory. In unified memory, all the memory is host memory and data can go from program to GPU with zero copy movements. The add…
That's my understanding, or, maybe a better word would be "guess". The CPU telling the GPU: this is your memory now.
If it runs well with Linux, I’m sold. A Windows pc will never see the inside of my network.
Assuming all that stuff is upstreamed (and they aren't using oddball webcam/input devices etc) it should have much better support than Qualcomm.
Fingers crossed!
The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…
>[..] take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. I'm honestly a little confused by what you mean here. Why would we want to maximize those things? Games are about consistent output under the frame deadline, not full saturation of the hardware. Why would anyone try to saturate a 5090 with their game? The addressable market is tiny and you'd have to hope their full spec runs as…
Earlier quoted context omitted.
Yep, but Apple products don’t spend most of their time running huge models. They are running lots of little ones all the time, using hardware designed for that.
It seems that you're agreeing with what I wrote above. They ship a general-purpose stock system and tailor their compute offering towards that. Accelerating 'lots of little models' fits naturally into what they offer, in a way that a more compute-intensive design might not.
If there's an M5 Ultra it'll be interesting to see what they've optimized it for.
The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…
The trouble with this is that the different types of memory have different characteristics. Latency for ordinary system memory is actually better than it is for GDDR, because GDDR is optimized for bandwidth. RTX 5090 has 1.8TB/s of memory bandwidth with a 512-bit memory bus. The same bus width for DDR5-9600 would have better latency but only a third of the bandwidth.
CPU workloads are generally bounded by latency and GPU workloads are generally bounded by bandwidth, which is why they use two different types.
> Reduce overall memory cost, by letting system builders purchase a single type of memory in bulk instead of having to figure out GDDR vs DDR memory placement (important for SFF/portable machines)
The trouble with this is cost. In principle you could get the same 1.8TB/s of memory bandwidth as the RTX 5090 has, with the better latency of DDR5, by using DDR5 with a 1536-bit bus. This is indeed with multi-socket servers do, two sockets with 768-bit in memory channels per socket, but now check how much those system boards cost.
But the remaining alternatives are both worse. If you use GDDR for the unified memory then GDDR costs more than DDR and you're going to have significantly worse latency for the CPU. If you use DDR without a 3-4 times wider bus than the already-wide GPU then the GPU gets starved for bandwidth.
Earlier quoted context omitted.
To this day I do not get why Intel doesn't just offer massive memory options for their cards. Just charge what it costs to add the extra memory, no upcharge, and they will never be able to keep up with demand. Cheap VRAM is enough to justify a lot of open source investment into challenging CUDA.
Memory is just one part. AMD has had offerings competitive to NVIDIA for quite some time, but nobody uses AMD cards. The biggest advantage with NVIDIA is CUDA.
AMD is selling every MI card it makes, and the market wants more of them.