721–730 of 1001 posts
Thunderbolt 5 (TB 5) is pretty handy, you can have a very thin and lightweight laptop, then can get access to external GPU or eGPU via TB 5 if needed [1]. Now you can have your cake (lightweight laptop) and eat it too (potent GPU). [1] Asus just announced the world’s first Thunderbolt 5 eGPU: https://www.theverge.com/24336135/asus-thunderbolt-5-externa...
Earlier quoted context omitted.
Some possible groups of reasons: 1. Until recently RAM amount was something the end user liked to configure, so little market demand. 2. Technically, building such a large system on a chip or collection of chiplets was not possible. 3. RAM speed wasn't a bottleneck for most tasks, it was IO or CPU. LLMs changed this.
M1 came out before the LLM rush, though
Earlier quoted context omitted.
TB 5 seems like the sort of thing you could 'slap on' to a beefy enough chip. Or the sort of thing you put onto a successor when you had your fingers crossed that the spec and hardware would finalize in time for your product launch but the fucking committee went into paralysis again at the last moment and now your product has to ship 4 months before you can put TB 5 hardware on shelves. So you put your TB4 circuitry…
Sounds like you’ve seen some things.
Earlier quoted context omitted.
Given that the M1 Ultra and M2 Ultra also exist, I'd expect either straight binning, or two designs that use mostly the same designs for the cores but more of them and a few extra features. I love Apple but they love to speak in half truths in product launches. Are they saying the M3 Ultra is their first Thunderbolt 5 computer? I don't recall seeing any previous announcements.
M4 Pro MacBook and Mini have TB5.
(NB: I've been long on AAPL since $7 a share but I'm also allergic to bullshit)
Earlier quoted context omitted.
Yep, it's apples to oranges. But sometimes you want apples, and sometimes you want oranges, so it's all good! There's a wide spectrum of potential requirements between memory capacity, memory bandwidth, compute speed, compute complexity, and compute parallelism. In the past, a few GB was adequate for tasks that we assigned to the GPU, you had enough storage bandwidth to load the relevant scene into memory and generat…
Sure, if you want to do training get an NVIDIA card. My point is that it's not worth comparing either Mac or CPU x86 setup to anything with NVIDIA in it. For inference setups, my point is that instead of paying $10000-$15000 for this Mac you could build an x86 system for The "+$4000" for 512GB on the Apple configurator would be "+$1000" outside the Apple world.
Earlier quoted context omitted.
I assume even this one won't run on an RTX 5090 due to constrained memory size: https://news.ycombinator.com/item?id=43270843
sure on consumer GPUs but that is not what is constraining the model inference in most actual industry setups. technically even then, you are CPU-GPU memory bandwidth bound more than just GPU memory, although that is maybe splitting hairs
Earlier quoted context omitted.
"unified memory" funny that people think this is so new, when CRAY had Global Heap eons ago...
The real hardware needed for artificial intelligence wasn't NVIDIA, it was a CRAY XMP from 1982 all along
we walked in and getting our bearings, we come upon CRAY office. WTF?!
I tried the doors, locked - and it was clearly empty... but damn did I want to steal their office door signage.
Thunderbolt 5 (TB 5) is pretty handy, you can have a very thin and lightweight laptop, then can get access to external GPU or eGPU via TB 5 if needed [1]. Now you can have your cake (lightweight laptop) and eat it too (potent GPU). [1] Asus just announced the world’s first Thunderbolt 5 eGPU: https://www.theverge.com/24336135/asus-thunderbolt-5-externa...
Except that you're stuck with macOS, so there aren't any drivers for NVIDIA, AMD or Intel GPUs.
Earlier quoted context omitted.
M1 came out before the LLM rush, though
Apple has always liked to integrate as much as possible on the same chip. It was only natural that they would come to this conclusion, with the improved perf the cherry on top.
Let's say you want to have the absolute max memory(512GB) to run AI models and let's say that you are O.K. with plugging a drive to archive your model weights then you can get this for a little bit shy of $10K. What a dream machine. Compared to Nvidia's Project DIGITS which is supposed to cost $3K and be available "soon", you can get a specs matching 128GB & 4TB version of this Mac for about $4700 and the difference…
The full deepseek R1 model needs more memory than 512GB. The model is 720GB alone. You can run a quantized version on it, but not the full model.