Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple M3 Ultra

apple.com

Post reply on HN
Print view View original

661–670 of 1001 posts

Re: Apple M3 Ultra

#661

Post by stego-tech »

kllrnohj wrote: post #347
stego-tech wrote: post #248

Earlier quoted context omitted.

I cannot express how dirt cheap that pricepoint is for what's on offer, especially when you're comparing it to rackmount servers. By the time you've shoehorned in an nVidia GPU and all that RAM, you're easily looking at 5x that MSRP; sure, you get proper redundancy and extendable storage for that added cost, but now you also need redundant UPSes and have local storage to manage instead of centralized SANs or NASes. F…

> By the time you've shoehorned in an nVidia GPU and all that RAM, you're easily looking at 5x that MSRP That nvidia GPU setup will actually have the compute grunt to make use of the RAM, though, which this M3 Ultra probably realistically doesn't. After all, if the only thing that mattered was RAM then the 2TB you can shove into an Epyc or Xeon would already be dominating the AI industry. But they aren't, because it…

You're forgetting what Apple's been baking into their silicon for (nearly? over?) a decade: the Neural Processing Unit (NPU), now called the "Neural Engine". That's their secret sauce that makes their kit more competitive for endpoint and edge inference than standard x86 CPUs. It's why I can get similarly satisfying performance on my old M1 Pro Macbook Pro with a scant 16GB of memory as I can on my 10900k w/ 64GB RAM and an RTX 3090 under the hood. Just to put these two into context, I ran the latest version of LM Studio with the deepseek-r1-distill-llama-8b model @ Q8_0, both with the exact same prompt and maximally offloaded onto hardware acceleration and memory, with a context window that was entirely empty:

  Write me an AWS CloudFormation file that does the following:
  
  * Deploys an Amazon Kubernetes Cluster
  * Deploys Busybox in the namespace "Test1", including creating that Namespace
  * Deploys a second Busybox in the namespace "Test3", including creating that Namespace
  * Creates a PVC for 60GB of storage
The M1Pro laptop with 16GB of Unified Memory:

  * 21.28 seconds for "Thinking"
  * 0.22s to the first token
  * 18.65 tokens/second over 1484 tokens in its responses
  * 1m:23s from sending the input to completion of the output
The 10900k CPU, with 64GB of RAM and a full-fat RTX 3090 GPU in it:

  * 10.88 seconds for "thinking"
  * 0.04s to first token
  * 58.02 tokens/second over 1905 tokens in its responses
  * 0m:34s from sending the input to completion of the output
Same model, same loader, different architectures and resources. This is why a lot of the AI crowd are on Macs: their chip designs, especially the Neural Engine and GPUs, allow quite competent edge inference while sipping comparative thimbles of energy. It's why if I were all-in on LLMs or leveraged them for work more often (which I intend to, given how I'm currently selling my generalist expertise to potential employers), I'd be seriously eyeballing these little Mac Studios for their local inference capabilities.

Re: Apple M3 Ultra

#662

Post by LPisGood »

xp84 wrote: post #500
raydev wrote: post #488

Earlier quoted context omitted.

> hasn't there always been a tradeoff between "fastest single core" vs "lots of cores" (and thus best multicore)? Not in the Apple Silicon line. The M2 Ultra has the same single core performance as the M2 Max and Pro. No benchmarks for the M3 Ultra yet but I'm guessing the same vs M3 Max and Pro.

Okay, good to know. Interesting change then.

I think the traditional reason for this is that other chips like to use complex scheduling logic to have more logical cores than physical cores. This costs single threaded speed but allows you to run more threads faster.

Re: Apple M3 Ultra

#663

Post by martin_a »

> Apple today announced M3 Ultra, the highest-performing chip it has ever created

Well, duh, it would be a shame if you made a step backwards, wouldn't it? I hate that stupid phrase...

Re: Apple M3 Ultra

#664

Post by jonplackett »

GeekyBear wrote: post #655
InTheArena wrote: post #29

Whoa. M3 instead of M4. I wonder if this was basically binning, but I thought that I had read somewhere that the interposer that enabled this for the M1 chips where not available. That Said, 512GB of unified ram with access to the NPU is absolutely a game changer. My guess is that Apple developed this chip for their internal AI efforts, and are now at the point where they are releasing it publicly for others to use.…

I also wondered about binning, so I pulled together how heavily Apple's Max chips were binned in shipping configurations. M1 Max - 24 to 32 GPU cores M2 Max - 30 to 38 GPU cores M3 Max - 30 to 40 GPU cores M4 Max - 32 to 40 GPU cores I also looked up the announcement dates for the Max and the Ultra variant in each generation. M1 Max - October 18, 2021 M1 Ultra - March 8, 2022 M2 Max - January 17, 2023 M2 Ultra - June…

I’m missing the point. What is it you’re concluding from these dates?

Re: Apple M3 Ultra

#665

Post by perfmode »

kridsdale1 wrote: post #642
rjeli wrote: post #312

Wow, incredible. I told myself I’d stop waffling and just buy the next 800gb/s mini or studio to come out, so I guess I’m getting this. Not sure how much storage to get. I was floating the idea of getting less storage, and hooking it up to a TB5 NAS array of 2.5” SSDs, 10-20tb for models + datasets + my media library would be nice. Any recommendations for the best enclosure for that?

It depends on your bandwidth needs. I also want to build the thing you want. There are no multi SSD M2 TB5 bays. I made one that holds 4 drives (16TB) at TB3 and even there the underlying drives are far faster than the cable. My stuff is in OWC Express 4M2.

Are you running RAID?

Re: Apple M3 Ultra

#666

Post by matwood »

iambateman wrote: post #14

People who know more than me: they’re talking a lot about RAM and not much about GPU. Do you expect this will be able to handle AI workloads well? All I’ve heard for the past two years is how important a beefy GPU is. Curious if that holds true here too.

I was able to run and use the DeepSeek distilled 24gb on an M1 Max with 64gb of ram. It wasn't speedy, but it was usable. I imagine the M3/4s are much faster, especially on smaller, more specific models.

Re: Apple M3 Ultra

#667

Post by newsclues »

jsheard wrote: post #60
lauritz wrote: post #46

They update the Studio to M3 Ultra now, so M4 Ultra can presumably go directly into the Mac Pro at WWDC? Interesting timing. Maybe they'll change the form factor of the Mac Pro, too? Additionally, I would assume this is a very low-volume product, so it being on N3B isn't a dealbreaker. At the same time, these chips must be very expensive to make, so tying them with luxury-priced RAM makes some kind of sense.

> Maybe they'll change the form factor of the Mac Pro, too? Either that or kill the Mac Pro altogether, the current iteration is such a half-assed design and blatantly terrible value compared to the Studio that it feels like an end-of-the-road product just meant to tide PCIe users over until they can migrate everything to Thunderbolt. They recycled a design meant to accommodate multiple beefy GPUs even though GPUs ar…

The Mac Pro could exist as a PCIe expansion slot storage case that accepts a logic board from the frequently updated consumer models. Or multiple Mac Studio logic boards all in one case with your expansion cards all working together.

Re: Apple M3 Ultra

#668

Post by hu3 »

wewewedxfgdf wrote: post #660

Computers these days - the more appealing, exciting, cooler desirable, the higher the price, into the stratosphere. $9499 What ever happening to competition in computing? Computing hardware competition used to be cut throat, drop dead, knife fight, last man standing brutally competitive. Now it's just a massive gold rush cash grab.

It doesn't even run Linux properly.

Could cost half of that and it would still be uninteresting for my use cases.

For AI, on-demand cloud processing is magnitudes better in speed and software compatibility anyway.

Re: Apple M3 Ultra

#670

Post by niek_pas »

wewewedxfgdf wrote: post #660

Computers these days - the more appealing, exciting, cooler desirable, the higher the price, into the stratosphere. $9499 What ever happening to competition in computing? Computing hardware competition used to be cut throat, drop dead, knife fight, last man standing brutally competitive. Now it's just a massive gold rush cash grab.

The Macintosh plus, released in 1986, cost $2600 at the time, or $7460 adjusted for inflation.
Post reply on HN