Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple M3 Ultra

apple.com

Post reply on HN
Print view View original

241–250 of 1001 posts

Re: Apple M3 Ultra

#241

Post by pavlov »

bigyabai wrote: post #233
cxie wrote: post #222

512GB of unified memory is truly breaking new ground. I was wondering when Apple would overcome memory constraints, and now we're seeing a half-terabyte level of unified memory. This is incredibly practical for running large AI models locally ("600 billion parameters"), and Apple's approach of integrating this much efficient memory on a single chip is fascinating compared to NVIDIA's solutions. I'm curious about how…

For enterprise markets, this is table stakes. A lot of datacenter customers will probably ignore this release altogether since there isn't a high-bandwidth option for systems interconnect.

The Mac Studio isn’t meant for data centers anyway? It’s a small and silent desktop form factor — in every respect the opposite of a design you’d want to put in a rack.

A long time ago Apple had a rackmount server called Xserve, but there’s no sign that they’re interested in updating that for the AI age.

Re: Apple M3 Ultra

#242

Post by ewzimm »

kokada wrote: post #188
InTheArena wrote: post #29

Whoa. M3 instead of M4. I wonder if this was basically binning, but I thought that I had read somewhere that the interposer that enabled this for the M1 chips where not available. That Said, 512GB of unified ram with access to the NPU is absolutely a game changer. My guess is that Apple developed this chip for their internal AI efforts, and are now at the point where they are releasing it publicly for others to use.…

> This hardware is really being held back by the operating system at this point. Apple could either create a 2U rack hardware and support Linux (and I mean Apple supporting it, not hobbysts), or have a build of Darwin headless that could run on that hardware. But in the later case, we probably wouldn't have much software available (though I am sure people would eventually starting porting software to it, there is alr…

There has to be someone at Apple with a contact at IBM that could make Fedora Apple Remix happen. It may not be on-brand, but this is a prime opportunity to make the competition look worse. File it under Community projects at https://opensource.apple.com/projects

Re: Apple M3 Ultra

#243

Post by bigyabai »

dagmx wrote: post #210
bigyabai wrote: post #176

Earlier quoted context omitted.

> After all, the 5090 alone is multiple times the wattage of this SoC. FWIW, normalizing the wattages (or even underclocking the GPU) will still give you an Nvidia advantage most days. Apple's GPU designs are closer to AMD's designs than Nvidia's, which means they omit a lot of AI accelerators to focus on a less-LLM-relevent raster performance figure. Yes, the GPU is faster than the NPU. But Apple's GPU designs haven…

M2 Ultra is ~250W (averaging various reports since Apple don’t publish) for the entire SoC. 5090 is 575W without the CPU. You’d have to cut the Nvidia to a quarter and then find a comparable CPU to normalize the wattage for an actual comparison. I agree that Apple GPUs aren’t putting the dedicated GPU companies in danger on the benchmarks, but they’re also not really targeting it? They’re in completely different zone…

Well, select your hardware of choice and see for yourself then: https://browser.geekbench.com/opencl-benchmarks

> but they’re also not really targeting it?

That's fine, but it's not an excuse to ignore the power/performance ratio.

Re: Apple M3 Ultra

#244

Post by jmyeet »

InTheArena wrote: post #29

Whoa. M3 instead of M4. I wonder if this was basically binning, but I thought that I had read somewhere that the interposer that enabled this for the M1 chips where not available. That Said, 512GB of unified ram with access to the NPU is absolutely a game changer. My guess is that Apple developed this chip for their internal AI efforts, and are now at the point where they are releasing it publicly for others to use.…

I've been looking at the potential for Apple to make really interesting LLM hardware. Their unified memory model could be a real game-changer because NVidia really forces market segmentation by limiting memory.

It's worth adding the M3 Ultra has 819GB/s memory bandwidth [1]. For comparison the RTX 5090 is 1800GB/s [2]. That's still less but the M4 Mac Minis have 120-300GB/s and this will limit token throughput so 819GB/s is a vast improvement.

For $9500 you can buy a M3 Ultra Mac Studio with 512GB of unified memory. I think that has massive potential.

[1]: https://www.apple.com/mac-studio/specs/

[2]: https://www.nvidia.com/en-us/geforce/graphics-cards/50-serie...

Re: Apple M3 Ultra

#245

Post by PaulHoule »

bigyabai wrote: post #233
cxie wrote: post #222

512GB of unified memory is truly breaking new ground. I was wondering when Apple would overcome memory constraints, and now we're seeing a half-terabyte level of unified memory. This is incredibly practical for running large AI models locally ("600 billion parameters"), and Apple's approach of integrating this much efficient memory on a single chip is fascinating compared to NVIDIA's solutions. I'm curious about how…

For enterprise markets, this is table stakes. A lot of datacenter customers will probably ignore this release altogether since there isn't a high-bandwidth option for systems interconnect.

That article says you can connect them through the Thunderbolt 5 somehow to form clusters.

Re: Apple M3 Ultra

#246

Post by bigyabai »

pavlov wrote: post #241
bigyabai wrote: post #233

Earlier quoted context omitted.

For enterprise markets, this is table stakes. A lot of datacenter customers will probably ignore this release altogether since there isn't a high-bandwidth option for systems interconnect.

The Mac Studio isn’t meant for data centers anyway? It’s a small and silent desktop form factor — in every respect the opposite of a design you’d want to put in a rack. A long time ago Apple had a rackmount server called Xserve, but there’s no sign that they’re interested in updating that for the AI age.

It's the Ultra chip, the same one that goes into the rackmount Mac Pro. I don't think there's much confusion as to who this is for.

> there’s no sign that they’re interested in updating that for the AI age.

https://security.apple.com/blog/private-cloud-compute/

Re: Apple M3 Ultra

#247

Post by zamadatix »

api wrote: post #202

Half a terabyte could run 8 bit quantized versions of some of those full size llama and deepseek models. Looking forward to seeing some benchmarks on that.

Deepseek would need Q5ish level quantization to fit.

Re: Apple M3 Ultra

#248

Post by stego-tech »

AlchemistCamp wrote: post #197
InTheArena wrote: post #29

Whoa. M3 instead of M4. I wonder if this was basically binning, but I thought that I had read somewhere that the interposer that enabled this for the M1 chips where not available. That Said, 512GB of unified ram with access to the NPU is absolutely a game changer. My guess is that Apple developed this chip for their internal AI efforts, and are now at the point where they are releasing it publicly for others to use.…

Keep in mind the minimum configuration that has 512GB of unified RAM is $9,499.

I cannot express how dirt cheap that pricepoint is for what's on offer, especially when you're comparing it to rackmount servers. By the time you've shoehorned in an nVidia GPU and all that RAM, you're easily looking at 5x that MSRP; sure, you get proper redundancy and extendable storage for that added cost, but now you also need redundant UPSes and have local storage to manage instead of centralized SANs or NASes.

For SMBs or Edge deployments where redundancy isn't as critical or budgets aren't as large, this is an incredibly compelling offering...if Apple actually had a competent server OS to layer on top of that hardware, which it does not.

If they did, though...whew, I'd be quaking in my boots if I were the usual Enterprise hardware vendors. That's a damn frightening piece of competition.

Re: Apple M3 Ultra

#249

Post by vr46 »

nsteel wrote: post #223
AlchemistCamp wrote: post #197

Earlier quoted context omitted.

Keep in mind the minimum configuration that has 512GB of unified RAM is $9,499.

And how is it only £9,699.00!! Does that dollar price include sales tax or are Brits finally getting a bargain?

The US prices never include state sales tax IIRC. Maybe we're finally getting some parity.

Re: Apple M3 Ultra

#250

Post by p_ing »

flats wrote: post #145
InTheArena wrote: post #92

Earlier quoted context omitted.

No native docker support, no headless management options (enterprise strength), Limited QoS management, lack of robust python support (out of the box), interactive user focused security model.

I feel you on a lot of this! But out of the box Python support? Does anybody actually want that? It’s pretty darn quick & straightforward to get a Python environment up & running on MacOS. Maybe I’m misunderstanding what you mean here.

No one would want OOTB Python support. You'd be stuck on a version you didn't want to use.
Post reply on HN