Apple M3 Ultra • Hacker News Board

Re: Apple M3 Ultra

#621

Post by kridsdale1 » Wed, Mar 05, 2025, 7:53 PM UTC

hinkley wrote: post #578

InTheArena wrote: post #29

Whoa. M3 instead of M4. I wonder if this was basically binning, but I thought that I had read somewhere that the interposer that enabled this for the M1 chips where not available. That Said, 512GB of unified ram with access to the NPU is absolutely a game changer. My guess is that Apple developed this chip for their internal AI efforts, and are now at the point where they are releasing it publicly for others to use.…

Given that the M1 Ultra and M2 Ultra also exist, I'd expect either straight binning, or two designs that use mostly the same designs for the cores but more of them and a few extra features. I love Apple but they love to speak in half truths in product launches. Are they saying the M3 Ultra is their first Thunderbolt 5 computer? I don't recall seeing any previous announcements.

M4 Pro MacBook and Mini have TB5.

Re: Apple M3 Ultra

#622

Post by samstave » Wed, Mar 05, 2025, 7:53 PM UTC

cxie wrote: post #222

512GB of unified memory is truly breaking new ground. I was wondering when Apple would overcome memory constraints, and now we're seeing a half-terabyte level of unified memory. This is incredibly practical for running large AI models locally ("600 billion parameters"), and Apple's approach of integrating this much efficient memory on a single chip is fascinating compared to NVIDIA's solutions. I'm curious about how…

"unified memory"

funny that people think this is so new, when CRAY had Global Heap eons ago...

Re: Apple M3 Ultra

#623

Post by kridsdale1 » Wed, Mar 05, 2025, 7:55 PM UTC

stego-tech wrote: post #389

cubefox wrote: post #313

Earlier quoted context omitted.

I assume there is a very good reason why AMD and Intel aren't releasing a similar product.

From my outsider perspective, it's pretty straightforward why they don't. In Intel's case, there's ample coverage of the company's lack of direction and complacency on existing hardware, even as their competitors ate away at their moat, year after year. AMD with their EPYC chips taking datacenter share, Apple moving to in-house silicon for their entire product line, Qualcomm and Microsoft partnering with ongoing expl…

Well said.

Re: Apple M3 Ultra

#624

Post by _zoltan_ » Wed, Mar 05, 2025, 7:55 PM UTC

qwertox wrote: post #478

_zoltan_ wrote: post #455

Earlier quoted context omitted.

which professional card has 96GB of VRAM?

Like the NVIDIA H100 NVL 94GB HBM3 PCIe 5.0 Data Center GPU for 27.651,20 € https://www.primeline-solutions.com/de/nvidia-h100-nvl-94gb-...

that's not a professional line card, but a data center card.

Re: Apple M3 Ultra

#625

Post by alwillis » Wed, Mar 05, 2025, 7:57 PM UTC

Thaxll wrote: post #598

exabrial wrote: post #203

Earlier quoted context omitted.

If Apple supported Linux (headless) natively, and we could rack m4 pros, I absolutely would use them in our Colo. The CPUs have zero competition in terms of speed, memory bandwidth. Still blown away no other company has been able to produce Arm server chips that can compete.

Apple does not make server CPUs, they make consumer low W CPUs, it's very different. FYI Apple runs Linux in their DC, so no Apple hardware in their own servers.

> Apple does not make server CPUs, they make consumer low W CPUs, it's very different.

This is silly. Given the performance per watt, the M series would be great in a data center. As you all know, electricity for running the servers and cooling for the servers are the two biggest ongoing costs for a data center; the M series requires less power and runs more efficiently than the average Intel or AMD-based server.

> FYI Apple runs Linux in their DC, so no Apple hardware in their own servers.

That's certainly no longer the case. Apple announced their Private Cloud Compute [1] initiative—Apple designed servers running Apple Silicon to support Apple Intelligence functions that can't run on-device.

BTW, Apple just announced a $500 billion investment [2] in US-based manufacturing, including a 250,000 square foot facility to make servers. Yes, these will obviously be for their Private Cloud Compute servers… but it doesn't have to be only for that purpose.

From the press release:

As part of its new U.S. investments, Apple will work with manufacturing partners to begin production of servers in Houston later this year. A 250,000-square-foot server manufacturing facility, slated to open in 2026, will create thousands of jobs.

Previously manufactured outside the U.S., the servers that will soon be assembled in Houston play a key role in powering Apple Intelligence, and are the foundation of Private Cloud Compute, which combines powerful AI processing with the most advanced security architecture ever deployed at scale for AI cloud computing. The servers bring together years of R&D by Apple engineers, and deliver the industry-leading security and performance of Apple silicon to the data center.

Teams at Apple designed the servers to be incredibly energy efficient, reducing the energy demands of Apple data centers — which already run on 100 percent renewable energy. As Apple brings Apple Intelligence to customers across the U.S., it also plans to continue expanding data center capacity in North Carolina, Iowa, Oregon, Arizona, and Nevada.

[1]: https://security.apple.com/blog/private-cloud-compute/

[2]: https://www.apple.com/newsroom/2025/02/apple-will-spend-more...

Re: Apple M3 Ultra

#626

Post by justincormack » Wed, Mar 05, 2025, 7:58 PM UTC

dragonwriter wrote: post #541

valine wrote: post #535

Earlier quoted context omitted.

That will only have 128GB of unified memory

128GB for 3K; per the announcement their ConnectX networking allows two Project Digits devices to be plugged into eachother and work together as one device giving you 256GB for $6k, and, AFAIK, existing frameworks can split models across devices, as well, hence, presumably, the upthread suggestion that Project Digits would provide 512GB for $12k, though arguably the last step is cheating.

the reason Nvidia only talk about two machines over the network is I think they only have one network port, so you need to add costs for a switch.

Re: Apple M3 Ultra

#627

Post by hajile » Wed, Mar 05, 2025, 7:58 PM UTC

InTheArena wrote: post #29

Whoa. M3 instead of M4. I wonder if this was basically binning, but I thought that I had read somewhere that the interposer that enabled this for the M1 chips where not available. That Said, 512GB of unified ram with access to the NPU is absolutely a game changer. My guess is that Apple developed this chip for their internal AI efforts, and are now at the point where they are releasing it publicly for others to use.…

One of the leakers who got this Mac Studio right claims Apple is reserving the M4 ultra for the Mac Pro to differentiate the products a bit more.

Re: Apple M3 Ultra

#628

Post by behnamoh » Wed, Mar 05, 2025, 7:59 PM UTC

gatienboquet wrote: post #111

behnamoh wrote: post #83

Earlier quoted context omitted.

> My guess is that Apple developed this chip for their internal AI efforts what internal AI efforts? Apple Intelligence is bunkers, and Apple MLX framework remains a hobby project for Apple

They use their own M chips for IA. They are far more advanced on AI than the majority of company. They are using OpenAI for now but in couple months they will own the full chain of value.

we’ve heard that claim for the past three years, but every effort by them points to the opposite. don’t get me wrong, I would love for Apple Intelligence to be smart enough on my iPhone and on my Mac, but honestly, the current version is a complete disappointment.

Re: Apple M3 Ultra

#629

Post by hatthew » Wed, Mar 05, 2025, 7:59 PM UTC

hinkley wrote: post #587

aurareturn wrote: post #421

Earlier quoted context omitted.

between 4.25 to 3.5 TPS (tokens per second) on the Q4 671b full model. 3.5 - 4.25 tokens/s. You're torturing yourself. Especially with a reasoning model. This will run it at 40 tokens/s based on rough calculation. Q4 quant. 37b active parameters. 5x higher price for 10x higher performance.

Also you don't have to deal with Windows. Which people who do not understand Apple are very skilled at not noticing. If you've ever used git, svn, or an IDE side by side on corporate Windows versus Apple I don't know why you would ever go back.

Is there a reason one couldn't use linux?

Re: Apple M3 Ultra

#630

Post by bigyabai » Wed, Mar 05, 2025, 7:59 PM UTC

hinkley wrote: post #587

aurareturn wrote: post #421

Earlier quoted context omitted.

between 4.25 to 3.5 TPS (tokens per second) on the Q4 671b full model. 3.5 - 4.25 tokens/s. You're torturing yourself. Especially with a reasoning model. This will run it at 40 tokens/s based on rough calculation. Q4 quant. 37b active parameters. 5x higher price for 10x higher performance.

Also you don't have to deal with Windows. Which people who do not understand Apple are very skilled at not noticing. If you've ever used git, svn, or an IDE side by side on corporate Windows versus Apple I don't know why you would ever go back.

The PC doesn't have to run Windows either. Strictly speaking, professional applications see MacOS support as an Apple-sanctioned detriment.

> If you've ever used git, svn, or an IDE side by side

I still reach for Windows, even though it's a dogshit OS. I would rather use WSL to write and deploy a single app, as opposed to doing my work in a Linux VM or (god forbid) writing and debugging multiple versions just to support my development runtime. If I'm going to use an ad-encumbered commercial service-slop OS, I might as well pick the one that doesn't actively block my work.