Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple M3 Ultra

apple.com

Post reply on HN
Print view View original

991–1000 of 1001 posts

Re: Apple M3 Ultra

#991

Post by teknologist »

icedchai wrote: post #964
kmacdough wrote: post #780

Earlier quoted context omitted.

That or it's the luckiest coincidence! In all seriousness, Apple is fairly consistent about not pushing specs that don't matter and >256GB is just unnecessary for most other common workloads. Factors like memory bandwidth, core count and consumption/heat would have higher impact. That said, I doubt it was explicitly for R1, but rather based the industry a few years ago when GPT 3s 170B was SOTA, but the industry was…

The high end Intel Macs supported over a TB of RAM, over 5 years ago. It's kinda crazy Apple's own high end chips didn't support more RAM. Also, the LLM use case isn't new... Though DeepSeek itself may be. RAM requirements always go up.

Just to clarify. There is an important difference between unified memory, meaning accessible by both CPU and GPU, and regular RAM that is only accessible by CPU.

Re: Apple M3 Ultra

#993

Post by jjtheblunt »

cayleyh wrote: post #985
jjtheblunt wrote: post #942

Earlier quoted context omitted.

Is there a teardown link available for what you wrote? If so, that’s interesting.

This has been pretty clear about all Apple chip designs, going back to some of the first A series afaik. They are "unified memory" but not "memory on die", they've always been "memory on package"-- ie. the ram is packaged together with the CPU, often under a single heat spreader, but they are separate components. Apple's own product shots have shown this. Here's a bunch of links that clearly show the memory as separa…

neat! thanks

Re: Apple M3 Ultra

#994

Post by [deleted] »

doctorpangloss wrote: post #492
ein0p wrote: post #467

Earlier quoted context omitted.

What people who did not actually work with this stuff in practice don't realize is the above statement only holds for batch size 1, sequence size 1. For processing the prompt you will need to read all the weights (which isn't a problem, because prefill is compute-bound, which, in turn is a problem on a weak machine like this Mac or an "EPYC build" someone else mentioned). Even for inference, batch size greater than 1…

Sure, nuance. This is why Apple makes so much fucking money: people will craft the wildest narratives about how they’re going to use this thing. It’s part of the aesthetics of spending $10,000. For every person who wants a solution to the problem of running a 400b+ parameter neural network, there are 19 who actually want an exciting experience of buying something, which is what Apple really makes. It has more in comm…

[deleted]

Re: Apple M3 Ultra

#996

Post by seec »

xp84 wrote: post #507
dylan604 wrote: post #144

Earlier quoted context omitted.

I have Mac hardware and and have spent $0 through the Mac App Store. I do not use iCloud on it either. I do on iDevices though. I must be an edge case though.

All of us on HN are basically edge cases. The main target market of Macs is super dependent on Apple service subscriptions. Maybe that's why they ship with insultingly-small SSDs by default, so that as people's photo libraries, Desktop and Documents folders fill up, Apple can "fix your problem" for you by selling you the iCloud/Apple One plan to offload most of the stuff to only live in iCloud. Either they spend the…

Of course this is the reason. And this is why Apple has become so bad for the tech enthusiasts, no matter how good the OS/software can be, you have to pay a tax that is way too big because you already have the competence that should allow you to bypass it.

It's like learning about growing vegetables in your garden but then having to pay the seeds for it much more because you actually know how to produce value with them.

The philosophy at Apple has changed from premium tools for professional to luxury device for normies that makes them pay for their incompetence.

Re: Apple M3 Ultra

#997

Post by fennecfoxy »

tgma wrote: post #824
bustling-noose wrote: post #778

Earlier quoted context omitted.

My thoughts too. This product was in the pipeline maybe 2-3 years ago. Maybe with LLMs getting popular a year ago they tried to fit more memory but it’s almost impossible to do that that close to a launch. Especially when memory is fused not just a module you can swap.

Your conclusion is correct but to be clear the memory is not "fused." It's soldered close to the main processor. Not even a Package-on-Package (two story) configuration. See photo without heatspreader here: https://wccftech.com/apple-m2-ultra-soc-delidded-package-siz...

Sheesh, the...comments on that link.

Re: Apple M3 Ultra

#998

Post by tempaccount420 »

happyopossum wrote: post #926
lynguist wrote: post #718

Earlier quoted context omitted.

I actually think it’s not a coincidence and they specifically built this M3 Ultra for DeepSeek R1 4-bit. They also highlight in their press release that they tested it with 600B class LLMs (DeepSeek R1 without referring to it by name). And they specifically did not stop at 256 GB RAM to make this happen. Maybe I’m reading too much into it.

> they specifically built this M3 Ultra for DeepSeek R1 4-bit Which came out in what, mid January? Yeah, there's no chance Apple (or anyone) has built a new chip in the last 45 days.

Don't they build these Macs just-in-time? The bandwidth doesn't change with the RAM, so surely it couldn't have been that hard to just... use higher capacity RAM modules?

Re: Apple M3 Ultra

#999

Post by angoragoats »

teknologist wrote: post #991
icedchai wrote: post #964

Earlier quoted context omitted.

The high end Intel Macs supported over a TB of RAM, over 5 years ago. It's kinda crazy Apple's own high end chips didn't support more RAM. Also, the LLM use case isn't new... Though DeepSeek itself may be. RAM requirements always go up.

Just to clarify. There is an important difference between unified memory, meaning accessible by both CPU and GPU, and regular RAM that is only accessible by CPU.

As mentioned elsewhere in this thread, unified memory has existed long before Apple released the M1 CPU, and in fact many Intel processors that Apple used before supported it (though the Mac pros that supported 1.5TB of RAM did not, as they did not have integrated graphics).

The presence of unified memory does not necessarily make a system better. It’s a trade off: the M-series systems have high memory bandwidth thanks to the large number of memory channels, and the integrated GPUs are faster than most others. But you can’t swap in a faster GPU, and when using large LLMs even a Mac Studio is quite slow compared to using discrete GPUs.

Re: Apple M3 Ultra

#1000

Post by ngneer »

astrange wrote: post #937
ngneer wrote: post #924

Earlier quoted context omitted.

Hardly. x86 and ARM follow similar security underpinnings and outcomes. ARM has TrustZone, x86 has TEEs. I cannot think of a single attack demonstrated on x86 and not on ARM or vice versa. Could you please cite one?

Who cares about TrustZone? x86 doesn't have PAC or MTE.

ARM/IoT community: https://community.arm.com/support-forums/f/architectures-and... https://community.arm.com/arm-community-blogs/b/internet-of-...

Security practitioners and academics: https://github.com/enovella/TEE-reversing/ https://media.ccc.de/v/36c3-10859-trustzone-m_eh_breaking_ar... https://www.blackhat.com/docs/us-15/materials/us-15-Shen-Att... https://i.blackhat.com/USA-19/Thursday/us-19-Peterlin-Breaki...

Sorry you could not cite an attack demonstrated on x86 and not on ARM or vice versa. Well, maybe TikTag (https://github.com/compsec-snu/tiktag) was demonstrated against ARM MTE, since you mention, and Pacman (https://pacmanattack.com/) against PAC. I was hoping for an actual response, no offense.

Post reply on HN