Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple M3 Ultra

apple.com

Post reply on HN
Print view View original

161–170 of 1001 posts

Re: Apple M3 Ultra

#161

Post by dghlsakjg »

dwedge wrote: post #146
darthrupert wrote: post #110

Earlier quoted context omitted.

Yeah, if only Apple at least semi-supported Linux, their computers would have no competition.

I've been buying and using MBP for 6 or 7 years now, and just assumed I could run Linux on one if I wanted to. I just spent a couple of days trying to get a 2018 MBP working with Linux and found out [edit to clarify] that my other ARM MBP basically won't work. I just want a break from MacOS, I'll be buying a Thinkpad and will probably never come back. This isn't my moaning, I understand it's their market, but if thei…

A 2018 MacBook would be an intel x86 chip. It’s incredibly easy to get Linux running on that machine.

Re: Apple M3 Ultra

#162

Post by codedokode »

jeroenhd wrote: post #141
TheTxT wrote: post #8

512GB unified memory is absolutely wild for AI stuff! Compared to how many NVIDIA GPUs you would need, the pricing looks almost reasonable.

If you're going to overthrow your entire AI workflow to use a different API anyway, surely the AMD Instinct accelerator cards make more sense. They're expensive, but also a lot faster, and you don't need to deal with making your code work on macOS.

I don't think API has any value because writing software is free and hardware for ML is super expensive.

Re: Apple M3 Ultra

#163

Post by dagmx »

NorwegianDude wrote: post #122

The memory amount is fantastic, memory bandwidth is half decent(~800 GB/s), and the compute capabilities are terrible(36 TOPS). For comparison, a single consumer card like the RTX 5090 is only 32 GB of memory, has 1792 GB/s memory and 3593 TOPS of compute. The use cases will be limited. While you can't run a 600B model directly like Apple says(cause you need more memory for that), you can run a quantized version, but…

You're comparing two different things.

The compute level you’re talking about on the M3 Ultra is the neural engine. Not including the GPU.

I expect the GPU here will be behind a 5090 for compute but not by the unrelated numbers you’re quoting. After all, the 5090 alone is multiple times the wattage of this SoC.

Re: Apple M3 Ultra

#164

Post by 827a »

InTheArena wrote: post #55
ksec wrote: post #22

Previous model of M2 Ultra had max memory of 192GB. Or 128GB for Pro and some other M3 model, which I think is plenty for even 99.9% of professional task. They now bump it to 512GB . Along with insane price tag of $9499 for 512GB Mac Studio. I am pretty sure this is some AI Gold rush.

Every single AI shop on the planet is trying to figure out if there is enough compute or not to make this a reasonable AI path. If the answer is yes, that 10k is a absolute bargain.

Is this actually true? Were people doing this with the 192gb of the M2 Ultra?

I'm curious to learn how AI shops are actually doing model development if anyone has experience there. What I imagined was: Its all in the "cloud" (or, their own infra), and the local machine doesn't matter. If it did matter, the nvidia software stack is too important, especially given that a 512gb M3 Ultra config costs $10,000+.

Re: Apple M3 Ultra

#165

Post by [deleted] »

dwedge wrote: post #146
darthrupert wrote: post #110

Earlier quoted context omitted.

Yeah, if only Apple at least semi-supported Linux, their computers would have no competition.

I've been buying and using MBP for 6 or 7 years now, and just assumed I could run Linux on one if I wanted to. I just spent a couple of days trying to get a 2018 MBP working with Linux and found out [edit to clarify] that my other ARM MBP basically won't work. I just want a break from MacOS, I'll be buying a Thinkpad and will probably never come back. This isn't my moaning, I understand it's their market, but if thei…

[deleted]

Re: Apple M3 Ultra

#166

Post by re-thc »

dylan604 wrote: post #117
c0deR3D wrote: post #82

When would Apple silicons made natively support for OSes such as Linux? Apple seemlingly reluctant to release detailed technical reference manual for M-series SoCs, which makes running Linux natively on Apple silicon challenging.

That’s what’s weird to me too. It’s not like they would lose sales of macOS as it is given away with the hardware. So if someone wants to buy Apple hardware to run Linux, it does not have a negative affect to AAPL

> So if someone wants to buy Apple hardware to run Linux, it does not have a negative affect to AAPL

It does. Support costs. How do you prove it's a hardware failure or software? What should they do? Say it "unofficially" supports Linux? People would still try to get support. Eventually they'd have to test it themselves etc.

Re: Apple M3 Ultra

#167

Post by LordIllidan »

dwedge wrote: post #146
darthrupert wrote: post #110

Earlier quoted context omitted.

Yeah, if only Apple at least semi-supported Linux, their computers would have no competition.

I've been buying and using MBP for 6 or 7 years now, and just assumed I could run Linux on one if I wanted to. I just spent a couple of days trying to get a 2018 MBP working with Linux and found out [edit to clarify] that my other ARM MBP basically won't work. I just want a break from MacOS, I'll be buying a Thinkpad and will probably never come back. This isn't my moaning, I understand it's their market, but if thei…

2018 MBP is Intel unless you're referring to the T2 chip?

Re: Apple M3 Ultra

#168

Post by internetter »

InTheArena wrote: post #55
ksec wrote: post #22

Previous model of M2 Ultra had max memory of 192GB. Or 128GB for Pro and some other M3 model, which I think is plenty for even 99.9% of professional task. They now bump it to 512GB . Along with insane price tag of $9499 for 512GB Mac Studio. I am pretty sure this is some AI Gold rush.

Every single AI shop on the planet is trying to figure out if there is enough compute or not to make this a reasonable AI path. If the answer is yes, that 10k is a absolute bargain.

No AI shop is buying macs to use as a server. Apple should really release some server macOS distribution, maybe even rackable M-series chips. I believe they have one internally.

Re: Apple M3 Ultra

#169

Post by ttul »

gatienboquet wrote: post #107
behnamoh wrote: post #35

819GB/s bandwidth... what's the point of 512GB RAM for LLMs on this Mac Studio if the speed is painfully slow? it's as if Apple doesn't want to compete with Nvidia... this is really disappointing in a Mac Studio. FYI: M2 Ultra already has 800GB/s bandwidth

NVIDIA RTX 4090: ~1,008 GB/s NVIDIA RTX 4080: ~717 GB/s AMD Radeon RX 7900 XTX: ~960 GB/s AMD Radeon RX 7900 XT: ~800 GB/s How's that slow exactly ? You can have 10000000Gb/s and without enough VRAM it's useless.

I have a 4090 and, out of curiosity, I looked up the FLOPS in comparison with Apple chips.

Nvidia RTX 4090 (Ada Lovelace)

FP32: Approximately 82.6 TFLOPS

FP16: When using its 4th‑generation Tensor Cores in FP16 mode with FP32 accumulation, it can deliver roughly 165.2 TFLOPS (in non‑tensor mode, the FP16 rate is similar to FP32).

FP8: The Ada architecture introduces support for an FP8 format; using this mode (again with FP32 accumulation), the RTX 4090 can achieve roughly 330.3 TFLOPS (or about 660.6 TOPS, depending on how you count operations).

Apple M1 Ultra (The previous‑generation top‑end Apple chip)

FP32: Around 15.9 TFLOPS (as reported in various benchmarks)

FP16: By similar scaling, FP16 performance would be roughly double that value—approximately 31.8 TFLOPS (again, an estimate based on common patterns in Apple’s GPU designs)

FP8: Like the M3 family, the M1 Ultra does not support a dedicated FP8 precision mode.

So a $2000 Nvidia 4090 gives you about 5x the FLOPS, but with far less high speed RAM (24GB vs. 512GB from Apple in the new M3 Ultra). The RAM bandwidth on the Nvidia card is over 1TBps, compared with 800GBps for Apple Silicon.

Apple is catching up here and I am very keen for them to continue doing so! Anything that knocks Nvidia down a notch is good for humanity.

Post reply on HN