Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple M3 Ultra

apple.com

Post reply on HN
Print view View original

611–620 of 1001 posts

Re: Apple M3 Ultra

#611

Post by kridsdale1 »

kllrnohj wrote: post #555
matt-p wrote: post #554

Earlier quoted context omitted.

That config would also use about 10x the power, and you still wouldn't be able to run a model over 32GB whereas the studio can easily cope with 70B llama and plenty of space to grow. I think it actually is perfect for local inference in a way that build or any other pc build in this price range would be.

The M3 Ultra studio also wouldn't be able to run path traced Cyberpunk at all no matter how much RAM it has. Workloads other than local inference LLMs exist, you know :) After all, if the only thing this was built to do was run LLMs then they wouldn't have bothered adding so many CPU cores or video engines. CPU cores (along with networking) being 2 of the specs highlighted by the person I was responding to, so they w…

The core customer market for this thing remains Video Editors. That’s why they talk about simultaneous 8K encoding streams.

Apple’s Pro segment has been video editors since the 90s.

Re: Apple M3 Ultra

#612

Post by sroussey »

klausa wrote: post #297
sroussey wrote: post #273

Earlier quoted context omitted.

I’m guessing it’s not really a M3. No M3 has thunderbolt 5. This is a new chip with M3 marketing. I’d expect this from Intel, not Apple.

Baseline M4 doesn't have Thunderbolt 5 either; only the Pro/Max variants do. The press-release even calls TB5 out: >Each Thunderbolt 5 port is supported by its own custom-designed controller directly on the chip. Given that they're doing the same on A-series chips (A18 Pro with 10Gbps USB-C; A18 with USB 2.0); I imagine it's just relatively simple to swap the I/O blocks around and they're doing this for cost and/or p…

Which means this is a whole new chip. It may be M3 based, but with added interposer support and new thunderbolt stuff.

Which, at this point, why not just use M4 as a base?

Re: Apple M3 Ultra

#613

Post by kridsdale1 »

rbanffy wrote: post #522
ykl wrote: post #501

Earlier quoted context omitted.

This is cheap compared to GB200, which has a street price of >$70k for just the chip alone if you can even get one. Also GB200 technically has only 192GB per GPU and access to more than that happens over NVLink/RDMA, whereas here it’s just one big flat pool of unified memory without any tiered access topology.

We finally encountered the situation where an Apple computer is cheaper than its competition ;-) All joking aside, I don't think Apples are that expensive compared to similar high-end gear. I don't think there is any other compact desktop computer with half a terabyte of RAM accessible to the GPU.

And yet all that cash still just goes to TSMC

Re: Apple M3 Ultra

#614

Post by kridsdale1 »

phillco wrote: post #445
wtallis wrote: post #386

Earlier quoted context omitted.

The rackmount Mac Pro is for A/V studios, not datacenters.

Don't forget CI/CD farms for iOS builds, although I think it's much more cost effective to just make Minis or Studios work, despite their nonstandard formfactor

Google and Facebook have vast fleets of Minis in custom chassis for this purpose.

Re: Apple M3 Ultra

#615

Post by vlovich123 »

Espressosaurus wrote: post #460
vlovich123 wrote: post #451

Earlier quoted context omitted.

You're adding detail that's not relevant to anything I said. I was saying this statement: > VRAM is what takes a model from "can not run at all" to "can run" (even if slowly), hence the emphasis. Is false. Regardless of how much VRAM you have, if the criteria is "can run even if slowly", all machines can run all models because you have swap. It's unusably slow but that's not what OP was claiming the difference is.

The criteria for purchase for anybody trying to use it is "run slowly but acceptably" vs. "run so slow as to be unusable". My memory is wrong, it was the 32b. I'm running the 70b against a similar prompt and the 5950X is probably going to take over an hour for what the M4 managed in about 7 minutes. edit: an hour later and the 5950 isn't even done thinking yet. Token generation is generously around 1 token/s. edit ed…

I don't disagree. I'm just taking OP at the literal statement they made.

Re: Apple M3 Ultra

#616

Post by kridsdale1 »

cibyr wrote: post #419
phonon wrote: post #285

Earlier quoted context omitted.

Thunderbolt 5 can do bi-directional 80 Gbps....and Mac Studio Ultra has 6 ports...

That's still not even competitive with 100G Ethernet on a per-port basis. An overall bandwidth of 480 Gbps pales in comparison with, for example, the 3200 Gbps you get with a P5 instance on EC2.

Cost competitive though?

Re: Apple M3 Ultra

#617

Post by kridsdale1 »

icecube123 wrote: post #475
exabrial wrote: post #203

Earlier quoted context omitted.

If Apple supported Linux (headless) natively, and we could rack m4 pros, I absolutely would use them in our Colo. The CPUs have zero competition in terms of speed, memory bandwidth. Still blown away no other company has been able to produce Arm server chips that can compete.

Yea ive been thinking about this for a few years. The Mx series’s chip would sell into data centers like crazy if apple went after that market. Especially if they created a server tuned chip. It could probably be their 2nd biggest product line behind the iphone. The performance and efficiency is awesome. I guess it would be meat to see some web serving and database benchmarks to really know.

TSMC couldn’t make enough at the leading node in addition to all the iPhone chips Apple has to sell. There’s a physical thoughput limit. That’s why this isn’t M4.

Re: Apple M3 Ultra

#618

Post by theshrike79 »

alok-g wrote: post #564

Two questions for the fellow HNers: 1. What are various average joe (as opposed to researchers, etc.) use cases for running powerful AI models locally vs. just using cloud AI. Privacy of course is a benefit, but it by itself may not justify upgrades for an average user. Or are we expecting that new innovation will lead to much more proliferation of AI and use cases that will make running locally more feasible? 2. Wit…

For 1: censorship

A local model will do anything you ask it to, as far as it "knows" about it. It doesn't need to please investors or be afraid of bad press.

LM Studio + a group of select models from huggingface and you can do whatever you want.

For generic coding assistance and knowledge, online services are still better quality.

Re: Apple M3 Ultra

#619

Post by kridsdale1 »

sroussey wrote: post #612
klausa wrote: post #297

Earlier quoted context omitted.

Baseline M4 doesn't have Thunderbolt 5 either; only the Pro/Max variants do. The press-release even calls TB5 out: >Each Thunderbolt 5 port is supported by its own custom-designed controller directly on the chip. Given that they're doing the same on A-series chips (A18 Pro with 10Gbps USB-C; A18 with USB 2.0); I imagine it's just relatively simple to swap the I/O blocks around and they're doing this for cost and/or p…

Which means this is a whole new chip. It may be M3 based, but with added interposer support and new thunderbolt stuff. Which, at this point, why not just use M4 as a base?

Could be that M4 requires a different TSMC fab that is at full production doing iPhones.

Re: Apple M3 Ultra

#620

Post by kridsdale1 »

hinkley wrote: post #583
sroussey wrote: post #273

Earlier quoted context omitted.

I’m guessing it’s not really a M3. No M3 has thunderbolt 5. This is a new chip with M3 marketing. I’d expect this from Intel, not Apple.

TB 5 seems like the sort of thing you could 'slap on' to a beefy enough chip. Or the sort of thing you put onto a successor when you had your fingers crossed that the spec and hardware would finalize in time for your product launch but the fucking committee went into paralysis again at the last moment and now your product has to ship 4 months before you can put TB 5 hardware on shelves. So you put your TB4 circuitry…

Sounds like you’ve seen some things.
Post reply on HN