Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple M3 Ultra

apple.com

Post reply on HN
Print view View original

951–960 of 1001 posts

Re: Apple M3 Ultra

#951

Post by badc0ffee »

hervature wrote: post #637
someothherguyy wrote: post #523

Earlier quoted context omitted.

In the US, normal circuits aren't always 20A, especially in residential buildings, where they are more commonly 15A in bedrooms and offices. https://en.wikipedia.org/wiki/NEMA_connector

To clarify, the circuit is almost always 20A with 15A being used for lighting. However, the outlet itself is almost always 15A because you put multiple outlets on a single circuit. You are going to see very few 20A in outlets (which have a T shaped prong) in residential.

To clarify further, "20A" circuit just means a 20A breaker and suitable wire (12 AWG or larger).

Re: Apple M3 Ultra

#952

Post by badc0ffee »

tobyhinloopen wrote: post #929
Sharlin wrote: post #890

Earlier quoted context omitted.

You have a point; technically they aren't impossible to run if you have enough system RAM (or hell, SSD/HDD space for that mater). But in practice neither running on the CPU, nor on the GPU by constantly paging data in and out of VRAM, is a very attractive option (~10x slowdown at least).

So the only reason the mac is faster is because the RAM is accessible by its GPU, right? Not because the RAM is faster than regular RAM, because AFAIK it isn't far off from workstation RAM speeds.

The RAM is faster. 8 DDR5 64GB sticks at 8800 MT/s would in theory give you a maximum of 563.2 GB/s. Whereas the M3 Ultra is 819 GB/s.

Re: Apple M3 Ultra

#953

Post by baobabKoodaa »

codedokode wrote: post #162
jeroenhd wrote: post #141

Earlier quoted context omitted.

If you're going to overthrow your entire AI workflow to use a different API anyway, surely the AMD Instinct accelerator cards make more sense. They're expensive, but also a lot faster, and you don't need to deal with making your code work on macOS.

I don't think API has any value because writing software is free and hardware for ML is super expensive.

Daniel Ek spotted!

Re: Apple M3 Ultra

#954

Post by badc0ffee »

cubefox wrote: post #761
KerrAvon wrote: post #749

Earlier quoted context omitted.

You seem to be arguing with a strawman here -- who said you need an M3 Ultra for energy efficient Face ID?

"stouset" implied that those are merely possible but not energy efficient on the older mobile hardware.

The original question was asking what features have taken advantage of a NPU. Face ID was introduced with Apple's first "Neural Engine" CPU, the A11 Bionic.

You're confusing this with what features/enhancements new generations of NPUs bring, which nobody else was talking about. Everyone else in the conversation is comparing pre- and post-NPU.

Re: Apple M3 Ultra

#955

Post by perfmode »

VVilhelmsen wrote: post #846

Who is this made for? Who needs a personal computer this powerful? Not trying to be funny - it's a genuine question. Gamers don't generally use a mac because of the lack of games and I'm guessing those who are really into LLMs use Linux for the flexibility. Video editing can be done on much cheaper hardware. Very rich LLM enthusiasts who wants to try out mac?

I would love 32 cores for audio editing in RX 11.

Re: Apple M3 Ultra

#956

Post by jrflowers »

lynguist wrote: post #718
lhl wrote: post #560

Earlier quoted context omitted.

Since no one specifically answered your question yet, yes, you should be able to get usable performance. A Q4_K_M GGUF of DeepSeek-R1 is 404GB. This is a 671B MoE that "only" has 37B activations per pass. You'd probably expect in the ballpark of 20-30 tok/s (depends on how much actually MBW can be utilized) for text generation. From my napkin math, the M3 Ultra TFLOPs is still relatively low (around 43 FP16 TFLOPs?),…

I actually think it’s not a coincidence and they specifically built this M3 Ultra for DeepSeek R1 4-bit. They also highlight in their press release that they tested it with 600B class LLMs (DeepSeek R1 without referring to it by name). And they specifically did not stop at 256 GB RAM to make this happen. Maybe I’m reading too much into it.

> they specifically built this M3 Ultra for DeepSeek R1 4-bit.

This makes sense. They started gluing M* chips together to make Mac Studios three years ago, which must have been in anticipation of DeepSeek R1 4-bit

Re: Apple M3 Ultra

#957

Post by varjag »

jltsiren wrote: post #731
varjag wrote: post #39

Earlier quoted context omitted.

It is exactly the opposite. Every computer architecture in production addresses memory in the powers of two. SI has no business in memory size nomenclature as it is not derived from fundamental physical units. The whole klownbyte change was pushed through by hard drive marketers in 1990s.

It's more complicated than that. Data storage sizes are not connected to fundamental physical units, but data transfer rates are. Things get annoying when a 1 MB/s connection cannot transfer a megabyte in a second.

Line discipline rarely has sequences of bytes without any service information (parity, delimiters, preambles etc). So I don't see it as a practical issue.

Re: Apple M3 Ultra

#958

Post by [deleted] »

SV_BubbleTime wrote: post #931
happyopossum wrote: post #926

Earlier quoted context omitted.

> they specifically built this M3 Ultra for DeepSeek R1 4-bit Which came out in what, mid January? Yeah, there's no chance Apple (or anyone) has built a new chip in the last 45 days.

No one is saying they built a new chip. But the decision to come to market with a 512GB sku may have changed from not making sense to “people will buy this”.

[deleted]

Re: Apple M3 Ultra

#959

Post by gield »

cubefox wrote: post #339
gield wrote: post #198

Earlier quoted context omitted.

Face ID, taking pictures, Siri, ARKit, voice-to-text transcription, face recognition and OCR in photos, noise filtering, ...

These have been possible in much smaller smartphone chips for years.

Indeed, but the neural engine does this faster and using heavier models. For example, on-device Siri was not possible until the introduction of the neural engine in 2017.

Re: Apple M3 Ultra

#960

Post by ZeroTalent »

jauntywundrkind wrote: post #747
ZeroTalent wrote: post #687

Earlier quoted context omitted.

No, because there is no CUDA. We have fast and cheap alternatives to NVIDIA, but they do not have CUDA. This is why NVIDIA has 90% margins on its hardware.

CUDA is simply not important for modern vLLM and many many others. DeepSeek V3 works great on SGLang. https://www.amd.com/en/developer/resources/technical-article... Can you do absolutely everything? No. But most models will run or retrain fine now without CUDA. This premise keeps getting recycled from the past, even as that past has grown ever more distant.

CUDA is becoming more critical, not less, every day. Software developed around CUDA is vastly outpacing what other companies produce. And saving a few millions when creating new models doesn't matter; NVIDIA is pretty efficient at scale.

I don't know if you've heard, but NVIDIA is about to add a monthly payment for additional CUDA features and I'm almost certain that many big companies will be happy to pay for them.

> But most models will run or retrain fine now without CUDA.

This is correct for some small startups, not big companies.

Post reply on HN