Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple M3 Ultra

apple.com

Post reply on HN
Print view View original

981–990 of 1001 posts

Re: Apple M3 Ultra

#981

Post by whimsicalism »

burnerthrow008 wrote: post #790
whimsicalism wrote: post #322

Earlier quoted context omitted.

with a vanishingly small fraction of flops and a small fraction of memory bandwidth

> with a vanishingly small fraction of flops and a small fraction of memory bandwidth Is it though? Wikipedia says [1] an M3 Max can do 14 TFLOPS of FP32, so an M3 Ultra ought to do 28 TFLOPS. nVidia claims [2] a Blackwell GPU does 80 TFLOPs of FP32. So M3 Ultra is 1/3 the speed of a Blackwell. Calling that "a vanishingly small fraction" seems like a bit of an exaggeration. I mean, by that metric, a single Blackwell…

TFLOPS are teraflops not “tensor flops”.

Blackwell and modern AI chips are built for fp16. B100 has 1750 tflops of fp16. M3 ultra has ~80tflops of fp16 or about 4% that of b100

Re: Apple M3 Ultra

#982

Post by SV_BubbleTime »

cyanydeez wrote: post #962
SV_BubbleTime wrote: post #931

Earlier quoted context omitted.

No one is saying they built a new chip. But the decision to come to market with a 512GB sku may have changed from not making sense to “people will buy this”.

Dies are designed in years. This was just a coincidence.

What part of “no one is saying they designed a new chip” is lost here?

Re: Apple M3 Ultra

#983

Post by barryrandall »

ryao wrote: post #240
stego-tech wrote: post #228

Earlier quoted context omitted.

> This hardware is really being held back by the operating system at this point. It really is. Even if they themselves won't bring back their old XServe OS variant, I'd really appreciate it if they at least partnered with a Linux or BSD (good callout, ryao) dev to bring a server OS to the hardware stack. The consumer OS, while still better (to my subjective tastes) than Windows, is increasingly hampered by bloat and…

Given shared heritage, I would expect to see Apple work with FreeBSD before I would expect Apple to work with Linux.

Sure, but FreeBSD also has a Linux compatability layer. For a company that's given up on the server market so many times, making MacOS compatible with _THE_ server OS makes a lot of sense.

Re: Apple M3 Ultra

#984

Post by __MatrixMan__ »

pmarreck wrote: post #917
tomn wrote: post #454

Earlier quoted context omitted.

This is incoherent to me. Your complaints are about packaging, but the elixir wrapper doesn't deal with that in any way -- it just wraps UV, which you could use without elixir. What am I missing? Also, typically when people say things like > Tell me, which combination of the 15+ virtual environments, dependency management and Python version managers It means they have been trapped in a cycle of thinking "just one mor…

> mostly just work that's not good enough. If I'm in the business of writing Python code, I (ideally) don't want to _also_ be in the business of working around Python design deficiencies. Either solve the problem definitively, or do not try to solve the problem at all, because the middle road just leads to endless headaches for people WHILE ALSO disincentivizing a better solution. Node has better dependency managemen…

> If I'm in the business of writing Python code

The thing is, most people who are writing python code are not in the business of writing python code. They're students, scientists, people with the word "business" or "analyst" in their title. They have bigger fish to fry than learning a different language ecosystem.

It took 30 years to get them to switch from excel to python. I think it's unrealistic to expect that they're going to switch from python any time soon. So for better or worse, these are problems that we have to solve.

Re: Apple M3 Ultra

#985

Post by cayleyh »

jjtheblunt wrote: post #942
wtallis wrote: post #820

Earlier quoted context omitted.

> but chip lithography errors (thus, yields) at the huge memory density might be partially driving up the cost for huge memory. Apple's not having TSMC fab a massive die full of memory. They're buying a bunch of small dies of commodity memory and putting them in a package with a pair of large compute dies. How many of those small commodity memory dies they use has nothing to do with yield.

Is there a teardown link available for what you wrote? If so, that’s interesting.

This has been pretty clear about all Apple chip designs, going back to some of the first A series afaik. They are "unified memory" but not "memory on die", they've always been "memory on package"-- ie. the ram is packaged together with the CPU, often under a single heat spreader, but they are separate components.

Apple's own product shots have shown this. Here's a bunch of links that clearly show the memory as separate. Lots of these modules you can make out the serial or model numbers and look up the manufacturer of them from directly :)

- Side-by-side teardown of M1 Pro vs M2 Pro laptop motherboards showing separate ram chips with discussion on how apple is moving to different type of ram configurations: https://www.ifixit.com/News/71442/tearing-down-the-14-macboo...

- M2 teardown with the chip + ram highlighted: https://www.macrumors.com/2022/07/18/macbook-air-m2-chip-tea...

- Photo of the A12 with separate ram chips on a single "package": https://en.wikipedia.org/wiki/Apple_A12X

- M1 Ultra with heat spreader removed, clearly showing 3rd party ram chips onpackage: https://iphone-mania.jp/news-487859/

Re: Apple M3 Ultra

#988

Post by teknologist »

jahewson wrote: post #867
reitzensteinm wrote: post #813

Earlier quoted context omitted.

Chip? Yes. Product? Not necessarily... It's not completely out of the question that the 512gb version of M3 Ultra was built for their internal Apple silicon servers powering Private Compute Cloud, but not intended for consumer release, until a compelling use case suddenly arrived. I don't _think_ this is what happened, but I wouldn't go as far as to call it impossible.

That's absurd. Fabing custom silicon is not something anybody does for a few thousand internal servers. The unit economics simply don't work. Plus Apple is using OpenAI to provide its larger models anyway, so the need never even existed.

That could be why they're also selling it as the Mac Studio M3 Ultra

Re: Apple M3 Ultra

#989

Post by aldonius »

bigyabai wrote: post #653
cynicalpeace wrote: post #351

Earlier quoted context omitted.

> keeping their raster performance competitive while also accelerating AI and ML. Meanwhile, Apple is directing most of their performance to just the raster stuff. They could pivot to an Nvidia-style design, but that would be pretty unprecedented (even if a seemingly correct decision). Isn't Apple also focusing on the AI stuff? How has it not already made that decision? What would prevent Apple from making that decis…

> Isn't Apple also focusing on the AI stuff? Yes, but not with their GPU architecture. Apple's big bet was on low-power NPU hardware, assuming the compute cost of inference would go down as the field progressed. This was the wrong bet - LLMs and other AIs have scaled up better than they scaled down. > How has it not already made that decision? What would prevent Apple from making that decision? I mean, for one, Apple…

I think betting on low-power NPU hardware wasn't necessarily wrong - if you're Apple you're trying to optimise performance/watt across the system as a whole. So in a context where you're shipping first-party bespoke on-device ML features it can make sense to have a modestly sized dedicated accelerator.

I'd say the biggest problem with the NPU is that you can only use it from Core ML. Even MLX can't access it it!

As you say the big world-changing LLMs are scaling up, not down. At the same time (at least so far) LLM usage is intermittent - we want to consume thousands of tokens in seconds, but a couple of times a minute. That's a client-server timesharing model for as long as the compute and memory demand can't fit on a laptop.

Re: Apple M3 Ultra

#990

Post by bustling-noose »

tgma wrote: post #824
bustling-noose wrote: post #778

Earlier quoted context omitted.

My thoughts too. This product was in the pipeline maybe 2-3 years ago. Maybe with LLMs getting popular a year ago they tried to fit more memory but it’s almost impossible to do that that close to a launch. Especially when memory is fused not just a module you can swap.

Your conclusion is correct but to be clear the memory is not "fused." It's soldered close to the main processor. Not even a Package-on-Package (two story) configuration. See photo without heatspreader here: https://wccftech.com/apple-m2-ultra-soc-delidded-package-siz...

I think by fuse I mean't its stuck on to the SOC module, not part of the SOC as I may have worded. While you could maybe still add NANDs later in the manufacturing process, it's probably not easy, especially if you need more NANDs and a larger module which might cause more design problems. The NAND is closer cause the controller is in the SOC. So the memory controller probably would also change with higher memory sizes which would mean this cannot be a last minute change.
Post reply on HN