Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple M3 Ultra

apple.com

Post reply on HN
Print view View original

831–840 of 1001 posts

Re: Apple M3 Ultra

#831

Post by jjcob »

diggan wrote: post #402
aurareturn wrote: post #365

Earlier quoted context omitted.

High TDP? You mean server-grade CPUs? Apple doesn't make those.

Isn't the rack-mounted Mac Pro supposedly "server-grade" ( https://www.apple.com/shop/buy-mac/mac-pro/rack )? At least judging by the mounts, they want them to be used that way, even though the CPU might not fit with the de facto industry label for "server-grade".

The rack mount Mac Pro doesn't really make sense for a data center. It's 5U high, which is much too big for a data center. It doesn't have standard server features like redundant power supplies.

The only use case I can think of is for audio workstations, where people have lots of rack mount equipment, so you can have everything including the computer in the rack. But even for that use case it's quite big.

Re: Apple M3 Ultra

#833

Post by saagarjha »

tgma wrote: post #821
lynguist wrote: post #718

Earlier quoted context omitted.

I actually think it’s not a coincidence and they specifically built this M3 Ultra for DeepSeek R1 4-bit. They also highlight in their press release that they tested it with 600B class LLMs (DeepSeek R1 without referring to it by name). And they specifically did not stop at 256 GB RAM to make this happen. Maybe I’m reading too much into it.

Pretty sure this has absolutely nothing to do with Deepseek and even local LLM at large, which has been a thing for a while and an obvious use case original Llama leak and llama.cpp coming around. Fact is Mac Pros in the Intel days supported 1.5TB RAM in some configurations[1] and that was 6 years ago expectations of their high end customer base. They needed to address the gap for those customers so they would have s…

The thing that people are excited about here is unified memory that the GPU can address. Mac Pro had discrete GPUs with their own memory.

Re: Apple M3 Ultra

#834

Post by saagarjha »

astrange wrote: post #759
deepGem wrote: post #633

Earlier quoted context omitted.

Any idea what the sRAM to uRAM ratio is on these new GPUs ? If they have meaningfully higher sRAM than the Hopper GPUs, it could lead to meaningful speedups in large model training. If they didn't increase the memory bandwidth, then 512GB will enable longer context lengths and that's about it right? No speedups For any speedups You may need some new variant of FlashAttention3 or something along similar lines to be pu…

I don't know what you mean by s and u, but there is only one kind of memory in the machine, that's what unified memory means.

I assume they mean SRAM versus unified (D)RAM?

Re: Apple M3 Ultra

#835

Post by hot_gril »

wtallis wrote: post #786
hot_gril wrote: post #696

Earlier quoted context omitted.

M1 came out before the LLM rush, though

The M1 is in a product segment where discrete GPUs have been gone for decades , in favor of integrated graphics that shares one pool of RAM with the CPU. The better question to ask is why Apple kept using that unified memory design even when moving up to larger chips like the M1 Max and M1 Ultra.

They put the M1 into the desktops too

Re: Apple M3 Ultra

#836

Post by TylerE »

hot_gril wrote: post #694
samstave wrote: post #622

Earlier quoted context omitted.

"unified memory" funny that people think this is so new, when CRAY had Global Heap eons ago...

It's new for mainstream PCs to have it.

New for performance machines maybe. I remember "integrated graphics" when that meant some shitty co-processor and 16 or 32MB of semi-reserved system RAM.

Re: Apple M3 Ultra

#837

Post by pjmlp »

hot_gril wrote: post #694
samstave wrote: post #622

Earlier quoted context omitted.

"unified memory" funny that people think this is so new, when CRAY had Global Heap eons ago...

It's new for mainstream PCs to have it.

Nope, it was common in 8 and 16 bit home computers, and in respect to PCs themselves graphics memory was mapped into the main memory until the arrival of 3D dedicated cards.

And even with 3D, integrated GPUs have existed for years.

Re: Apple M3 Ultra

#838

Post by globular-toast »

alok-g wrote: post #594
archagon wrote: post #584

Earlier quoted context omitted.

I don’t currently use AIs, but if I did, they would be local. Simply put: I can’t build my professional career around tools that I do not own.

>> ... around tools that I do not own. That just may be dependent on how much trust you have on the providers you use. Or do you do your own electricity generation?

In my country things like electricity and water supply are considered a right and a supplier has to go to court to get a supply shut off. Unfortunately we don't yet consider an internet connection in the same way, despite the government essentially requiring it these days.

Re: Apple M3 Ultra

#839

Post by pjmlp »

pavlov wrote: post #604
choilive wrote: post #566

Earlier quoted context omitted.

never understood the hate on the trash can. Isn't the mac studio basically the same idea as the trash can but even less upgradeable?

The Mac Studio hit a sweet spot in 2023 that the trash can Mac Pro couldn't ten years earlier. It's mostly thanks to the high integration of Apple Silicon and improved device availability and speed of Thunderbolt. The 2013 Mac Pro was stuck forever with its original choice of Intel CPU and AMD GPU. And it was unfortunately prone to overheating due to these same components.

Folks that want to keep the customisation aspect of Mac Pro hardly see that.

In fact a very famous podcaster is still holding out to his.

Re: Apple M3 Ultra

#840

Post by TylerE »

kllrnohj wrote: post #513
matt-p wrote: post #453

Earlier quoted context omitted.

Not really like for like. The pricing isn't as insane as you'd think, 96 to 256GB is 1500 which isn't 'cheap' but, it could be worse. All in 5,500 gets you a ultra with 256GB memory, 28 cores, 60 GPU cores, 10Gb network - I think you'd be hard pushed to build a server for less.

5,500 easily gets me either vastly more CPU cores if I care more about that or a vastly faster GPU if I care more about that. Or for both a 9950x + 5090 (assuming you can actually find one in stock) is ~$3000 for the pair + motherboard, leaving a solid $2500 for whatever amount of RAM, storage, and networking you desire. The M3 strikes a very particular middle ground for AI of lots of RAM but a significantly slower G…

The Mac almost fits in the palm of your hand, and runs, if not silently, practically so. It doesn't draw excessive power or generate noticeable heat.

None of those will be true for any PC/Nvidia build.

It's hard to put a price on quality of life.

Post reply on HN