371–380 of 1001 posts
Earlier quoted context omitted.
It's the Ultra chip, the same one that goes into the rackmount Mac Pro. I don't think there's much confusion as to who this is for. > there’s no sign that they’re interested in updating that for the AI age. https://security.apple.com/blog/private-cloud-compute/
Outside of extremely niche use cases, who is racking apple products in 2025?
Earlier quoted context omitted.
They didn't increase the memory bandwidth. You can get the same memory bandwidth, which is available on the M2 Studio. Yes, yes, of course you can get 512 gigabytes of uRAM for 10 grand. The the question is if a llm will run with usable performance at that scale? The point is there's diminishing returns despite having enough uRAM with the same amount of memory bandwidth even with increased processing speed of the new…
Guess what? I'm on a mission to completely max out all 512GB of mem...maybe by running DeepSeek on it. Pure greed!
819GB/s bandwidth... what's the point of 512GB RAM for LLMs on this Mac Studio if the speed is painfully slow? it's as if Apple doesn't want to compete with Nvidia... this is really disappointing in a Mac Studio. FYI: M2 Ultra already has 800GB/s bandwidth
what's the point of 512GB RAM for LLMs on this Mac Studio if the speed is painfully slow?
Earlier quoted context omitted.
For enterprise markets, this is table stakes. A lot of datacenter customers will probably ignore this release altogether since there isn't a high-bandwidth option for systems interconnect.
That article says you can connect them through the Thunderbolt 5 somehow to form clusters.
512GB of unified memory is truly breaking new ground. I was wondering when Apple would overcome memory constraints, and now we're seeing a half-terabyte level of unified memory. This is incredibly practical for running large AI models locally ("600 billion parameters"), and Apple's approach of integrating this much efficient memory on a single chip is fascinating compared to NVIDIA's solutions. I'm curious about how…
Earlier quoted context omitted.
If I read this right, the r8g.48xlarge at AMZN [1] has 192 cores and 1536GB which exceeds the M3 Ultra in some metrics. It reminds me of the 1990s when my old school was using Sun machines based on the 68k series and later SPARC and we were blown away with the toaster-sized HP PA RISC machine that was used for student work for all the CS classes. Then Linux came out and it was clear the 386 trashed them all in terms…
In fairness, the sun and dec boxes I used back then (up to about 1999) could hold their own against intel machines. Then, one day, we built a 5 machine amd athlon xp linux cluster for $2000 ($400/machine) that beat all the unix and windows server hardware by at least 10x on $/perf. It’s nice that we have more than one viable cpu vendor these days, though it seems like there’s only one viable fab company.
512GB of unified memory is truly breaking new ground. I was wondering when Apple would overcome memory constraints, and now we're seeing a half-terabyte level of unified memory. This is incredibly practical for running large AI models locally ("600 billion parameters"), and Apple's approach of integrating this much efficient memory on a single chip is fascinating compared to NVIDIA's solutions. I'm curious about how…
It will cost 4X what it costs to get 512GB on an x86 server motherboard.
512GB of unified memory is truly breaking new ground. I was wondering when Apple would overcome memory constraints, and now we're seeing a half-terabyte level of unified memory. This is incredibly practical for running large AI models locally ("600 billion parameters"), and Apple's approach of integrating this much efficient memory on a single chip is fascinating compared to NVIDIA's solutions. I'm curious about how…
They didn't increase the memory bandwidth. You can get the same memory bandwidth, which is available on the M2 Studio. Yes, yes, of course you can get 512 gigabytes of uRAM for 10 grand. The the question is if a llm will run with usable performance at that scale? The point is there's diminishing returns despite having enough uRAM with the same amount of memory bandwidth even with increased processing speed of the new…
Earlier quoted context omitted.
Probably some kind of local LLM server. 1TB of 1.6 TB/s memory if you link 2 together. $20k total. Half the price of a single Blackwell chip.
with a vanishingly small fraction of flops and a small fraction of memory bandwidth
You can run the full Deepseek 671b q8 model at 40 tokens/s. Q4 model at 80 tokens/s. 37B active params at a time because R1 is MoE.
Linking 2 of these together let's you run a model more capable (R1) than GPT4o at a comfortable speed at home. That was simply fantasy a year ago.
Earlier quoted context omitted.
What about serviceability? These come with soldered in ssd? That would be an issue for server use, Its too expensive to throw it away all for a broken ssd.
Nah, in many businesses, everything is on a schedule. For desktop computers, a common cycle is 4 years. For servers, maybe a little longer, but not by much. After that date arrives, it’s liquidate everything and rebuild. Having things consistently work is much cheaper than down days caused by your ancient equipment. Apple’s SSDs will make it to 5 years no problem - and more likely, 10-15 years.
Fast forward 2 years: The $50-$250K machines have a 100% drive failure rate, and some poor bastard has to fly from data center to data center to swap the $60 drive for a $120 one, then re-rack and re-image each machine.
Anyway, soldering a decent SSD to the motherboard board would actually improve reliability at all those places.