Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

developer.apple.com

Post reply on HN
Print view View original

231–240 of 304 posts

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#231

Post by NSUserDefaults »

lukeh wrote: post #198
FireBeyond wrote: post #76

Earlier quoted context omitted.

I doubt many of them are, either. When the 2019 Mac Pro came out, it was "amazing" how many still photography YouTubers all got launch day deliveries of the same BTO Mac Pro, with exactly the same spec: 18 core CPU, 384GB memory, Vega II Duo GPU and an 8TB SSD. Or, more likely, Apple worked with them and made sure each of them had this Mac on launch day, while they waited for the model they actually ordered . Because…

Still rocking a 2019 Mac Pro with 192GB RAM for audio work, because I need the slots and I can’t justify the expense of a new one. But I’m sure a M4 Mini is faster.

How crazy do you have to get with # of tracks or plugins before it starts to struggle? I was under the impression that most studios would be fine with an Intel Mac Mini + external storage.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#232

Post by rpastuszak »

mechagodzilla wrote: post #55
reilly3000 wrote: post #48

dang I wish I could share md tables. Here’s a text edition: For $50k the inference hardware market forces a trade-off between capacity and throughput: * Apple M3 Ultra Cluster ($50k): Maximizes capacity (3TB). It is the only option in this price class capable of running 3T+ parameter models (e.g., Kimi k2), albeit at low speeds (~15 t/s). * NVIDIA RTX 6000 Workstation ($50k): Maximizes throughput (>80 t/s). It is sup…

You can keep scaling down! I spent $2k on an old dual-socket xeon workstation with 768GB of RAM - I can run Deepseek-R1 at ~1-2 tokens/sec.

Nice! What do you use it for?

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#233

Post by r0b05 »

dghlsakjg wrote: post #162
inferiorhuman wrote: post #158

Earlier quoted context omitted.

People that can reliably predict the future You don't need to be a genius or a billionaire to realize that when most of the global supply of a product becomes unavailable the remaining supply gets more expensive. here’s an equivalent speced pc available in the US for $439 with a prime membership. So with prime that's $439+139 for $578 which is only slightly higher than the cost without prime of $549.99.

> You don't need to be a genius or a billionaire to realize that when most of the global supply of a product becomes unavailable the remaining supply gets more expensive. Yes. Absolutely correct if you are talking about the short term. I was talking about the long term, and said that. If you are so certain would you take this bet: any odds, any amount that within 1 month I can buy 32gb of new retail DDR5 in the US fo…

What is your estimate for when memory prices will decrease?

I agree that we've seen similar fluctuations in the past and the price of compute trends down in the long-term. This could be a bubble, which it likely is, in which case prices should return to baseline eventually. The political climate is extremely challenging at this time though so things could take longer to stabilize. Do you think we're in this ride for months or years?

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#234

Post by r0b05 »

jacquesm wrote: post #226
mechagodzilla wrote: post #55

Earlier quoted context omitted.

You can keep scaling down! I spent $2k on an old dual-socket xeon workstation with 768GB of RAM - I can run Deepseek-R1 at ~1-2 tokens/sec.

I did the same, then put in 14 3090's. It's a little bit power hungry but fairly impressive performance wise. The hardest parts are power distribution and riser cards but I found good solutions for both.

I think 14 3090's are more than a little power hungry!

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#235

Post by mjlee »

bigyabai wrote: post #67
andy99 wrote: post #30

Earlier quoted context omitted.

I’m hoping this isn’t as attractive as it sounds for non-hobbyists because the performance won’t scale well to parallel workloads or even context processing, where parallelism can be better used. Hopefully this makes it really nice for people that want the experiment with LLMs and have a local model but means well funded companies won’t have any reason to grab them all vs GPUs.

The lack of official Linux/BSD support is enough to make it DOA for any serious large-scale deployment. Until Apple figures out what they're doing on that front, you've got nothing to worry about.

Why? AWS manages to do it (https://aws.amazon.com/ec2/instance-types/mac/). Smaller companies too - https://macstadium.com

Having used both professionally, once you understand how to drive Apple's MDM, Mac OS is as easy to sysadmin as Linux. I'll grant you it's a steep learning curve, but so is Linux/BSD if you're coming at it fresh.

In certain ways it's easier - if you buy a device through Apple Business you can have it so that you (or someone working in a remote location) can take it out of the shrink wrap, connect it to the internet, and get a configured and managed device automatically. No PXE boot, no disk imaging, no having it shipped to you to configure and ship out again. If you've done it properly the user can't interrupt/corrupt the process.

The only thing they're really missing is an iLo, I can imagine how AWS solved that, but I'd love to know.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#236

Post by trvz »

madeofpalk wrote: post #218
DrStartup wrote: post #118

Earlier quoted context omitted.

I'm neither and have 2. 24/7 async inference against github issues. Free. (once you buy the macs that is)

I'm not sure who 'home users' are, but i doubt they're buying two $9,499 computers.

Peanuts for people who make their living with computers.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#238

Post by dghlsakjg »

jeroenhd wrote: post #225
dghlsakjg wrote: post #93

Earlier quoted context omitted.

Huh? Home PCs are as cheap as they’ve ever been. Adjusted for inflation the same can be said about “home use” Macs. The list price of an entry level MacBook Air has been pretty much the same for more than a decade. Adjust for inflation, and you get a MacBook air for less than half the real cost of the launch model that is massively better in every way. A blip in high end RAM prices has no bearing on affordable home c…

> I can get a Ryzen 7 system with 32gb of ddr5, and a 1tb drive delivered to my house before dinner tomorrow for $500 + tax That's an amazing price, but I'd like to see where you're getting it. 32GB of RAM alone costs €450 here (€250 if you're willing to trust Amazon's February 2026 delivery dates). Getting a PC isn't that expensive, but after the blockchain hype and then the AI hype, prices have yet to come down. Al…

Amazon[0] link below. Equivalent systems also available at Newegg for the same price since someone nitpicked that you need a $15 prime membership to get that Amazon deal.

Shipping might screw you but here’s in stock 32gb kits of name brand RAM from a well known retailer in the US for $280[1].

Edit: same crucial RAM kit is 220GBP in stock at amazon[2]

(0)https://www.amazon.com/BOSGAME-P3-Gigabit-Ethernet-Computer/...

(1)https://www.bhphotovideo.com/c/product/1809983-REG/crucial_c...

(2) https://www.amazon.co.uk/dp/B0CTHXMYL8?tag=pcp0f-21&linkCode...

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#239

Post by jon-wood »

trvz wrote: post #236
madeofpalk wrote: post #218

Earlier quoted context omitted.

I'm not sure who 'home users' are, but i doubt they're buying two $9,499 computers.

Peanuts for people who make their living with computers.

So, not a home user then. If you make your living with computers in that manner you are by definition a professional, and just happen to have your work hardware at home.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#240

Post by jacquesm »

r0b05 wrote: post #234
jacquesm wrote: post #226

Earlier quoted context omitted.

I did the same, then put in 14 3090's. It's a little bit power hungry but fairly impressive performance wise. The hardest parts are power distribution and riser cards but I found good solutions for both.

I think 14 3090's are more than a little power hungry!

to the point that I had to pull an extra circuit... but tri phase so good to go even if I would like to go bigger.

I've limited power consumption to what I consider the optimum, each card will draw ~275 Watts (you can very nicely configure this on a per-card basis). The server itself also uses some for the motherboard, the whole rig is powered from 4 1600W supplies, the gpus are divided 5/5/4 and the mother board is connected to its own supply. It's a bit close to the edge for the supplies that have five 3090's on them but so far it held up quite well, even with higher ambient temps.

Interesting tidbit: at 4 lanes/card throughput is barely impacted, 1 or 2 is definitely too low. 8 would be great but the CPUs don't have that many lanes.

I also have a threadripper which should be able to handle that much RAM but at current RAM prices that's not interesting (that server I could populate with RAM that I still had that fit that board, and some more I bought from a refurbisher).

Post reply on HN