Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

developer.apple.com

Post reply on HN
Print view View original

281–290 of 304 posts

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#281

Post by inferiorhuman »

dghlsakjg wrote: post #162
inferiorhuman wrote: post #158

Earlier quoted context omitted.

People that can reliably predict the future You don't need to be a genius or a billionaire to realize that when most of the global supply of a product becomes unavailable the remaining supply gets more expensive. here’s an equivalent speced pc available in the US for $439 with a prime membership. So with prime that's $439+139 for $578 which is only slightly higher than the cost without prime of $549.99.

> You don't need to be a genius or a billionaire to realize that when most of the global supply of a product becomes unavailable the remaining supply gets more expensive. Yes. Absolutely correct if you are talking about the short term. I was talking about the long term, and said that. If you are so certain would you take this bet: any odds, any amount that within 1 month I can buy 32gb of new retail DDR5 in the US fo…

  At this point I can't tell if you are arguing in bad faith, or just unfamiliar with how prime works. Just in case: You have cited the cost of prime for a full year.
Oh for the love of fuck. I don't subscribe to Prime or pay any attention to how it's priced. I've gotten offers for free trials of Prime before, should I just ignore that for most people Prime is something they have to pay for?

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#282

Post by inferiorhuman »

dghlsakjg wrote: post #250
r0b05 wrote: post #233

Earlier quoted context omitted.

What is your estimate for when memory prices will decrease? I agree that we've seen similar fluctuations in the past and the price of compute trends down in the long-term. This could be a bubble, which it likely is, in which case prices should return to baseline eventually. The political climate is extremely challenging at this time though so things could take longer to stabilize. Do you think we're in this ride for…

I can’t be more clear: specificity around predicting the future is close to impossible. There are 9 figure bets on both sides of the RAM issue, and strategic national concerns. I say that prices will go down at some point in the future for reasons highlighted already, but I have no clue when. Keep in mind what I myself have said about human ability to predict the future. You would be a fool to believe anyone’s specif…

  I can’t be more clear: specificity around predicting the future is close to impossible.
And I can't be more clear: a single entity bought more than 70% of the wafer production for the next year. That's across all types of memory modules. That will increase prices.

  people complaining about RAM prices being so high, and moaning that they bought less RAM
  because of it are actually signaling through action that they think that prices will go
  down or have leveled off
No, no they're not. They're saying nothing about what they think future prices will be.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#283

Post by jabbywocker »

nntwozz wrote: post #164
jabbywocker wrote: post #122

Earlier quoted context omitted.

There’s been rumors of Apple working on M-chips that have the GPU and CPU as discrete chiplets. The original rumor said this would happen with the M5 Pro, so it’s potentially on the roadmap. Theoretically they could farm out the GPU to another company but it seems like they’re set on owning all of the hardware designs.

Apple always strives for complete vertical integration. SJ loved to quote Alan Kay: "People who are really serious about software should make their own hardware." Qualcomm are the latest on the chopping block, history repeating itself. If I were a betting man I'd say Apple's never going back.

Yeah outside of TSMC, I don’t see them ever going back to having a hardware partner.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#284

Post by jabbywocker »

alwillis wrote: post #268
jabbywocker wrote: post #111

Earlier quoted context omitted.

I’m expecting Apple to release a new Mac Pro in the next couple years who’s main marketing angle is exactly this

> I’m expecting Apple to release a new Mac Pro in the next couple years I think Apple is done with expansion slots, etc. You'll likely see M5 Mac Studios fairly soon.

I’m not saying a Mac Pro with expansion slots, I’m saying a Mac Pro whose marketing angle is locally running AI models. A hungry market that would accept moderate performance and is already used to bloated price tags has to have them salivating.

I think the hold up here is whether TSMC can actually deliver the M5 Pro/Ultra and whether the MLX team can give them a usable platform.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#285

Post by _zoltan_ »

wtallis wrote: post #278
rbanffy wrote: post #260

Earlier quoted context omitted.

> have a bunch of ARM CPU cores filling in the interior of the die The main OS needs to run somewhere. At least for now.

Sure, but 72x Neoverse V3 (approximately Cortex X3) is a choice that seems more driven by convenience than by any real need for an AI server to have tons of somewhat slow CPU cores.

there are uses cases where those cores are used for aux processing. there is more to these boxes than AI :-)

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#286

Post by platevoltage »

teaearlgraycold wrote: post #206
morshu9001 wrote: post #159

Earlier quoted context omitted.

It's also gotten cheaper nominally. I just got a new base MBA for $750. Kinda surprised, like there has to be some catch.

I feel bad for their competitors. We need good competition in the long run but over the last few years it's made less and less sense to get something other than an Apple laptop for most use cases.

I don't. They're being weighed down by Windows and to a lesser extent, x86. If they want to excel in the market, make a change. Use what Valve is doing as an example.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#287

Post by bigyabai »

mjlee wrote: post #235
bigyabai wrote: post #67

Earlier quoted context omitted.

The lack of official Linux/BSD support is enough to make it DOA for any serious large-scale deployment. Until Apple figures out what they're doing on that front, you've got nothing to worry about.

Why? AWS manages to do it ( https://aws.amazon.com/ec2/instance-types/mac/ ). Smaller companies too - https://macstadium.com Having used both professionally, once you understand how to drive Apple's MDM, Mac OS is as easy to sysadmin as Linux. I'll grant you it's a steep learning curve, but so is Linux/BSD if you're coming at it fresh. In certain ways it's easier - if you buy a device through Apple Business you can h…

Where the in the world are you working where MDM is the limiting factor on Linux deployments? North Korea?

Macs are a minority in the datacenter even compared to Windows server. The concept of a datacenter Mac would disappear completely if Apple let free OSes sign macOS/iOS apps.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#288

Post by r0b05 »

jacquesm wrote: post #280
nonplus wrote: post #279

Earlier quoted context omitted.

If you end up trying it please share your findings! I've basically been putting this kind of gear in my cart, and then deciding I dont want to manage more than the 2 3090s, 4090 and a5000 I have now, then I take the PLX out of my cart. Seeing you have the cards already it could be a good fit!

Yes, it could be. Unfortunately I'm a bit distracted by both paid work and some more urgent stuff but eventually I will get back to it. By then this whole rig might be hopelessly outdated but we've done some fun experiments with it and have kept our confidential data in-house which was the thing that mattered to me.

Yes, the privacy is amazing, and there's no rate limiting so you can be as productive as you want. There's also tons of learnings in this exercise. I have just 2x 3090's and I've learnt so much about pcie and hardware that just makes the creative process that more fun.

The next iteration of these tools will likely be more efficient so we should be able to run larger models at a lower cost. For now though, we'll run nvidia-smi and keep an eye on those power figures :)

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#290

Post by jacquesm »

r0b05 wrote: post #288
jacquesm wrote: post #280

Earlier quoted context omitted.

Yes, it could be. Unfortunately I'm a bit distracted by both paid work and some more urgent stuff but eventually I will get back to it. By then this whole rig might be hopelessly outdated but we've done some fun experiments with it and have kept our confidential data in-house which was the thing that mattered to me.

Yes, the privacy is amazing, and there's no rate limiting so you can be as productive as you want. There's also tons of learnings in this exercise. I have just 2x 3090's and I've learnt so much about pcie and hardware that just makes the creative process that more fun. The next iteration of these tools will likely be more efficient so we should be able to run larger models at a lower cost. For now though, we'll run n…

You can tune that power down to what gives you the best tokencount per joule, which I think is a very important metric by which to optimize these systems and by which you can compare them as well.

I have a hard time understanding all of these companies that toss their NDA's and client confidentiality into the wind and feed newfangled AI companies their corporate secrets with abandon. You'd think there would be a more prudent approach to this.

Post reply on HN