Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple M3 Ultra

apple.com

Post reply on HN
Print view View original

901–910 of 1001 posts

Re: Apple M3 Ultra

#901

Post by fweimer »

wtallis wrote: post #703
olyjohn wrote: post #463

Earlier quoted context omitted.

What does soldering it to the board have to do with reliability? If they were soldered onto those systems you talk about, all those would have had to be replaced instead of just having the drive swapped out and re-imaged.

I think the implication was that a soldered SSD doesn't give the customer as much chance to pick the wrong SSD. But it's still possible for the customer to have a different use case in mind than the OEM did when the OEM is picking what SSD to include.

It wouldn't solve other mismatched expectations. For example, the vendor might ship those SSDs only to store firmware-initiated crash dumps. They don't expect them to be used to meet production storage requirements. Maybe to occasionally boot a maintenance system, but that's it. To them, this is kind of obvious because everybody has a SAN anyway. But of course, this is not actually true in practice because customers vary a bit.

Re: Apple M3 Ultra

#902

Post by rafram »

egorfine wrote: post #887
mannyv wrote: post #547

Earlier quoted context omitted.

Funny, I ran a bunch of Mac minis in colo for over a decade with no problems. Maybe you have a config problem? Of course, I had a LOM/KVM and redundant networking etc. They were substantially more reliable than the Dell equipment that I used in my day job for sure.

Hardware-wise I have exactly zero complaints. Software-wise it's much different to an expected behavior. For example, macOS won't let you in over SSH until you log in via Remote Desktop. You'll get "connection closed" immediately. Or sometimes it will. And that depends not on the count of connection attempts or anything you can do locally but rather on the boot process somehow. Sometimes it boots in a way that permit…

https://support.apple.com/lt-lt/guide/mac-help/mchlp1066/mac

Re: Apple M3 Ultra

#903

Post by cubefox »

xu_ituairo wrote: post #803
cubefox wrote: post #707

Earlier quoted context omitted.

If the energy efficiency of things like Face ID was indeed so far so bad that you need a more efficient M3 Ultra, how come Face ID was integrated into smartphones years ago, apparently without significant negative impact on battery life?

FaceID was just one example they gave (which is probably faster and more energy efficient now). Image recognition, OCR, AR and more are applications of the NPU that didn’t exist at all on older iPhones because they have would be too intensive for the chips and batteries.

That's false. Face ID is in fact a complex form of image recognition, so image recognition was definitely possible on older NPUs. OCR is the simplest form of image recognition (OCR was literally the first application of LeCun's CNN), so this was definitely possible as well. "AR" is an extremely vague term. If you refer to Snapchat style video overlays, those have been possible for a long time as well.

Re: Apple M3 Ultra

#904

Post by pier25 »

toasterlovin wrote: post #812
pier25 wrote: post #84

Earlier quoted context omitted.

I've always maintained that the M2 Mac Pro was really a dev kit for manufacturers of PCI parts. It's such a meaningless product otherwise.

IMO they had plans for a Mac Pro chip that didn’t work out, so they released the M2 version to let their Mac Pro customers know that they’re still committed to the product in the Apple Silicon era.

Could be. I'm not sure if this current incarnation of the Mac Pro signals a commitment to the product though. Same performance as the Mac Studio but 2-3x the price just to get PCI slots.

Re: Apple M3 Ultra

#905

Post by asadalt »

rtwld wrote: post #881
jjuliano wrote: post #870

Currently, Docker does not support Metal GPUs. When running LLMs on Docker with an Apple M3 or M4 chip, they will operate in CPU mode regardless of the chip's class, as Docker only supports Nvidia and Radeon GPUs. If you're developing LLMs on Docker, consider getting a Framework laptop with an Nvidia or Radeon GPU instead. Source: I develop an AI agent framework that runs LLMs inside Docker on an M3 Max ( https://kde…

Podman does support GLU acceleration through libkrun with virtio-gpu (venus) on Mac: https://podman-desktop.io/docs/podman/gpu

how is it in practice (if you have tried). I have some vulkan work and I am too lazy to setup a new ec2 for it.

Re: Apple M3 Ultra

#906

Post by Glide »

someothherguyy wrote: post #523
Spooky23 wrote: post #187

Earlier quoted context omitted.

> that 10k is a absolute bargain The higher end NVidia workstation boxes won’t run well on normal 20amp plugs. So you need to move them to a computer room (whoops, ripped those out already) or spend months getting dedicated circuits run to office spaces.

In the US, normal circuits aren't always 20A, especially in residential buildings, where they are more commonly 15A in bedrooms and offices. https://en.wikipedia.org/wiki/NEMA_connector

I would check your breaker box as well. If a hair dryer trips anything then… well yeah probably older construction.

Re: Apple M3 Ultra

#908

Post by asdffdasy »

cxie wrote: post #222

512GB of unified memory is truly breaking new ground. I was wondering when Apple would overcome memory constraints, and now we're seeing a half-terabyte level of unified memory. This is incredibly practical for running large AI models locally ("600 billion parameters"), and Apple's approach of integrating this much efficient memory on a single chip is fascinating compared to NVIDIA's solutions. I'm curious about how…

still not ECC

Re: Apple M3 Ultra

#909

Post by Cane_P »

justincormack wrote: post #626
dragonwriter wrote: post #541

Earlier quoted context omitted.

128GB for 3K; per the announcement their ConnectX networking allows two Project Digits devices to be plugged into eachother and work together as one device giving you 256GB for $6k, and, AFAIK, existing frameworks can split models across devices, as well, hence, presumably, the upthread suggestion that Project Digits would provide 512GB for $12k, though arguably the last step is cheating.

the reason Nvidia only talk about two machines over the network is I think they only have one network port, so you need to add costs for a switch.

It clearly have two ports. Just watch on the right side of the picture:

https://www.storagereview.com/wp-content/uploads/2025/01/Sto...

You will however get half of the bandwidth and a lot more latency if you have to go through multiple systems.

Re: Apple M3 Ultra

#910

Post by ryao »

pklausler wrote: post #826
musicale wrote: post #817

Earlier quoted context omitted.

> so you had to have lots of operand re-use to not be memory-bound Looking at Nvidia's spec sheet, an H100 SXM can do 989 tf32 teraflops (or 67 non-tensor core fp32 teraflops?) and 3.35 TB/s memory (HBM) bandwidth, so ... similar problem?

There is caching today.

The cache hitrate is effectively 0 for LLMs since the datasets are so huge.
Post reply on HN