Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple M3 Ultra

apple.com

Post reply on HN
Print view View original

391–400 of 1001 posts

Re: Apple M3 Ultra

#391

Post by jeffhuys »

petecooper wrote: post #359
jeffhuys wrote: post #328

Earlier quoted context omitted.

I went to Transmission years and years ago because it's just simple. It has all the options if you need them, but no HUUUGE interface with RSS feeds, 10001 stats about your download, categories, tags, etc etc etc. Transmission is just a small, floating window with your downloads. Click for more. It fits in the macOS vibe. But I'm a person that fully adopted the original macOS "way of working" - kicked the full-screen…

>why would you go FROM Transmission to qBittorrent? In my case: some torrents wouldn't find known-good seeds in Transmission but worked fine in qBittorrent; there's reasonable (but not perfect) support for libtorrent 2.0 in qBittorrent; my download speeds and overall responsiveness is anecdotally better in qBittorrent, and; I make use of some of the nitty gritty settings in qBittorrent.

Well there's a list of good reasons! Thanks for answering. I haven't had any problems with finding seeds, and no need for libtorrent but now I know how to fix that when I do encounter those situations.

Re: Apple M3 Ultra

#392

Post by FloatArtifact »

valine wrote: post #378
FloatArtifact wrote: post #301

Earlier quoted context omitted.

They didn't increase the memory bandwidth. You can get the same memory bandwidth, which is available on the M2 Studio. Yes, yes, of course you can get 512 gigabytes of uRAM for 10 grand. The the question is if a llm will run with usable performance at that scale? The point is there's diminishing returns despite having enough uRAM with the same amount of memory bandwidth even with increased processing speed of the new…

Probably helps that models like deepseek are mixture of expert. Having all weights in VRAM means you don’t have to unlod/reload. Memory bandwidth usage should be limited to the 37B active parameters.

> Probably helps that models like deepseek are mixture of expert. Having all weights in VRAM means you don’t have to unlod/reload. Memory bandwidth usage should be limited to the 37B active parameters.

"Memory bandwidth usage should be limited to the 37B active parameters."

Can someone do a deep dive above quote. I understand having the entire model loaded into RAM helps with response times. However, I don't quite understand the memory bandwidth to active parameters.

Context window?

How much the model can actively be processed despite being fully loaded into memory based on memory bandwidth?

Re: Apple M3 Ultra

#393

Post by diggan »

FloatArtifact wrote: post #301
cxie wrote: post #222

512GB of unified memory is truly breaking new ground. I was wondering when Apple would overcome memory constraints, and now we're seeing a half-terabyte level of unified memory. This is incredibly practical for running large AI models locally ("600 billion parameters"), and Apple's approach of integrating this much efficient memory on a single chip is fascinating compared to NVIDIA's solutions. I'm curious about how…

They didn't increase the memory bandwidth. You can get the same memory bandwidth, which is available on the M2 Studio. Yes, yes, of course you can get 512 gigabytes of uRAM for 10 grand. The the question is if a llm will run with usable performance at that scale? The point is there's diminishing returns despite having enough uRAM with the same amount of memory bandwidth even with increased processing speed of the new…

> The the question is if a llm will run with usable performance at that scale?

This is the big question to have answered. Many people claim Apple can now reliably be used as a ML workstation, but from the numbers I've seen from benchmarks, the models may fit in memory, but the performance for tok/sec is so slow to not feel worth it, compared to running it on NVIDIA hardware.

Although it be expensive as hell to get 512GB of VRAM with NVIDIA today, maybe moves like this from Apple could push down the prices at least a little bit.

Re: Apple M3 Ultra

#394

Post by derefr »

aurareturn wrote: post #365
hedora wrote: post #349

Earlier quoted context omitted.

That’s a laptop part, so it makes different tradeoffs. Somewhere on the internet there is a tdp wattage vs performance x-y plot. There’s a pareto optimal region where all the apple and amd parts live. Apple owns low tdp, AMD owns high tdp. They duke it out in the middle. Intel is nowhere close to the line. I’d guess someone has made one that includes datacenter ARM, but I’ve never seen it.

High TDP? You mean server-grade CPUs? Apple doesn't make those.

True, but these "Ultra" chips do target the same niche as (some) high-TDP chips.

Workstations (like the Mac Studio) have traditionally been a space where "enthusiast"-grade consumer parts (think Threadripper) and actual server parts competed. The owner of a workstation didn't usually care about their machine's TDP; they just cared that it could chew through their workloads as quickly as possible. But, unlike an actual server, workstations didn't need the super-high core count required for multitenant parallelism; and would go idle for long stretches — thus benefitting (though not requiring) more-efficient power management that could drive down baseline TDP.

Re: Apple M3 Ultra

#395

Post by least »

dwedge wrote: post #369
least wrote: post #350

Earlier quoted context omitted.

I think the only laptops you won't find weird issues with linux are from smaller manufacturers dedicated to shipping them like the kde laptop or system76. Every other hardware manufacturer, including those that ship laptops with linux preinstalled, probably have weird hardware incompatibilities because they don't fully customize their SKUs with linux support in mind. Not that I'm discouraging you from switching or an…

Something like the brightness buttons not working, or sleep being a little erratic is ok. No released wifi drivers, bluetooth issues, and audio and the keyboard not working are not ok. Apple going backwards in terms of supporting Linux is not something I'm ok with.

There are wifi drivers; you just have to install them separately because they use broadcom chips. It's a proprietary blob. The other things do work, but it requires special packages and you'll need an external keyboard while installing. It's a pain to install, for sure, but it's not insurmountably difficult to get it installed.

Apple Silicon chips are arguably more compatible with Asahi Linux [1], but that's largely in thanks to the hard work of Marcan, who's stepped down as project lead from the project [2].

Overall I still think the right choice is to find a laptop better suited for the purpose of running linux on it, just something that requires more careful consideration than people think. Framework laptops, which seem well suited since ideologically it meshes well with linux users, can be a pain to set up as well.

[1] https://asahilinux.org/

[2] https://marcan.st/2025/02/resigning-as-asahi-linux-project-l...

Re: Apple M3 Ultra

#396

Post by kcb »

fintechie wrote: post #302

IMO this is a bigger blow to the AI big boys than Deepseek's release. This is massive for local inference. Exciting times ahead for open source AI.

The market for local inference and $10k+ Macs is not nearly significant enough to effect the big boys.

Re: Apple M3 Ultra

#397

Post by wpm »

Alupis wrote: post #318
bigyabai wrote: post #246

Earlier quoted context omitted.

It's the Ultra chip, the same one that goes into the rackmount Mac Pro. I don't think there's much confusion as to who this is for. > there’s no sign that they’re interested in updating that for the AI age. https://security.apple.com/blog/private-cloud-compute/

Outside of extremely niche use cases, who is racking apple products in 2025?

AWS

Re: Apple M3 Ultra

#398

Post by varjag »

School-Cotton wrote: post #75
varjag wrote: post #39

Earlier quoted context omitted.

It is exactly the opposite. Every computer architecture in production addresses memory in the powers of two. SI has no business in memory size nomenclature as it is not derived from fundamental physical units. The whole klownbyte change was pushed through by hard drive marketers in 1990s.

> Every computer architecture in production addresses memory in the powers of two. What does it mean to "address memory in powers of two" ? There are certainly machines with non-power-of-two memory quantities; 96 GiB is common for example. > The whole klownbyte change was pushed through by hard drive marketers in 1990s. The metric prefixes based on powers of 10 have been around since the 1790s.

> What does it mean to "address memory in powers of two" ? There are certainly machines with non-power-of-two memory quantities; 96 GiB is common for example.

I challenge you to show me any SKU from any memory manufacturer that has a power of 10 capacity. Or a CPU whose address space is a power of 10. This is an unavoidable artefact of using a binary address bus.

> The metric prefixes based on powers of 10 have been around since the 1790s.

And Babylonians used power of 60, what gives?

Re: Apple M3 Ultra

#399

Post by e40 »

petecooper wrote: post #173
e40 wrote: post #139

Earlier quoted context omitted.

I torrent things from two different hosts on my gigabit network. The macos stack literally cannot handle the full bandwidth I have. It fails and the machine needs to be rebooted to fix it. It’s not pretty on the way into this state, either. Other remote connections to the computer are unreliable. On Linux, running the same app in a docker container works perfectly. Transmission is the app.

>Transmission is the app. Former Transmission user here. I realise you didn't ask, but you might find some improvements in qBittorrent.

The Linux version, in a container no less, handles the entire gigabit bandwidth.

And let's be clear, it wasn't the app that had problems, the Apple Remote Desktop connection to the machine failed when the speeds got above 40MB/s and the network interface stopped working around 80MB/s.

I think Transmission works perfectly fine. I've been using it for 10+ years with no issues at all on Linux.

I forgot to mention this is a Mac mini/Intel (2018).

Re: Apple M3 Ultra

#400

Post by dheera »

valine wrote: post #382
dheera wrote: post #375

Earlier quoted context omitted.

It will cost 4X what it costs to get 512GB on an x86 server motherboard.

What would it cost to get 512GB of VRAM on an Nvidia card? That’s the real comparison.

Apples to oranges. NVIDIA cards have an order of magnitude more horsepower for compute than this thing. A B100 has 8 TB/s of memory bandwidth, 10 times more than this. If NVIDIA made a card with 512GB of HBM I'd expect it to cost $150K.

The compute and memory bandwidth of the M3 Ultra is more in-line with what you'd get from a Xeon or Epyc/Threadripper CPU on a server motherboard; it's just that the x86 "way" of doing things is usually to attach a GPU for way more horsepower rather than squeezing it out of the CPU.

This will be good for local LLM inference, but not so much for training.

Post reply on HN