Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple M3 Ultra

apple.com

Post reply on HN
Print view View original

841–850 of 1001 posts

Re: Apple M3 Ultra

#841

Post by TylerE »

pavlov wrote: post #604
choilive wrote: post #566

Earlier quoted context omitted.

never understood the hate on the trash can. Isn't the mac studio basically the same idea as the trash can but even less upgradeable?

The Mac Studio hit a sweet spot in 2023 that the trash can Mac Pro couldn't ten years earlier. It's mostly thanks to the high integration of Apple Silicon and improved device availability and speed of Thunderbolt. The 2013 Mac Pro was stuck forever with its original choice of Intel CPU and AMD GPU. And it was unfortunately prone to overheating due to these same components.

The Studio also hits a sweet spot for home users like me that want tons of IO and no built in input devices.

Re: Apple M3 Ultra

#842

Post by pjmlp »

kokada wrote: post #188
InTheArena wrote: post #29

Whoa. M3 instead of M4. I wonder if this was basically binning, but I thought that I had read somewhere that the interposer that enabled this for the M1 chips where not available. That Said, 512GB of unified ram with access to the NPU is absolutely a game changer. My guess is that Apple developed this chip for their internal AI efforts, and are now at the point where they are releasing it publicly for others to use.…

> This hardware is really being held back by the operating system at this point. Apple could either create a 2U rack hardware and support Linux (and I mean Apple supporting it, not hobbysts), or have a build of Darwin headless that could run on that hardware. But in the later case, we probably wouldn't have much software available (though I am sure people would eventually starting porting software to it, there is alr…

Apple was once in the server market, they decided a few times actually, that isn't where they want to be.

Re: Apple M3 Ultra

#843

Post by water9 »

saagarjha wrote: post #833
tgma wrote: post #821

Earlier quoted context omitted.

Pretty sure this has absolutely nothing to do with Deepseek and even local LLM at large, which has been a thing for a while and an obvious use case original Llama leak and llama.cpp coming around. Fact is Mac Pros in the Intel days supported 1.5TB RAM in some configurations[1] and that was 6 years ago expectations of their high end customer base. They needed to address the gap for those customers so they would have s…

The thing that people are excited about here is unified memory that the GPU can address. Mac Pro had discrete GPUs with their own memory.

Intel integrated graphics, technically also used unified memory with the standard dram

Re: Apple M3 Ultra

#844

Post by pjmlp »

orangecat wrote: post #752
thesuitonym wrote: post #558

Earlier quoted context omitted.

> But you have to bring your own engineering effort to actually make another OS run. I mean, that's usually how it works though. When IBM launched the PS/2, they didn't support anything other than PC-DOS and OS/2, Microsoft had to make MS-DOS work for it (I mean... they did get support from IBM, but not really), the 386BSD and Linux communities brought the engineering effort without IBM's involvement. When Apple was…

When Apple was making Motorola Macs, they may have given Be a little help, but didn't support any other OSes that appeared. Same with PowerPC. Apple briefly supported a Linux distribution on PowerPC Macs: https://en.wikipedia.org/wiki/MkLinux .

However to be more accurate that is from a time and age, where how management was going, maybe there wouldn't be an Apple today.

Re: Apple M3 Ultra

#845

Post by water9 »

nightski wrote: post #794
lynguist wrote: post #718

Earlier quoted context omitted.

I actually think it’s not a coincidence and they specifically built this M3 Ultra for DeepSeek R1 4-bit. They also highlight in their press release that they tested it with 600B class LLMs (DeepSeek R1 without referring to it by name). And they specifically did not stop at 256 GB RAM to make this happen. Maybe I’m reading too much into it.

$10k to run a 4 bit quantized model. Ouch.

The M4 MacBook Pro 128GB can run a 32B perimeter model with an 8 bit quantized model just fine

Re: Apple M3 Ultra

#846

Post by VVilhelmsen »

Who is this made for? Who needs a personal computer this powerful? Not trying to be funny - it's a genuine question.

Gamers don't generally use a mac because of the lack of games and I'm guessing those who are really into LLMs use Linux for the flexibility. Video editing can be done on much cheaper hardware.

Very rich LLM enthusiasts who wants to try out mac?

Re: Apple M3 Ultra

#848

Post by jamesy0ung »

bastardoperator wrote: post #479
radlad wrote: post #422

Earlier quoted context omitted.

It is much slower than nVidia, but for a lot of personal-use LLM scenarios, it's very workable. And it doesn't need to be anywhere near as fast considering it's really the only viable (affordable) option for private, local inference, besides building a server like this, which is no faster: https://news.ycombinator.com/item?id=42897205

It's fast enough for me to cancel monthly AI services on a mac mini m4 max.

I presume you're using the Pro, not the Max.

Anyways, what ram config, and what model are you using?

Re: Apple M3 Ultra

#849

Post by eric-burel »

VVilhelmsen wrote: post #846

Who is this made for? Who needs a personal computer this powerful? Not trying to be funny - it's a genuine question. Gamers don't generally use a mac because of the lack of games and I'm guessing those who are really into LLMs use Linux for the flexibility. Video editing can be done on much cheaper hardware. Very rich LLM enthusiasts who wants to try out mac?

I don't think people into LLMs necessarily use Linux, most devs I see around use a mac, and I think I'll buy one and move out of Ubuntu if I start using them more seriously. The perfs are the key selling point as many professional use cases benefits from running LLMs locally.

Re: Apple M3 Ultra

#850

Post by saagarjha »

devmor wrote: post #279
kstrauser wrote: post #267

Earlier quoted context omitted.

Ah, I see what you’re saying. Basically, Darwin doesn’t support cgroups, so Docker runs Linux in a VM to get that.

I don't think it supports userland namespaces either, which is another important part of container isolation.

It has partial namespaces support for the iOS simulator.
Post reply on HN