341–350 of 1001 posts
Earlier quoted context omitted.
Lack of focus on quality of software affects all types of workloads, not just consumer-oriented or professional-oriented in isolation.
Nah, if I ever wrote an article about the software crisis on the Linux desktop, there’d be flames here making Apple’s issues look small.
Whoa. M3 instead of M4. I wonder if this was basically binning, but I thought that I had read somewhere that the interposer that enabled this for the M1 chips where not available. That Said, 512GB of unified ram with access to the NPU is absolutely a game changer. My guess is that Apple developed this chip for their internal AI efforts, and are now at the point where they are releasing it publicly for others to use.…
https://digitalspaceport.com/how-to-run-deepseek-r1-671b-ful...
Earlier quoted context omitted.
Transformers are typically memory- bandwidth bound during decoding. This chip is going to have a much worse memory b/w than the nvidia chips. My guess is that these chips could be compute-bound though given how little compute capacity they have.
VRAM capacity is the initial gatekeeper, then bandwidth becomes the limiting factor.
Earlier quoted context omitted.
VRAM is what takes a model from "can not run at all" to "can run" (even if slowly), hence the emphasis.
No, with limited VRAM you could offload the model partially or split across CPU and GPU. And since CPU has swap, you could run the absolute largest model. It’s just really really slow.
Earlier quoted context omitted.
Keep in mind the minimum configuration that has 512GB of unified RAM is $9,499.
I cannot express how dirt cheap that pricepoint is for what's on offer, especially when you're comparing it to rackmount servers. By the time you've shoehorned in an nVidia GPU and all that RAM, you're easily looking at 5x that MSRP; sure, you get proper redundancy and extendable storage for that added cost, but now you also need redundant UPSes and have local storage to manage instead of centralized SANs or NASes. F…
Earlier quoted context omitted.
NVIDIA RTX 4090: ~1,008 GB/s NVIDIA RTX 4080: ~717 GB/s AMD Radeon RX 7900 XTX: ~960 GB/s AMD Radeon RX 7900 XT: ~800 GB/s How's that slow exactly ? You can have 10000000Gb/s and without enough VRAM it's useless.
h100 sxm - 3TB/s vram is not really the limiting factor for serious actors in this space
Earlier quoted context omitted.
Keep in mind the minimum configuration that has 512GB of unified RAM is $9,499.
I cannot express how dirt cheap that pricepoint is for what's on offer, especially when you're comparing it to rackmount servers. By the time you've shoehorned in an nVidia GPU and all that RAM, you're easily looking at 5x that MSRP; sure, you get proper redundancy and extendable storage for that added cost, but now you also need redundant UPSes and have local storage to manage instead of centralized SANs or NASes. F…
That nvidia GPU setup will actually have the compute grunt to make use of the RAM, though, which this M3 Ultra probably realistically doesn't. After all, if the only thing that mattered was RAM then the 2TB you can shove into an Epyc or Xeon would already be dominating the AI industry. But they aren't, because it isn't. It certainly hits at a unique combination of things, but whether or not that's maximally useful for the money is a completely different story.
Earlier quoted context omitted.
LLMs are primarily "memory-bound" rather than "compute-bound" during normal use. The model weights (billions of parameters) must be loaded into memory before you can use them. Think of it like this: Even with a very fast chef (powerful CPU/GPU), if your kitchen counter (VRAM) is too small to lay out all the ingredients, cooking becomes inefficient or impossible. Processing power still matters for speed once everythin…
Transformers are typically memory- bandwidth bound during decoding. This chip is going to have a much worse memory b/w than the nvidia chips. My guess is that these chips could be compute-bound though given how little compute capacity they have.
Not in case of language models, which are typically bound by memory size rather than bandwidth.
Earlier quoted context omitted.
The last I checked, AMD was outperforming Apple perf/dollar on the high end, though they were close on perf/watt for the TDPs where their parts overlapped. I’d be curious to know if this changes that. It’d take a lot more than doubling cores to take out the very high power AMD parts, but this might squeeze them a bit. Interestingly, AMD has also been investing heavily in unified RAM. I wonder if they have / plan an S…
The M4 Pro is 56% faster in ST performance against AMD’s new Strix Halo while being 3.6x more efficient. Source: https://www.notebookcheck.net/AMD-Ryzen-AI-Max-395-Analysis-... Cinebench 2024 results.
Somewhere on the internet there is a tdp wattage vs performance x-y plot. There’s a pareto optimal region where all the apple and amd parts live. Apple owns low tdp, AMD owns high tdp. They duke it out in the middle. Intel is nowhere close to the line.
I’d guess someone has made one that includes datacenter ARM, but I’ve never seen it.
Earlier quoted context omitted.
Yeah, if only Apple at least semi-supported Linux, their computers would have no competition.
I've been buying and using MBP for 6 or 7 years now, and just assumed I could run Linux on one if I wanted to. I just spent a couple of days trying to get a 2018 MBP working with Linux and found out [edit to clarify] that my other ARM MBP basically won't work. I just want a break from MacOS, I'll be buying a Thinkpad and will probably never come back. This isn't my moaning, I understand it's their market, but if thei…
Not that I'm discouraging you from switching or anything. If Linux is what you want/need, there's definitely better laptops to be had than a Macbook for that purpose. It's just that weird incompatibilities and having to fight with the operating system on random issues is, at least in my experience, normal when using a linux laptop. Even my T480 which has overall excellent compatibility isn't trouble-free.