291–300 of 349 posts
Earlier quoted context omitted.
At least in the UK, that's exactly what they're doing right now. They're allowing pre-orders, with shipping on Sept 22.
There are some options, though, that you can’t even pre-order until October.
Earlier quoted context omitted.
My personal laptop is a 2018 ThinkPad Carbon X1 Gen 6. It's eight years old now. Still running Linux pretty well, but I really need to figure out what to do for a replacement sooner than later...
The Framework laptop looks interesting
I may need to replace some parts at some point, but this far it's not been necessary.
I'd love to have the 16", which didn't exist when I bought the 13", but thanks to Altman, hardware prices are insane these days. :/
Earlier quoted context omitted.
Don't worry, there are exactly zero Apple chips that are good for any sort of local inference right now. Unified ram is fast for ram, but dogshit slow compared to actual dedicated VRAM on graphics cards.
Your statement is not really true any more. I have an old M1 Max with 32GB that runs inference models fine, especially those using MLX. Dedicated graphics cards are faster but not to the point that it matters. That’s a six year old machine. My newer machine M5 max 128GB will far outperform your typical 32GB gaming card once the model exceeds memory. To even get close to that on a dedicated graphics card you are payin…
To be specific, 5 models at 57 gb means you are using crap quantized models, which suck for any real agentic work. I mean, sure they give you some inference, but compared to the full parameter models like Qwen3.8 and Gemma4 that can run full agentic loops, you may as well just use cloud inference for the price.
You of course could "run" those larger models, but we both know that the tok/sec is dogshit on Macs for those.
And 57 gb is split across 3 cards quite easily, which will all be cheaper than your comparable Mac and way faster.
You really need to educated yourself on how running local models works and what the models like Gemma 4 are capable of, so you don't continue to waste money on Macs.
Earlier quoted context omitted.
Thank you, found some benchmarks and they look really promising. To my understanding those Neural Accelerators are like AMX but for GPUs. With those accelerators the GPU performance on a M5 Max in LLM inferencing would totally be on par with a 5090, that's quite impressive!
Which inference benchmark are you looking at? The prefill speeds should be comparable on some LLMs, but the 5090 has much a higher theoretical max decode speed.
With larger (also really fast) unified memory on the chip, it could easily load larger weights, and the thing is that the quantized results on the M5s are really good, I believe that for most local inferencing, users would be using INT4 (maybe more bits per weight sometimes) quantized models, that might be where the Neural Accelerators kick in. In raw power, a desktop 5090 easily outperforms the M5 Max, but for this specific use case, I believe that the M5 Max is good enough.
Earlier quoted context omitted.
The big difference you're not looking at is the memory bandwidth. The M5 Pro has nearly twice the one of the M6: 170GB/s memory bandwidth Configurable up to 32GB unified memory 307GB/s memory bandwidth Configurable up to 64GB unified memory That is also the factor you want to take into account for AI inference, of course.
you're looking at the wrong device. - Apple M5 Ultra chip - 30-core CPU with 10 super cores and 20 performance cores - 64-core GPU - Neural Accelerators - Hardware-accelerated ray tracing - 32-core Neural Engine - 1.2TB/s memory bandwidth one point 2 terrabyte a second memory bandwidth.... 512gb coming later this year but at the cost of a small car
One of the things that Steve Jobs got very right was "and you can order right now". Announcements where you have to wait a month to order throw away all of the hype. Even allowing pre-orders and shipping later is a better idea and so silly that Apple has started walking away from that.
Wonder how the cybertruck sales would have gone if tesla had done that. (250,000 pre-orders, vs now ~5000 sales/quarter)
Earlier quoted context omitted.
I know it's at least a little irrational, but I am indescribably angry with the pricing trajectory of personal computing. I'm not all that young and for pretty much all my life the computers you as a random person could buy have consistently become massively more performant and massively cheaper over time. Even when the exponential performance increases started to taper off, every few years you could buy or build a n…
I don't think it's an accident that the market is on a trajectory toward personal computer (and particularly storage) rental being the only economically viable game. There are significant players who will profit directly. Indirectly it helps companies for whom end users owning general purpose computers and storage threaten their business models. The industry has little incentive to fix this problem and as the market…
Earlier quoted context omitted.
you're looking at the wrong device. - Apple M5 Ultra chip - 30-core CPU with 10 super cores and 20 performance cores - 64-core GPU - Neural Accelerators - Hardware-accelerated ray tracing - 32-core Neural Engine - 1.2TB/s memory bandwidth one point 2 terrabyte a second memory bandwidth.... 512gb coming later this year but at the cost of a small car
Sadly maybe 20 years ago $17K was price of a car. Median car price these days is close to $50K and there are hardly any new cars less than 20
Earlier quoted context omitted.
I feel like we are basically reliving the COVID era supply chain crunch, that it's basically inevitable that in 18-36 months more capacity will come online and prices will crash back to earth. Whether OEMs like Apple pass these savings along to consumers is tbd though.
Memory orders for the whole 2027 were already reserved. There is little indication that in 2028 situation will be any better unless something drastic happens like AI crash. And then the opposite can happen like an unknown drone with 100 kg warhead striking memory production facility.
Earlier quoted context omitted.
I know it's at least a little irrational, but I am indescribably angry with the pricing trajectory of personal computing. I'm not all that young and for pretty much all my life the computers you as a random person could buy have consistently become massively more performant and massively cheaper over time. Even when the exponential performance increases started to taper off, every few years you could buy or build a n…
I don't think it's an accident that the market is on a trajectory toward personal computer (and particularly storage) rental being the only economically viable game. There are significant players who will profit directly. Indirectly it helps companies for whom end users owning general purpose computers and storage threaten their business models. The industry has little incentive to fix this problem and as the market…