161–170 of 570 posts
> M6 supports up to 32GB of unified memory to multitask across demanding apps and run LLMs on device for secure and private agentic tasks. It also provides up to 170GB/s of unified memory bandwidth — a 10 percent increase over M5 and a 2.5x increase over M1. Isn't 170GB/s slow for bandwidth?
Earlier quoted context omitted.
It would be interesting to compare the costs of a top of the line machine every decade or so. Costs were steadily decreasing until recently.
John Dvorak said many, many decades ago (80s/90s) that the computer you want will always cost $3000. That statement has been more/less true for some time periods than others, but with some wiggle room I’ve found it to be accurate enough. Care to guess the approximate price of the MBP I bought earlier this year?
Earlier quoted context omitted.
If you like to experiment with training / finetuning / etc on LLMs, these are actually incredibly ‘cheap’. 1.2TB/s memory bandwidth unlocks a lot with 256GB unified, and agentic AI is pretty good at optimising performance. For comparison, to get 256GB with NVIDIA, you’re looking at a DIY workstation build (need pcie lanes), and like $70k? The spark’s ~250gb/s bandwidth doesn’t really count here.
Neither is the right alternative to compare to. You aren’t going to hit 100% utilization (if you are, ignore me, this doesn’t some to you, and write a blogpost for me to read and share). The comparison should be against renting in the cloud for the duration of your task for training and research or using pay-per-api-call providers for general inference instead of buying your own hardware (and paying the electricity a…
The maths to me was basically equivalent to prepaying for 242 days of runpod pricing for the same GPU; and I reckon I'd be able to get 6+ years of use out of this card with 96GB.
Plus there's the resell value -- it's actually appreciated by ~50% since I bought it.
Plus I do really enjoy that it's 100% local. I wouldn't feel comfortable giving my agents this much information if inference wasn't 100% local.
I wouldn't get another one, I wouldn't have as much value, but one is definitely paying off for me on the financial side.
Due to pricing insanity (not that Apple prices weren’t insane before the ram/ssd shortages) I’m not in the market but I do wonder if my next computer should be a Mac Studio instead of a MBP that lives its life docked. Might be better to just run a Studio and Neo for the very few times I actually need remote capabilities.
Most, if not all, of our current work happens on remote cloud vms. Now I'm stuck with carrying a 3KG monstrosity to work.Every day.
Absolutely no positives compared to my ThinkPad that weighed less than half in my previous job.
Remote development is "good enough" these days. With VS Code, Development containers etc... Having a light weight, portable laptop is so much nicer than a laptop that you can't even rest on your lap for long duration.
The only thing you need to be mindful of using light weight laptops is not having enough RAM to fit all your browser tabs..
Earlier quoted context omitted.
Having replaced my MacBook with a Mac Mini, I would reconsider. The MacBook is just such a _complete_ package. Great speakers, great keyboard, fantastic screen, the fingerprint sensor thingy.. Takes a lot of gear to match that
Not an option if you're trying to drive 3 decent screens.
Boy, oh boy Apple is the new shovel seller during AI gold rush. Most people at Apple have already realized that their processors are already too powerful for regular users - heck, as a developer my M2 Pro with 32 GB RAM is more than enough for me. Regular users don’t care about local AI either. So, they will probably extract as much money as possible during AI gold rush, but then we will most likely see Apple a. Maki…
They're eating nvidia's lunch.
Earlier quoted context omitted.
you can't link onchip memory.
True, but you can link them up over thunderbolt or Ethernet. If your goal is to run local LLMs, not all weights need to live on the same computer. You can segment the workload by layers and pass the activations along the lower bandwidth interconnect with a small perf penalty. Also you get double the CPU/GPU cores allowing for better multi user/agent performance.
It looks like speculation that Apple would raise the base chip’s maximum RAM from 32GB to 48GB was wrong. Apple also launched the base M6 today with a 32GB RAM limit, suggesting 512GB may remain the maximum for Ultra chips for some time. Since these Ultra chips combine 16 base chips: 32GB × 16 = 512GB
They probably literally don't have enough NAND to go around. 768GB of memory (48GB x 16) is enough for nearly 100 iPhone 17s; that's $80k (corrected) of iPhones at MSRP, although likely much lower margins than these high-RAM boxes.
Boy, oh boy Apple is the new shovel seller during AI gold rush. Most people at Apple have already realized that their processors are already too powerful for regular users - heck, as a developer my M2 Pro with 32 GB RAM is more than enough for me. Regular users don’t care about local AI either. So, they will probably extract as much money as possible during AI gold rush, but then we will most likely see Apple a. Maki…
I dunno, Apple's been throwing many bones to their customers who use their Macs for AI stuff; like Apple working with, and signing TinyGPU's NVIDIA eGPU drivers. Plus introducing features like RDMA over thunderbolt, which is critical for distributed inference/training/etc. On the software side, Apple is investing heaps. It's still ridiculous they don't support expandable NVMes, but the memory being soldered makes sen…
EDIT: AMD too, it's not limited to Nvidia, nice.
Due to pricing insanity (not that Apple prices weren’t insane before the ram/ssd shortages) I’m not in the market but I do wonder if my next computer should be a Mac Studio instead of a MBP that lives its life docked. Might be better to just run a Studio and Neo for the very few times I actually need remote capabilities.
At my current job, one of my biggest blunders was thinking "Let me just order the same hardware as most of my teammates to avoid unnecessary complications". Most, if not all, of our current work happens on remote cloud vms. Now I'm stuck with carrying a 3KG monstrosity to work.Every day. Absolutely no positives compared to my ThinkPad that weighed less than half in my previous job. Remote development is "good enough"…