Here's me trying to justify this when I can run frontier models in the cloud for less than the monthly finance charge for this beast.
If you like to experiment with training / finetuning / etc on LLMs, these are actually incredibly ‘cheap’. 1.2TB/s memory bandwidth unlocks a lot with 256GB unified, and agentic AI is pretty good at optimising performance. For comparison, to get 256GB with NVIDIA, you’re looking at a DIY workstation build (need pcie lanes), and like $70k? The spark’s ~250gb/s bandwidth doesn’t really count here.
The comparison should be against renting in the cloud for the duration of your task for training and research or using pay-per-api-call providers for general inference instead of buying your own hardware (and paying the electricity and cooling bills on top), because let’s face it, the models you want to use are probably the same ones available on inference providers (but, yes, some are more trustworthy than others).
Speaking as someone that does ML/AI research, you are essentially paying a huge premium for being able to just run your Python script at any time without setting up a deployment script and harness to run the job remotely, while your hardware sits essentially idle the rest of the time.
The only way to make the math work is if you rent your hardware in the background for inference while you’re not using it in anger, but despite all the startups and promises that has never become as streamlined as mining bitcoins or shitcoins used to be and they don’t pay out as much as they say they would. Renting your hardware for training is another option but doing that is a lot more involved, options are fewer and farther in between, you won’t get as much utilization out of it, and doesn’t let you feasibly abort running tasks at a moment’s notice.