291–300 of 571 posts
Earlier quoted context omitted.
There is no "future proof" for >1T param models, there is no present or future where you can run a model that size on consumer hardware.
I don't know that "consumer hardware" is a useful distinction anymore, it's just "what's your budget and what's your speed requirement".
- 120v input plug
- not rack-mounted
- has a video out port
Earlier quoted context omitted.
If your computer never leaves your desk, the iMac is pretty competitive with those features. Even comes with a TouchID keyboard.
I’ve had iMacs for almost 20 years. My last one was indeed my last. Without target display mode (use the Mac as a monitor), I’m ditching a perfectly good monitor. I was going to buy a Mac Studio and a good monitor to replace the iMac until the spouse reminded me that we are now retired and will spend time in a camper. So a MBP for me (and BenQ’s Mac-specific monitor), but others might do well to consider a Mac Mini/S…
Mac Studio with M5 Ultra starts at $5,499 - computing this expensive belongs in a server room serving multiple people not on a single person’s desk. (But still cool).
10 grand for 256GB memory. Likely double that for 512GB, but won't be available or finalized until October. Thunderbolt 5 is highest bandwidth external IO available at 120Gb/s. 1.2TB/s claimed max internal memory bandwidth. Not exactly "future proof" for >1T parameter models but good for targeting specific lower-parameter models, or if you can rely on pipeline parallelism and run a cluster.
> Not exactly "future proof" Computers are never "future proof".
Due to pricing insanity (not that Apple prices weren’t insane before the ram/ssd shortages) I’m not in the market but I do wonder if my next computer should be a Mac Studio instead of a MBP that lives its life docked. Might be better to just run a Studio and Neo for the very few times I actually need remote capabilities.
I've been thinking about this a fair bit recently. We make a lot of price/performance compromises for having an attached screen and keyboard on our computer. That was what got me started. Then I remembered the days of having to go to a special corner of the house to use a computer, vs now when I have a computer with me all the time. In my bag, on the sofa, on the train. Hell, I'm writing this on the work MBP while wa…
Never ended up happening. I didn't get much done at home at all and what I did get done was mostly on the couch; not a great environment for serious work. (I have an office where I do for-profit work so it's not an issue, but still, I like my side projects too)
Eventually I had enough decommissioned computer parts that I could assemble them and re-commission them into a working desktop. So I now have a desktop again. And I actually end up sitting at the desk working on stuff on the desktop in a way I rarely did with the laptop.
Earlier quoted context omitted.
I don't know that "consumer hardware" is a useful distinction anymore, it's just "what's your budget and what's your speed requirement".
Consumer hardware means: - 120v input plug - not rack-mounted - has a video out port
Earlier quoted context omitted.
So 3x more, plus the cost of a GPU (another 10k?). How is that value for money to get slightly better performance?
It can be more than "slightly", particularly if the model you're interested in (or will be interested in in 6 months) doesn't fit on the mac studio. You also need to account for eg storing 10TB of random checkpoints, load time when experimenting, and so on. When you start actually needing throughput these are all capability gaps in practical use, not just x% benchmark differences. If you just want to run Qwen 3.8 27B…
Earlier quoted context omitted.
Not an option if you're trying to drive 3 decent screens.
My M1 Pro drives 4 screens no problem. The trick is that you can only attach two through one Thunderbolt port, so you have to attach the 3rd one via the other port instead of your dock. (The 4th screen is the laptop's built in screen)
Earlier quoted context omitted.
3 of those thunderbolt 5 ports, so you can do a fully connected 4 machine cluster topology.
It's unclear to me how bandwidth scales with multiple connections. Many-to-many does not seem ideal. Daisy chaining would be fine for straight pipeline work. There doesn't seem to be an equivalent of a ethernet switch for thunderbolt 5 though.