601–610 of 1001 posts
Earlier quoted context omitted.
Every single AI shop on the planet is trying to figure out if there is enough compute or not to make this a reasonable AI path. If the answer is yes, that 10k is a absolute bargain.
No AI shop is buying macs to use as a server. Apple should really release some server macOS distribution, maybe even rackable M-series chips. I believe they have one internally.
Two questions for the fellow HNers: 1. What are various average joe (as opposed to researchers, etc.) use cases for running powerful AI models locally vs. just using cloud AI. Privacy of course is a benefit, but it by itself may not justify upgrades for an average user. Or are we expecting that new innovation will lead to much more proliferation of AI and use cases that will make running locally more feasible? 2. Wit…
I don't think there's a huge use-case locally, if you're happy with the subscription cost and privacy. That is, yet. Give it maybe 2 years and someone will probably invent something which local inference would seriously benefit from. I'm anticipating inference for the home appliances (something mac mini form factor that plugs into your router) but that's based on what would make logical sense for consumers, not what…
>> Apple seems to be using LPDDR, but HBM will also likely be a key tech. SK Hynix and Samsung are the most reputable for both.
So not much Micron? Any US based stocks to invest in? :-)
Earlier quoted context omitted.
That will only have 128GB of unified memory
128GB for 3K; per the announcement their ConnectX networking allows two Project Digits devices to be plugged into eachother and work together as one device giving you 256GB for $6k, and, AFAIK, existing frameworks can split models across devices, as well, hence, presumably, the upthread suggestion that Project Digits would provide 512GB for $12k, though arguably the last step is cheating.
Earlier quoted context omitted.
I genuinely forgot the Mac Pro still exists. It’s been so long since I even saw one. And I’ve had every previous Mac tower design since 1999: G4, G5, the excellent dual Xeon, the horrible black trash can… But Apple Silicon delivers so much punch in the Studio form factor, the old school Pro has become very niche. Edit - looks like the new M3 Ultra is only available in Mac Studio anyway? So the existence of the Pro is…
never understood the hate on the trash can. Isn't the mac studio basically the same idea as the trash can but even less upgradeable?
The 2013 Mac Pro was stuck forever with its original choice of Intel CPU and AMD GPU. And it was unfortunately prone to overheating due to these same components.
Earlier quoted context omitted.
They didn't increase the memory bandwidth. You can get the same memory bandwidth, which is available on the M2 Studio. Yes, yes, of course you can get 512 gigabytes of uRAM for 10 grand. The the question is if a llm will run with usable performance at that scale? The point is there's diminishing returns despite having enough uRAM with the same amount of memory bandwidth even with increased processing speed of the new…
> The question is if a llm will run with usable performance at that scale? For the self-attention mechanism, memory bandwidth requirements scale ~quadratically with the sequence length.
Earlier quoted context omitted.
>> ... around tools that I do not own. That just may be dependent on how much trust you have on the providers you use. Or do you do your own electricity generation?
That's quite a reductio ad absurdum. No, I don't generate my own electricity (though I could). But I don't use tools for work that can change out from under me at any moment, or that can increase 10x in price on a corporate whim.
Earlier quoted context omitted.
>> ... around tools that I do not own. That just may be dependent on how much trust you have on the providers you use. Or do you do your own electricity generation?
That's quite a reductio ad absurdum. No, I don't generate my own electricity (though I could). But I don't use tools for work that can change out from under me at any moment, or that can increase 10x in price on a corporate whim.
I tend to do the same thing. I do not consider myself as a good representative of an average user though.
Earlier quoted context omitted.
That's quite a reductio ad absurdum. No, I don't generate my own electricity (though I could). But I don't use tools for work that can change out from under me at any moment, or that can increase 10x in price on a corporate whim.
And why would that require running AI models locally? You can be in essentially full control by using open source (/open weight) models (DeepSeek etc.) running on exchangable cloud providers that are as replaceable as your electricity provider.
Earlier quoted context omitted.
It's fast enough for me to cancel monthly AI services on a mac mini m4 max.
Hm, the AI services over 5 years cost half of m4 max minimal configuration which can barely run severely lobotomized LLaMA 70B. And they provide significantly better models.
Earlier quoted context omitted.
It's fast enough for me to cancel monthly AI services on a mac mini m4 max.
Smaller, dumber models are faster than bigger, slower ones. What model do you find fast enough and smart enough?