181–190 of 322 posts
Earlier quoted context omitted.
But once all that is done you still own a Mac in one case, and you don’t in the other, correct?
Yea this; it’s the same reason why mortgaging is cheaper than renting
Earlier quoted context omitted.
They're responding to the people doing things like buying the most expensive Mac they can find specifically to do local inference for their AI agents. Some do it to have control over their ability to use AI. Some do it because they think it will be cheaper to not have to pay a SaaS to generate tokens for them. But for those interested in the latter case, it seems like it's not actually cheaper after all, at least at…
It's worth paying a premium for the privacy (assuming that llama.cpp and ollama aren't sending my sessions back to the cloud regardless...), and for the concerns about not getting a surprise bill.
Earlier quoted context omitted.
Yeah. The speed is the biggest issue. The intelligence of open models is good enough for serious work (though still worse than the frontier models), but the cloud models are often 3-7 times faster, and you can get more parallelization and so get speeds on the order of hundreds of tokens per second, which makes things fast!
Even extremely slow LLMs can generate Part B faster than I can audit Part A. So the LLM can generate Part A while I look over my email. Then it can worry over Part B while I look over Part A. It can worry over Part C while I have my 10:30 group meet. And it can worry over Part D while I do whatever other silly, time-wasting thing all humans do in almost all organizations. Then I still haven't reviewed Part B, yet, so…
Sure, I can wait hours for my local model to finish, or I can spend basically as much and get the answer right away
There’s a lot of exciting stuff with local LLMs despite the speed, but for me I don’t have the discipline and working memory to jump from project to project.
I'm even surprised people ignorantly talking about advantages of buying very expensive device , run it only sometimes and aiming to beat cloud vendors. If small model is great it will be hosted with good electricity cost and will be utilized 24/7. Isn't it 2+2 of economics ? CPU is a commodity, and we are still buying cpu and ram from vendors for same reason
Unless I'm misunderstanding, this is counting the entire laptop in the cost of generating tokens. The calculation seems to omit that, in addition to receiving LLM output, you have also received a laptop in exchange for your money. If you intend to put this machine in a dark corner and run it solely as a token-munching server, a laptop would be an exceptionally poor choice of technology for this purpose. But if you in…
An unreasonable number of these people spent $10,000+ for Mac Studios that are still compute bottlenecked and don't have anything more efficient than Gemma 4 to run.
Mmmm, nope if you do the smart thing. MacBook M5 max 128gb is a premium laptop at 6k, but with it you can do many things and is your good main driver for the day. Then, it can also run DeepSeek V4 flash and perform non trivial tasks locally, without censorship or limitations, even without an internet connection and on very privacy sensitive data. That's a good deal. If you buy 25k for a dual Mac Studio 512gb to aband…
Another open secret is that that certain companies give you tens of thousands of tokens freely, with pretty respectable models such as Gemini 3.1 and GLM 4.6.
Unless I'm misunderstanding, this is counting the entire laptop in the cost of generating tokens. The calculation seems to omit that, in addition to receiving LLM output, you have also received a laptop in exchange for your money. If you intend to put this machine in a dark corner and run it solely as a token-munching server, a laptop would be an exceptionally poor choice of technology for this purpose. But if you in…
> in addition to receiving LLM output, you have also received a laptop in exchange for your money And, since it's a Mac, whenever you're ready to upgrade it'll still have a fairly decent resale value.
https://www.ebay.com/sch/i.html?_nkw=apple+mac+studio+m3+ult...
I think you'd need to tinker quickly, realize anything with CUDA (other than the awful DGX Spark) is better for learning, the prefill is killing your ability to actually run models large enough to justify that RAM, and then cure yourself before the rest of the crowd.
Earlier quoted context omitted.
Yea this; it’s the same reason why mortgaging is cheaper than renting
This is far from a universal truth: https://www.nytimes.com/interactive/2024/upshot/buy-rent-cal... Real estate is only a clearly good investment if you ignore opportunity cost.
Earlier quoted context omitted.
This is far from a universal truth: https://www.nytimes.com/interactive/2024/upshot/buy-rent-cal... Real estate is only a clearly good investment if you ignore opportunity cost.
Articles like that still miss a bit of the nuance. Imagine having your house paid for, and you grow old and you have no rent to pay. Yes, you could have invested but likely you would have spent some of that money on something else, or your investments might have not worked out so well, or any other reason. Human reasons, to be specific. Owning property is like a lock.
Earlier quoted context omitted.
You also need to pay close attention to rent vs purchase ratios. A lot of cities are cheap to rent but expensive to buy (eg beijing 10 years ago).
Key word being „ago“.