161–170 of 480 posts
I know it's not the same. But a lot of people buy expensive GPUs, just to find out they have no real use for smaller models.
Earlier quoted context omitted.
Or they just have lots of money and a hobby. Someone else might blow $48K to get an old Cessna and go have fun flying around. Not everything needs to have a purpose.
A self-described "broke grad student [who has] been saving up for this for years." Risking their own money and time instead of leveraging a PowerPoint to hire other peoples' labor with other peoples' money. I can respect that.
In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…
I’m not usually one to ask this because learning to do a thing can be fun, but why exactly have you spent 25 thousand dollars on getting an LLM someone else made to answer maths exam questions?
It's not financially a good idea: renting really does beat owning, and cloud beats both if you're only running inference on these machines. But I'm not just doing inference, and as a thing I can do silly stuff on to learn, it's hard to beat!
I did the math at least on a Macbook pro, and for inference it's definitely not worth it. - https://www.williamangel.net/blog/2026/05/17/offline-llm-ene... - Discussion: https://news.ycombinator.com/item?id=48168198
Earlier quoted context omitted.
I’m not usually one to ask this because learning to do a thing can be fun, but why exactly have you spent 25 thousand dollars on getting an LLM someone else made to answer maths exam questions?
Privacy and offline operation are valuable or non-negotiable in some cases, but the difference is pretty categorical between what can run on a single card and what can run on a DGX GB200 NVL72 cabinet. Doesn't mean it's not worth seeing how far local models can be pushed. Not every problem needs a senior engineer.
A better way of putting it is that you can run plenty of things on a single ordinary system, but you may be disappointed at the performance. Generally, you can't expect inference to be as quick as with cloud for SOTA-like models. You have to run smaller models for quick replies, and large models with a lot of real-world knowledge for less time-critical inference, possibly batching many requests simultaneously to improve throughput.
In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…
It's going to be a non-trivial haircut. This stuff depreciates pretty fast.
Earlier quoted context omitted.
Nitpicking, but the worst case of spending $25k is unforeseen circumstances that write off the entire asset. I don’t think -$2000 is a conservative enough figure for standard depreciation either (a lot can happen in a year)
Either I don't understand the used apple market.. or I agree this is crazy. Someone spends $25k on new hardware, waits a year, and expects to sell it for $23k? Unless the ram issues save him, and cost of new goes up, I don't see how that was going to work.
Earlier quoted context omitted.
I found them dumped out in the “street” in a place ordained by law as public domain. So I just grabbed up the media and use it in private.
How many Blurays are we talking about?
if some have more than one layer it could fewer but that's the order of magnitude
In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…
If you're in a decent sized city, you should be able to find a local buyer on Craigslist or FB Marketplace... Beyond that, for higher value, smaller items like your M3 Ultra, I would talk to your local police department and/or library to see if you can do the exchange there. Larger libraries usually have a police officer on site or nearby, and the PD office near you may also provide a "safe" exchange location... I'd…
I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…