121–130 of 480 posts
Earlier quoted context omitted.
I’m not usually one to ask this because learning to do a thing can be fun, but why exactly have you spent 25 thousand dollars on getting an LLM someone else made to answer maths exam questions?
I’ve spent twice that on hosting movies and tv for Plex, so… I think they are worthy of my praise. What a healthy outlet for money.
Earlier quoted context omitted.
The cost is obviously not that big of factor for OP as it might be for others. It's actually refreshing to hear the candid viewpoint that he expresses here.
25k is definitely a lot but I did the risk analysis and I figured worst case I would lose a 1000-2000 after a year of playing around with it, so I look at it more like renting (I'm going to keep the Macbook Pro no matter what since I needed a new one).
Earlier quoted context omitted.
> Why didn't you take into account [...] the fact that a laptop can still hold a decent % of its resale value, and is useful for many other tasks than running an LLM? Because that wasn't what they claimed to research? >> for inference it's definitely not worth it. It's entirely fine if you enjoy local LLMs on your computer, there are people doing horribly inefficient inference on smartphones now. But for pure inferen…
Who is going to buy a $4299 M5 Max MBP with 64GB of RAM just to run Gemma 4 31b? Firstly you don't need 64GB for that model. Secondly if you want a machine that sits in the corner and does nothing but LLM inference, you don't buy a MacBook Pro, you buy some GPUs which are going to cost you a fraction of that (~$1k for ~64GB of VRAM is possible). The people buying Apple Silicon for inference general aim for the Mac St…
You don't? It for sure doesn't run on my 32 GB M2 MAX.
Earlier quoted context omitted.
I’ve spent twice that on hosting movies and tv for Plex, so… I think they are worthy of my praise. What a healthy outlet for money.
You spent 50k for plex hosting? Why so expensive?
In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…
This is a real problem and why I've just about given up on ebay or fb marketplace, esp for computers. If you are in Canada though sellit9.com is a great solution to having to deal with sketchy buyers.
Earlier quoted context omitted.
Half a petabyte of RAID6 is the biggest line item, then the redundant 40gb networking and compute follow closely. I have a lot… too much even?
That’s a lot of blurays…
Earlier quoted context omitted.
25k is definitely a lot but I did the risk analysis and I figured worst case I would lose a 1000-2000 after a year of playing around with it, so I look at it more like renting (I'm going to keep the Macbook Pro no matter what since I needed a new one).
Was this risk analysis just AI slop?
Earlier quoted context omitted.
How do you use the RTX 6000 with the Macs? Exo? I would think that would be pretty snappy if configured properly.
This is on a separate Windows PC, I don't have it integrated with the Macs.
Point being, once the M5 Ultra is available, I suspect a lot of people will get very serious about making Macs work with RTX GPUs because that will yield an inference platform with a good bang:buck ratio. If so, you may find that your existing hardware is more powerful than it seems today. And it may be a lot more expensive to replace later if you sell it now.