Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was my $48K GPU server worth it?

rosmine.ai

Post reply on HN
Print view View original

171–180 of 480 posts

Re: Was my $48K GPU server worth it?

#171

Post by jaxn »

cronin101 wrote: post #73
freediddy wrote: post #66

Earlier quoted context omitted.

25k is definitely a lot but I did the risk analysis and I figured worst case I would lose a 1000-2000 after a year of playing around with it, so I look at it more like renting (I'm going to keep the Macbook Pro no matter what since I needed a new one).

Nitpicking, but the worst case of spending $25k is unforeseen circumstances that write off the entire asset. I don’t think -$2000 is a conservative enough figure for standard depreciation either (a lot can happen in a year)

I assume he is calculating the loss as depreciation - what they would have spent on cloud bills if they hadn’t been doing this locally.

Re: Was my $48K GPU server worth it?

#172

Post by thefounder »

ttoinou wrote: post #149
mountainriver wrote: post #115

Earlier quoted context omitted.

Running LLMs on Macs is still terribly slow. They simply lack the optimizations other platforms have. An RTX 6000 pro Blackwell is a pretty good card

M5 pro 48GB should be good and future proof

If you buy Mac get at least 256GB ram otherwise just buy a bunch of nvidia cards. It really does not make sense otherwise if you are looking for performance / $. The mac (studio) is unique as it has more ram than the alternatives(I.e consumer nvidia cards or spark stuff) so it can fit bigger models but otherwise its performance is worse.

Re: Was my $48K GPU server worth it?

#173

Post by LtdJorge »

marai2 wrote: post #151
iwontberude wrote: post #128

Earlier quoted context omitted.

I found them dumped out in the “street” in a place ordained by law as public domain. So I just grabbed up the media and use it in private.

How many Blurays are we talking about?

Can't reply to the other poster, but I have 4K HDR Blu-ray copies from discs I found in the street too, which are more in the 60GB ballpark.

Re: Was my $48K GPU server worth it?

#174

Post by tracker1 »

thefounder wrote: post #169
tracker1 wrote: post #147

Earlier quoted context omitted.

If you're in a decent sized city, you should be able to find a local buyer on Craigslist or FB Marketplace... Beyond that, for higher value, smaller items like your M3 Ultra, I would talk to your local police department and/or library to see if you can do the exchange there. Larger libraries usually have a police officer on site or nearby, and the PD office near you may also provide a "safe" exchange location... I'd…

Something is very wrong in some countries if you have to get police protection to sell a f* computer. I get it’s on the expensive side but still….

You don't have to... but it's a matter of a safe location for both parties. If it was more expensive, I'd probably work through a broker (like a car or house).

The buyer doesn't know who the seller is, and vice-versa... the level of trust you can bear depends on how much you're willing to lose. My advice is only in that there are safe venues you can use to make such an exchange.

Re: Was my $48K GPU server worth it?

#175

Post by zozbot234 »

joefourier wrote: post #60
bigyabai wrote: post #48

Earlier quoted context omitted.

> Why didn't you take into account [...] the fact that a laptop can still hold a decent % of its resale value, and is useful for many other tasks than running an LLM? Because that wasn't what they claimed to research? >> for inference it's definitely not worth it. It's entirely fine if you enjoy local LLMs on your computer, there are people doing horribly inefficient inference on smartphones now. But for pure inferen…

Who is going to buy a $4299 M5 Max MBP with 64GB of RAM just to run Gemma 4 31b? Firstly you don't need 64GB for that model. Secondly if you want a machine that sits in the corner and does nothing but LLM inference, you don't buy a MacBook Pro, you buy some GPUs which are going to cost you a fraction of that (~$1k for ~64GB of VRAM is possible). The people buying Apple Silicon for inference general aim for the Mac St…

> Firstly you don't need 64GB for that model.

You might need that to run it with a longer context, KV cache size is a known issue with that model series.

Re: Was my $48K GPU server worth it?

#176

Post by BLKNSLVR »

cronin101 wrote: post #73
freediddy wrote: post #66

Earlier quoted context omitted.

25k is definitely a lot but I did the risk analysis and I figured worst case I would lose a 1000-2000 after a year of playing around with it, so I look at it more like renting (I'm going to keep the Macbook Pro no matter what since I needed a new one).

Nitpicking, but the worst case of spending $25k is unforeseen circumstances that write off the entire asset. I don’t think -$2000 is a conservative enough figure for standard depreciation either (a lot can happen in a year)

Aren't things like this seeing 'negative depreciation' these days?

Re: Was my $48K GPU server worth it?

#177

Post by m463 »

dylan604 wrote: post #140
m463 wrote: post #100

Other things people spend "too much money" on: - muscle cars, with all the stuff, driven occasionally. - boats, that don't get taken out much - gamer x, where x=system or laptop or keyboard or mouse or desk or glasses or mousepad or speakers or ... usually with "> too much RGB" - children $48k for something constructive even if ai related? no problem, refreshing even.

One of these things is not like the others. If you don't spend the money on one of them, you can get a visit from government officials that might decide to take that "item" from you. You'd also be a worthless human to spend that money on the other 3 while not on the one.

I (probably not obviously) meant the "too much" part, where kids fail to grow/launch when (optional) things are given to them too easily.

I didn't mean food, shelter, medical, education, lego, others-where-required-by-law.

Re: Was my $48K GPU server worth it?

#178

Post by de6u99er »

freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

You definitely want to get rid of your M3 Ultra before the M5 Ultra get officially announced.

Re: Was my $48K GPU server worth it?

#179

Post by throwatdem12311 »

echelon wrote: post #99
throwatdem12311 wrote: post #51

Earlier quoted context omitted.

Not even a single mention of gaming. No wonder gamers hate AI bros.

> No wonder gamers hate AI bros. Personally, playing with AI models is way more fun than getting sucked into a game loop. Game loops feel like busy work hooked to an engineered dopamine drip. AI models are new frontiers and are exciting to build with, modify, lobotomize, and hack around with.

And some of us are doing AI stuff all freaking day at day job and just want to play some Tekken when we get home for 30 minutes after the kid is in bed. But now Playstations are 1000$ and Ram and GPU prices are astronomical.

Not everyone is hustling 24/7 like some kind of lunatic.

Re: Was my $48K GPU server worth it?

#180

Post by BLKNSLVR »

dylan604 wrote: post #140
m463 wrote: post #100

Other things people spend "too much money" on: - muscle cars, with all the stuff, driven occasionally. - boats, that don't get taken out much - gamer x, where x=system or laptop or keyboard or mouse or desk or glasses or mousepad or speakers or ... usually with "> too much RGB" - children $48k for something constructive even if ai related? no problem, refreshing even.

One of these things is not like the others. If you don't spend the money on one of them, you can get a visit from government officials that might decide to take that "item" from you. You'd also be a worthless human to spend that money on the other 3 while not on the one.

I have profited richly from my children, but not in a monetary sense.

However much it has cost me monetarily, it has repaid itself ten times over in value to my very soul.

Post reply on HN