Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was my $48K GPU server worth it?

rosmine.ai

Post reply on HN
Print view View original

71–80 of 480 posts

Re: Was my $48K GPU server worth it?

#71

Post by freediddy »

jmyeet wrote: post #55
freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

I looked into the M3 Ultra 512GB Mac Studio before it was discontinued and the as best as I could determine it just wasn't worth it... yet. The GFLOPS and memory bandwidth just arne't there even though it can hold a much larger model in memory. But the trend here is interesting. I think by 2030 you'll be able to buy fairly cheap hardware that is currently $10k+. I don't know what this does to the trillions invested i…

I agree that all these datacenter companies like Coreweave are investing billions in technology that has a very fast depreciation curve and I don't know how they will sustain income. The same goes for datacenters in space, what happens when those chips are obsolete? Will they sent astronauts to replace them or will they let them burn up and send new ones into orbit every year?

I feel that the open weight models pale in comparison to the frontier models, and I believe that if the gap closes quickly, that the open weight vendors will stop releasing it for free.

Re: Was my $48K GPU server worth it?

#72

Post by gwbas1c »

FYI: If you're in a similar situation, think very carefully before you build your own. The $17000 might sound like a lot; but when you take into account your time and risk tolerance, renting might be a much better solution.

Re: Was my $48K GPU server worth it?

#73

Post by cronin101 »

freediddy wrote: post #66
nickthegreek wrote: post #42

Earlier quoted context omitted.

The cost is obviously not that big of factor for OP as it might be for others. It's actually refreshing to hear the candid viewpoint that he expresses here.

25k is definitely a lot but I did the risk analysis and I figured worst case I would lose a 1000-2000 after a year of playing around with it, so I look at it more like renting (I'm going to keep the Macbook Pro no matter what since I needed a new one).

Nitpicking, but the worst case of spending $25k is unforeseen circumstances that write off the entire asset. I don’t think -$2000 is a conservative enough figure for standard depreciation either (a lot can happen in a year)

Re: Was my $48K GPU server worth it?

#74

Post by arjie »

freediddy wrote: post #68
arjie wrote: post #32

Earlier quoted context omitted.

All of these have appreciated in value. How much are you looking for the Ultra?

I've seen a lot of sales on eBay for over $20k, but I don't know if I believe it. Plus the lack of seller protection and the prevalence of scams on eBay make me too hesitant to actually want to risk it so I don't know what to do haha

Haha, yeah, it's about $23k or so. Should be twice the price what you bought it for if you got it last year. Tbh I don't know why. The RAM is large but the bandwidth and the compute isn't nearly enough. You can fit DeepSeek V3 on it quantized but inference is like 10 tok/s. Honestly, you'll be able to sell it locally for that in cash, and I would in your place.

I saw your heat comments about the RTX 6000 Pro as well. I bought a few of them recently and I'm running 2 of them in a 2U case in a colo. You need a lot of active airflow to keep them cool. Mine range from 23 C to 80 C.

Re: Was my $48K GPU server worth it?

#75

Post by freediddy »

wutwutwat wrote: post #65
freediddy wrote: post #16

Earlier quoted context omitted.

It's just a project I'm working on. I'm working on projects where AIs are processing and classifying large amounts of data that would be a lot of work for humans to do.

I think of LLMs as being well equipped for handling dynamic data or adapting to unforeseen circumstances well (random code requests, website's ever changing layouts, typos, non-standard formatting in docs, groking out important info, etc), but math problems are be definition a very specific set of instructions to run, so is the overhead and "thinking" aspect of a LLM/AI even needed here? I'm genuinely curious, btw, I…

> Can't these math problems just be yanked from some test file and rapid fired directly at a gpu/compute unit?

Yes this is exactly what I'm doing. I isolated the actual math question, and then sent it to my two servers to process and that's what's taking 10m+ to return. I'm asking them to solve the question and return the full answer along with their steps. I care about correctness so taking time is okay but I can't use 10m per solution.

Re: Was my $48K GPU server worth it?

#76

Post by shout5 »

doctorpangloss wrote: post #2

> Because of this I got a motherboard with slow GPU interconnect. It’s good for running many small experiments in parallel (which is my main use case) but horrible for any models split across gpus. :( you paid a professional pc builder and you weren't told this?

> paid a professional pc builder

They did not. That's a mining rig not a workstation. It's visible from the photo and the chart showing multiple failures over a short period of time including the risers -- which are visibly very low quality -- failing twice.

You have 50K, you call a real expert like Puget Systems or Digital Storm.

Re: Was my $48K GPU server worth it?

#77

Post by ryandrake »

bityard wrote: post #47
daemonologist wrote: post #26

Earlier quoted context omitted.

They have a subsequent post (from Monday) about what they've been working on: https://rosmine.ai/2026/05/18/fixing-llm-writing-with-distri... (I would assume they haven't made a lot of $ off of this, if nothing else because they've only just put out that post and demo. They do seem to have produced a model that doesn't sound very LLM-y to my ear, though it also seems rather weak for its size.)

Shallow take: They made an LLM that uses fewer emdashes. Cynical take: They made an LLM that can bypass existing AI slop detectors. Realistic take: They found a research problem they found interesting, dumped a bunch of capital and sweat equity into and (claimed to have, at least) found a solution. Neat!

Or they just have lots of money and a hobby. Someone else might blow $48K to get an old Cessna and go have fun flying around. Not everything needs to have a purpose.

Re: Was my $48K GPU server worth it?

#78

Post by forsalebypwner »

gwbas1c wrote: post #72

FYI: If you're in a similar situation, think very carefully before you build your own. The $17000 might sound like a lot; but when you take into account your time and risk tolerance, renting might be a much better solution.

I think their retrospective at the end of the article is grounded and logical:

"If I were to do this again, I wouldn’t do a custom build like this. I would buy a standard datacenter server and rent space in a colocation center"

I'm sure there are use cases when renting makes sense, but it can get crazy expensive really fast if you're not careful.

Re: Was my $48K GPU server worth it?

#79

Post by forsalebypwner »

pelasaco wrote: post #36

out of curiosity, did you check how much would cost to rent a cage in a colocation space? Having to power your computer from two different outlets sounds wild..

the very last line of the article:

"If I were to do this again, I wouldn’t do a custom build like this. I would buy a standard datacenter server and rent space in a colocation center. But then I would miss saying Hi to grumbl once in a while."

Re: Was my $48K GPU server worth it?

#80

Post by [deleted] »

ginko wrote: post #3
doctorpangloss wrote: post #2

> Because of this I got a motherboard with slow GPU interconnect. It’s good for running many small experiments in parallel (which is my main use case) but horrible for any models split across gpus. :( you paid a professional pc builder and you weren't told this?

Don't those Ada 6000 GPUs support NVLink? I think I can even see the cover for the connectors in OP's pic. edit: Hm, finding mixed information online on whether that's still supported or not. Apparently it was removed in workstation GPUs.

[deleted]
Post reply on HN