Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was my $48K GPU server worth it?

rosmine.ai

Post reply on HN
Print view View original

81–90 of 480 posts

Re: Was my $48K GPU server worth it?

#81

Post by LarsDu88 »

freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

Well if it makes you feel better those frontier LLMs are all technically taking a big loss, and they may all be in your shoes after a few years.

Re: Was my $48K GPU server worth it?

#82

Post by ryandrake »

mciancia wrote: post #4
ginko wrote: post #3

Earlier quoted context omitted.

Don't those Ada 6000 GPUs support NVLink? I think I can even see the cover for the connectors in OP's pic. edit: Hm, finding mixed information online on whether that's still supported or not. Apparently it was removed in workstation GPUs.

Nope, they don't support it. And afair even if they did, you would be limited to connecting only in pairs, not all 6 together

Honestly, I made the same mistake when I added a GPU to my (not $48K) existing homelab. I got a Ada 4000 for its slim form factor and low wattage, but realize after I bought it that it does not support NVLink, so I can't really effectively double it up later if I wanted to. Live and learn. I suppose you might research that a little before blowing that much money though LOL :)

Re: Was my $48K GPU server worth it?

#83

Post by ericd »

freediddy wrote: post #56
bethekind wrote: post #37

Earlier quoted context omitted.

Which of these has been the most productive for you? Sounds like you've enjoyed the RTX6000 the most?

RTX 6000 is some-what obviously my fastest card but my biggest problem with the RT 6000 is the immense heat. The GPU itself is almost 200F and the exhaust from the fans itself is over 150F. I'm worried that my hard drives are going to fail. I was told that the GDDR7 is even hotter than the GPU which is surprising to me. After my last run, I'm going to wait for the new case I ordered to come in and cannibalize my kid'…

I take it this wasn't the half-wattage Max Q version with blower fan?

Re: Was my $48K GPU server worth it?

#84

Post by forsalebypwner »

bityard wrote: post #47
daemonologist wrote: post #26

Earlier quoted context omitted.

They have a subsequent post (from Monday) about what they've been working on: https://rosmine.ai/2026/05/18/fixing-llm-writing-with-distri... (I would assume they haven't made a lot of $ off of this, if nothing else because they've only just put out that post and demo. They do seem to have produced a model that doesn't sound very LLM-y to my ear, though it also seems rather weak for its size.)

Shallow take: They made an LLM that uses fewer emdashes. Cynical take: They made an LLM that can bypass existing AI slop detectors. Realistic take: They found a research problem they found interesting, dumped a bunch of capital and sweat equity into and (claimed to have, at least) found a solution. Neat!

You were on the money with the Cynical take lol:

https://rosmine.ai/2026/05/18/fixing-llm-writing-with-distri...

Re: Was my $48K GPU server worth it?

#85

Post by freediddy »

iooi wrote: post #58
freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

I'll buy your macbook if you're trying to get rid of it!

I'm keeping that one for sure, I love it!

Re: Was my $48K GPU server worth it?

#87

Post by pelasaco »

forsalebypwner wrote: post #79
pelasaco wrote: post #36

out of curiosity, did you check how much would cost to rent a cage in a colocation space? Having to power your computer from two different outlets sounds wild..

the very last line of the article: "If I were to do this again, I wouldn’t do a custom build like this. I would buy a standard datacenter server and rent space in a colocation center. But then I would miss saying Hi to grumbl once in a while."

Yes, i mean, he could rent a cage and run grumbl it there. It doesn't have to be a standard datacenter server, even though a standard datacenter server would be better and cheaper.

Re: Was my $48K GPU server worth it?

#89

Post by jachee »

freediddy wrote: post #75
wutwutwat wrote: post #65

Earlier quoted context omitted.

I think of LLMs as being well equipped for handling dynamic data or adapting to unforeseen circumstances well (random code requests, website's ever changing layouts, typos, non-standard formatting in docs, groking out important info, etc), but math problems are be definition a very specific set of instructions to run, so is the overhead and "thinking" aspect of a LLM/AI even needed here? I'm genuinely curious, btw, I…

> Can't these math problems just be yanked from some test file and rapid fired directly at a gpu/compute unit? Yes this is exactly what I'm doing. I isolated the actual math question, and then sent it to my two servers to process and that's what's taking 10m+ to return. I'm asking them to solve the question and return the full answer along with their steps. I care about correctness so taking time is okay but I can't…

Nono, parent was asking “They’re bad and inefficient at that, so why have an LLM do math? Why not just use some code and the CPU/GPU that’s already good and efficient at basic math?”

Re: Was my $48K GPU server worth it?

#90

Post by ahmadyan »

freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

If you are in the bay area, i'm happy to buy that M3 Ultra from you, i've been unsuccessfully looking for one and can't find any.
Post reply on HN