Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was my $48K GPU server worth it?

rosmine.ai

Post reply on HN
Print view View original

181–190 of 480 posts

Re: Was my $48K GPU server worth it?

#181

Post by throwatdem12311 »

dekhn wrote: post #131
throwatdem12311 wrote: post #112

Earlier quoted context omitted.

If you want a GPU that has comparable performance on Linux to Windows- you want AMD. NVIDIA drivers are notoriously bad. Many of my games run better on Linux with the open source AMd drivers. (CachyOS rolling rolling rolling).

I have no interest in moving to AMD for video cards right now- the network effect of NVIDIA is just too high, and their peak performance is insane. I also haven't noticed any major issues with nvidia drivers, unless you mean specifically running Windows games on Linux machines with nvidia cards, where I have zero experience.

Network effect for graphics cards? Literally what? Your friends don’t care what GPU you run my guy and there is not much benefit of having brand loyalty to a company like Nvidia that gives absolutely zero fucks about people that aren’t their enterprise customers buying GPUs by the thousands. If there’s any “network effect” for gaming GPUs on Linux it’s in favor of AMD because of the immense amount of work Valve has been putting in to make it work well for their steam* hardware.

Nvidia’s drivers are trash for gaming on Linux and the majority of your “compatibility and framerate issues” are because you’re using a sub-par product for the job.

Re: Was my $48K GPU server worth it?

#183

Post by dheera »

ttshaw1 wrote: post #110
mrandish wrote: post #108

Earlier quoted context omitted.

I guess it was supposed to be a humorous aside, but it wasn't actually helpful because the relevant issue is when you pull more total amps from a single circuit than it's fused for (usually 15 or 20 amps in U.S. residences). The failure mode is usually tripping the circuit breaker. That issue can often be addressed fairly easily by splitting the power draw between two adjacent circuits. You can have an electrician do…

I think the relevant issue is you could conceivably have a house with two outlets with opposite phases. Bussing them together in the PSU will then create a short

> two outlets with opposite phases

This is actually THE standard in the US, which is actually fundamentally a 240V power grid but with an electrode stuck halfway down the secondary winding on every pole transformer, which becomes your "neutral". The two ends become L1 and L2, so that L1-N is 120Vrms, L2-N is 120Vrms, and L1-L2 is 240Vrms, and this is what goes into every home.

The power outlets connected to L1 are all opposite phase to all the ones connected to L2.

Rather than bussing the two outlets together, what you can safely do is get an electrician to just wire up an outlet with L1 and L2 and voila you have a 240V outlet. This is how you get all your dryer outlets, EV charging outlets, electric stove outlets, etc.

Re: Was my $48K GPU server worth it?

#184

Post by michaelt »

onlyrealcuzzo wrote: post #166
freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

> I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. It's going to be a non-trivial haircut. This stuff depreciates pretty fast.

Bizarrely, I brought a GPU new in Jun 2024, and there are sold ebay listings saying the used GPU is worth 4% more today.

Of course, this is an unusual state of affairs; I see my GPU purchase as consumption, not investment.

Re: Was my $48K GPU server worth it?

#185

Post by manyatoms »

BLKNSLVR wrote: post #176
cronin101 wrote: post #73

Earlier quoted context omitted.

Nitpicking, but the worst case of spending $25k is unforeseen circumstances that write off the entire asset. I don’t think -$2000 is a conservative enough figure for standard depreciation either (a lot can happen in a year)

Aren't things like this seeing 'negative depreciation' these days?

"inflation"

Re: Was my $48K GPU server worth it?

#186

Post by conradkay »

jcelerier wrote: post #159
throwawaytea wrote: post #116

Earlier quoted context omitted.

Either I don't understand the used apple market.. or I agree this is crazy. Someone spends $25k on new hardware, waits a year, and expects to sell it for $23k? Unless the ram issues save him, and cost of new goes up, I don't see how that was going to work.

You could almost sell a RTX 3090 for more today than what it cost brand new when it came out six years ago

It's still very contrarian to expect GPUs won't depreciate rapidly. Yes 3090s were a good investment then, but way worse than just buying Nvidia stock directly

Re: Was my $48K GPU server worth it?

#187

Post by dheera »

pelasaco wrote: post #36

out of curiosity, did you check how much would cost to rent a cage in a colocation space? Having to power your computer from two different outlets sounds wild..

The cheaper, easier solution would probably be just to get an electrician to wire up a high amperage 240V outlet just like your electric stove or dryer has, and then get a PSU that connects to that.

Would probably cost you $500-1000 depending on how difficult your home is.

Re: Was my $48K GPU server worth it?

#188

Post by digitaltrees »

freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

I have three m3 512gb units and want a fourth to run an exo set up. Like you, I am worried about scammers. Let’s discuss if you still want to sell.

https://calendly.com/ryanwmartin/open-office-hours

Re: Was my $48K GPU server worth it?

#189

Post by CobaltFire »

kgeist wrote: post #150

I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…

I have a 5090 machine sitting idle that I'm considering turning into a machine for my own small team (3 devs).

Are you willing to share any lessons learned, etc. that I could make use of? We are evaluating paying for a SOTA sub or trying this, and the talk about Qwen3.6-27B makes me want to try deploying this machine.

Re: Was my $48K GPU server worth it?

#190

Post by CamperBob2 »

skiing_crawling wrote: post #148
freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

I got an RTX 6000 pro too. I like running locally, I've learned a lot more than if I had used an API and there's less worry about overspending tokens. I accidentally spent $100 on claude api in like 2 days because I didn't know what I was doing. The problem is that while one these gpus is a huge improvement over a laptop or a single 3090, you very quickly wish you had more. I would buy a second one, but I did the mat…

A pair of RTX6000 cards will give you a good performance boost due to tensor parallelism, though. I haven't tried the newest predictive quants but I see about 35 tps when running the 8-bit Qwen 3.6 27B model on one board and about 50 tps on two. Probably could come close to 100 tps on an optimized setup with the latest GGUFs.

Also, the 4-bit quants of MiniMax 2.7 will run at 100 tps or so with two cards, which is pretty decent. It doesn't go any faster at all with 4 GPUs from what I've seen, so if you don't actively need 384 GB of VRAM, 2x RTX6000 is a good place to be.

Post reply on HN