Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was my $48K GPU server worth it?

rosmine.ai

Post reply on HN
Print view View original

381–390 of 480 posts

Re: Was my $48K GPU server worth it?

#381

Post by ChoGGi »

Silagi wrote: post #371
ChoGGi wrote: post #240

Earlier quoted context omitted.

> Too bad I don't live in a cold climate -- that 60c heat output would come in handy :) Used to overclock back in the day during winter with an intake duct rigged to suck in outside air, best thing about -30c :)

I've always thought about doing something like this in the Midwest US, but was always a bit nervous about condensation damaging the components over time; did you run with that sort of setup consistently, or only when pushing high scores? Ever run into issues with components failing?

That was 25 odd years ago, less sensitive hardware and cheaper... Nothing that failed though, did have some sketchy moments with condensation yeah :)

Not consistently, I did start using petroleum jelly till I upgraded and found out that wasn't very fun to clean up.

Re: Was my $48K GPU server worth it?

#382

Post by mbesto »

ai_fry_ur_brain wrote: post #308
tempay wrote: post #306

Earlier quoted context omitted.

Are they? I only ever see unsubstantiated claims for this whereas I see many justifications that interference is comfortably profitable in isolation.

Its basic math, go calculate max sessions for a certain tps on any hardware. Session# * tps * 86400 (secs in a day) * 30 days. You'll realize real quick its not profitible. You cant just say things you don't like to hear are unsubstantiated without verifying. Not to mention, subscriptions.. $2mm in GPUs being given out for 5 hrs a day at a cost of $200 a month. I could easily say that everyone who says its profitible…

> I could easily say that everyone who says its profitible is msking unsubstantiated claims lol.

And people who don't understand the difference between capex and opex are making uneducated claims. It's not basic math.

Running an inference data center is a mix of variable and fixed costs. The fixed costs are currently in the billions of billions of dollars for pretty much any investment in this space. Many of those fixed costs have (currently) unknown refresh cycles. So, unless you have access to the financial books of these companies it's currently just speculation whether inference is profitable.

Re: Was my $48K GPU server worth it?

#383

Post by simplyluke »

swiftcoder wrote: post #327
iwontberude wrote: post #125

Earlier quoted context omitted.

Half a petabyte of RAID6 is the biggest line item, then the redundant 40gb networking and compute follow closely. I have a lot… too much even?

does one really need 40gb networking to stream bluerays?

I am (clearly) not as far down the rabbithole as the commenter you're replying to, but almost certainly not. Streaming 4k blueray is on the order or ~100Mb/s, which means on a LAN bog-standard gigabit ethernet and associated networking hardware would be more than sufficient.

This is taking a hobby to its extremes, in much the same way that a $5k boat and $500k boat let you catch the same fish.

Re: Was my $48K GPU server worth it?

#385

Post by segmondy »

senectus1 wrote: post #218

You guys are nuts... I hope you're making enough money to justify this level of investment and power use (not to mention noise and heat management) in your home ... I'm just putting a 2nd hand 12gb 3060 into my lab box, but its only for use with HA/Paperless/Plex etc type things. I dont need multi-model agentic behavior for private use. If I did I reckon I'd renting infrastructure rather than filling my home with tha…

There's a reason folks like these can afford this and you can't. Go on with your cheap self

Re: Was my $48K GPU server worth it?

#386

Post by globnomulous »

cronin101 wrote: post #73
freediddy wrote: post #66

Earlier quoted context omitted.

25k is definitely a lot but I did the risk analysis and I figured worst case I would lose a 1000-2000 after a year of playing around with it, so I look at it more like renting (I'm going to keep the Macbook Pro no matter what since I needed a new one).

Nitpicking, but the worst case of spending $25k is unforeseen circumstances that write off the entire asset. I don’t think -$2000 is a conservative enough figure for standard depreciation either (a lot can happen in a year)

I wouldn't call this nitpicking. This is how people who are careful with money think. I learned embarrassingly late to stop justifying purchases by making predictions about future returns. I treat everything as having zero value as soon as I purchase it. Thinking otherwise is, for me, always a dangerous rationalization -- always a craving that's trying to outmaneuver sense.

Re: Was my $48K GPU server worth it?

#387

Post by jrmg »

freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

If you run it in the winter the electricity is “free” because it’s replacing a portion of whatever else heats your house.

Re: Was my $48K GPU server worth it?

#388

Post by Radle »

"If I were to do this again, I wouldn’t do a custom build like this. I would buy a standard datacenter server and rent space in a colocation center. But then I would miss saying Hi to grumbl once in a while."

Re: Was my $48K GPU server worth it?

#389

Post by kgeist »

nicman23 wrote: post #360
kgeist wrote: post #150

I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…

> 260k context with a single 5090?

Yep, Gated DeltaNet in Qwen3.6 requires much less VRAM for the KV cache than previous generations. Plus the KV cache is 8-bit.
Post reply on HN