Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was my $48K GPU server worth it?

rosmine.ai

Post reply on HN
Print view View original

371–380 of 480 posts

Re: Was my $48K GPU server worth it?

#371

Post by Silagi »

ChoGGi wrote: post #240
theYipster wrote: post #210

Earlier quoted context omitted.

That's about what my OC'd and watercooled 4090 runs at. The cards are designed for it. Only problem I have is when sitting next to the computer under load -- I either have to open windows or blast the AC. Too bad I don't live in a cold climate -- that 60c heat output would come in handy :)

> Too bad I don't live in a cold climate -- that 60c heat output would come in handy :) Used to overclock back in the day during winter with an intake duct rigged to suck in outside air, best thing about -30c :)

I've always thought about doing something like this in the Midwest US, but was always a bit nervous about condensation damaging the components over time; did you run with that sort of setup consistently, or only when pushing high scores? Ever run into issues with components failing?

Re: Was my $48K GPU server worth it?

#372

Post by rithdmc »

gbgarbeb wrote: post #311
digitaltrees wrote: post #273

Earlier quoted context omitted.

oh young grasshopper, I see you dont know that money launderers love the ebay hype cycle. Its REALLY common on high dollar hot items to have phantom transactions where parties are on both sides of the transaction to clean illicit money. The high price tag and high volume amount of transactions hides the illicit signal. I have tried to buy a few of these mac studios only to have the transaction cancelled because I was…

Bizarre. They don't care that eBay takes 14%?

Traditional money laundering loses upwards of 40%, so hell yeah.

Re: Was my $48K GPU server worth it?

#373

Post by rithdmc »

digitaltrees wrote: post #273
gbgarbeb wrote: post #262

Earlier quoted context omitted.

The ones sold for $25k from established sellers are legit. Filter by "sold." The 0-reputation account in Spain selling an M3U 512GB for $4200 is 100% fraud.

oh young grasshopper, I see you dont know that money launderers love the ebay hype cycle. Its REALLY common on high dollar hot items to have phantom transactions where parties are on both sides of the transaction to clean illicit money. The high price tag and high volume amount of transactions hides the illicit signal. I have tried to buy a few of these mac studios only to have the transaction cancelled because I was…

The transaction was cancelled? Sounds like you weren't defrauded.

Re: Was my $48K GPU server worth it?

#374

Post by jononor »

zozbot234 wrote: post #343
jononor wrote: post #294

Earlier quoted context omitted.

Foe multi GPU make sure you have enough PCIE lanes! That rules out consumer grade sockets like AM5, you would need Threadripper or EPYC.

Why are these sockets "ruled out"? Pipeline/layer parallelism doesn't need high bandwidth between nodes, and tensor parallelism has middling performance unless you have very fast networking and very slow compute. It all depends on what you're doing.

You are correct that bandwidth requirements depends a lot on the exact workload. And that in specific cases, it might be doable to have AM5 for multiple RTX6000Pro. The parent mentioned workloads that are general, and broader than inference-only. In that case I would consider spending a bit extra on the motherboard to ensure that PCIE bandwidth is not an issue.

Re: Was my $48K GPU server worth it?

#375

Post by tomelders »

tempay wrote: post #306
PowerElectronix wrote: post #298

Earlier quoted context omitted.

On top of that, AI providers are also eating a big loss on the service.

Are they? I only ever see unsubstantiated claims for this whereas I see many justifications that interference is comfortably profitable in isolation.

SpaceX's has disclosed that they're loosing $2Bln a quarter on A.I - and rising - in their IPO documents.

Anthropic told the Department of War-nee-Defence that they'd made $5bln total, which is a lot LOT less than what they're spending.

We'll see what's in OpenAi's IPO later this year I guess. I'll be very surprised if they're losing less that $100bln a year.

Re: Was my $48K GPU server worth it?

#376

Post by conormcleod »

jwilliams wrote: post #286

> The mentality shift of renting vs. owning the gpus is huge. When renting, each experiment costs money and I had to ask myself is it worth it. When owning, it feels like not running experiments is costing me money. I feel like there is some very deep generalizable wisdom buried here.

You should be able to achieve this mentality shift without owning a GPU. You just need to commit some money upfront to cloud GPU spend, in a way that is not feasible to go back on.

That way you get the experimentation-encouraging mentality shift to "If I don't use this, I'm wasting my money", without the cost inefficiencies associated with actually buying an accelerator, discussed by others in this thread -> you'll never be able to match the the utilisation and thus the cost amortisation of cloud GPUs.

Re: Was my $48K GPU server worth it?

#377

Post by NicoHartmann »

irusensei wrote: post #352
NicoHartmann wrote: post #88

Jensen Huang said 'the more you buy, the more you save,' and you actually took it personally.

I've hit command+f and then looked for this.

Glad I could fulfill your search query. Doing my part for the SEO of this comment section.

Re: Was my $48K GPU server worth it?

#378

Post by mixermachine »

gpt5 wrote: post #346
mixermachine wrote: post #334

Earlier quoted context omitted.

The 5h quota of Codex Pro on GPT 5.4 Medium lasts me for around an hour and a half, maybe 2 hours. And this is already the "savy" setup. Enable GPT 5.5 High fast and you will be beached in 30 minutes with active development. For continues all day work you definitely need a higher tier sub level. I'm actually looking into deploying a GPU at my company because we can not give out our code. Qwen 3.6 looks good

this might be true for the plus account. For the "Pro" tier ($100-$200/month) the 5h limit is never a problem.

Right, I did swap that. Still, you have to pay that 4k then every year and give out the code. I also assume that prices will go up as no AI company (but NVIDIA -> selling shovels) is currently making any money.

For some projects the giving out the code part might be ok (i use Codex there too) but for the core app at the company I'm working at there is currently a strict no-AI policy. A local GPU solves this.

Re: Was my $48K GPU server worth it?

#379

Post by rob »

lucrbvi wrote: post #348
zkmon wrote: post #345

Just curious - What exactly are you using that rig for? I see that you said research work. Are you building a product or training models? I ask because whether something is worth it or not depends largely on what you get out oof it and how you value what you get. It's perfectly fine to leave a FANG job and go for, say, pottery hobby. What gives you happiness and your value system - these will qualify your decisions.

In the article the author says they are doing reinforcement learning with LLMs.

Seems like they just want to play PewDiePie after making tons of money from their salaried job and have a bunch of spare time now.
Post reply on HN