Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was my $48K GPU server worth it?

rosmine.ai

Post reply on HN
Print view View original

351–360 of 480 posts

Re: Was my $48K GPU server worth it?

#351

Post by zozbot234 »

jvidalv wrote: post #340
r0b05 wrote: post #295

Earlier quoted context omitted.

How can a single 5090 serve 80 people? Something doesn't add up here.

I also call this "bollocks" there is no way this workflow is even 1/10 of what you can get with Codex/Claude Code. A normal engineer may be running a couple of sessions with every session spawning sub agents left and right. 80 persons or even 10 having this workflow on this setup doesn't work, and this is the standard engineer workflow today.

Subagent swarms are actually great for the local inference scenario because they can share a whole lot of KV cache. You get to raise the compute intensity of decode (i.e. the aggregate tok/s) essentially for free.

Re: Was my $48K GPU server worth it?

#353

Post by girvo »

adastra22 wrote: post #313
girvo wrote: post #291

Earlier quoted context omitted.

Yes I could, but that is annoying because of spot pricing and having my instance shut down, and it has fluctuating prices It’s also annoying because then I need to make sure my little “lab” setup is well automated, and I’m lazy :) Also, I literally said “ It's not financially a good idea” so I’m confused why you think I don’t know that.

Spot pricing and instance availability don’t apply to on metal hosting. You’d have your own machine dedicated to your own use only, at a locked in price.

> renting really does beat owning, and cloud beats both

Re: Was my $48K GPU server worth it?

#354

Post by titanomachy »

zkmon wrote: post #345

Just curious - What exactly are you using that rig for? I see that you said research work. Are you building a product or training models? I ask because whether something is worth it or not depends largely on what you get out oof it and how you value what you get. It's perfectly fine to leave a FANG job and go for, say, pottery hobby. What gives you happiness and your value system - these will qualify your decisions.

They posted their research results, it's linked at the end of the article.

https://rosmine.ai/2026/05/18/fixing-llm-writing-with-distri...

Re: Was my $48K GPU server worth it?

#356

Post by AdamN »

adrianN wrote: post #257
miki123211 wrote: post #227

Earlier quoted context omitted.

This is, sadly, obvious and inevitable in retrospect. The two major drivers of inference costs are GPUs and electricity. You can't get cheaper GPUs, but you can make existing GPUs not sit idle, and you do that by utilizing them 24/7, processing user B's request when user A is thinking, and handling many requests in parallel, neither of which you can do as an individual. You can get cheaper electricity... by moving, a…

Historically it was not uncommon for beds to be rented out to multiple people.

Yeah there are good accounts of this in Down and Out in Paris and London and also one of Hemingway's books - forgot which one.

Re: Was my $48K GPU server worth it?

#357

Post by joefourier »

gbgarbeb wrote: post #314
joefourier wrote: post #60

Earlier quoted context omitted.

Who is going to buy a $4299 M5 Max MBP with 64GB of RAM just to run Gemma 4 31b? Firstly you don't need 64GB for that model. Secondly if you want a machine that sits in the corner and does nothing but LLM inference, you don't buy a MacBook Pro, you buy some GPUs which are going to cost you a fraction of that (~$1k for ~64GB of VRAM is possible). The people buying Apple Silicon for inference general aim for the Mac St…

24GB GPUs are $700-2500. Please show me the 64GB GPU for $1k.

Not a single new 64GB GPU, but multiple used GPUs.

They’ve significantly increased in price (so much for hardware depreciation…) but you can still get a modded 22GB 2080 ti for $320, or a Mi50 32GB for ~$450 each (used to be $150 a few months ago, alas), or a Mi50 16GB or There’s also some more exotic configurations but those are probably the simplest options. You won’t get the performance of an RTX Pro 6000 Blackwell of course, and the power consumption will be pretty high so it’s only worth it if you have cheap electricity. But it is possible.

Re: Was my $48K GPU server worth it?

#358

Post by baq »

oblio wrote: post #305
latentsea wrote: post #250

Earlier quoted context omitted.

Waiting for them to come down any day now. Been waiting since 2017.

First it was crypto, now AI. Just because the market can stay irrational for very long doesn't mean crashes don't happen. What nobody knows is when.

at some point you have to accept that the market is actually rational

Re: Was my $48K GPU server worth it?

#360

Post by nicman23 »

kgeist wrote: post #150

I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…

> 260k context

with a single 5090?

Post reply on HN