Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was my $48K GPU server worth it?

rosmine.ai

Post reply on HN
Print view View original

331–340 of 480 posts

Re: Was my $48K GPU server worth it?

#331

Post by hacker_homie »

r0b05 wrote: post #295
kgeist wrote: post #150

I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…

How can a single 5090 serve 80 people? Something doesn't add up here.

They are using it as an assistant, bot running multiple fully automated agents loops?

Re: Was my $48K GPU server worth it?

#332

Post by KronisLV »

That’s very cool and very expensive - I think the cadastre value of the apartment that I live in is like 35k EUR or thereabout.

I wonder how much worse just a bunch of Intel Arc B70s might have been, software fuckery aside. Ofc if I’d need to run local inference or simple fine tunes and learning stuff, I’d probably get one of the SFF options - Mac Minis and all of those Sparks or new AMD AI chips. Then again, I’m broke so go figure.

I just fork over some money every month to Anthropic, have been trying out more DeepSeek and also Mistral (their Vibe tool is surprisingly passable under WSL).

Re: Was my $48K GPU server worth it?

#333

Post by tempay »

callmeal wrote: post #318
adastra22 wrote: post #310

Earlier quoted context omitted.

You got numbers? Because it seems perfectly possible to me. OpenAI and Anthropic’s marginal cost for inference is certainly far less than their API pricing.

See: https://www.wheresyoured.at/ He's been "numbering" for quite a while now.

Everything there is extremely speculative and I don't see anything that contradicts that inference itself could be profitable at massive scale. See https://youtu.be/xmkSf5IS-zw for example.

If the companies as a whole are destined to be profitable, or worth their valuations is a very different question. The only people who can truely answer that have time machines.

Re: Was my $48K GPU server worth it?

#334

Post by mixermachine »

gpt5 wrote: post #268
CobaltFire wrote: post #189

Earlier quoted context omitted.

I have a 5090 machine sitting idle that I'm considering turning into a machine for my own small team (3 devs). Are you willing to share any lessons learned, etc. that I could make use of? We are evaluating paying for a SOTA sub or trying this, and the talk about Qwen3.6-27B makes me want to try deploying this machine.

Sell the machine for $4K, use it to pay for Codex Pro for everyone for a year. Everyone will be significantly more productive and happy. It's not even a real comparison if they are actually using them for coding. If you are deploying always running agents (e.g. monitoring logs and services) then sure - a QWEN local server is a good choice. But for coding the cost in productivity of using a lower performing model is w…

The 5h quota of Codex Pro on GPT 5.4 Medium lasts me for around an hour and a half, maybe 2 hours. And this is already the "savy" setup. Enable GPT 5.5 High fast and you will be beached in 30 minutes with active development.

For continues all day work you definitely need a higher tier sub level.

I'm actually looking into deploying a GPU at my company because we can not give out our code. Qwen 3.6 looks good

Re: Was my $48K GPU server worth it?

#335

Post by kgeist »

r0b05 wrote: post #295
kgeist wrote: post #150

I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…

How can a single 5090 serve 80 people? Something doesn't add up here.

They don't use the server all at once. In the UI, users typically ask a question, get a response, and continue with their work. In the case of autonomous agentic loops, an agent simply waits its turn until the server is ready to accept the request. Agents don't hammer the server 24/7 every second either, because they either need to be triggered or are busy doing other work, such as compiling or running tests.

Re: Was my $48K GPU server worth it?

#336

Post by mixermachine »

r0b05 wrote: post #295
kgeist wrote: post #150

I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…

How can a single 5090 serve 80 people? Something doesn't add up here.

With parallelism of 16 you can still get around 25 to 30 tokens per user when all 16 channels are running. Not everyone will use the model at the same time but it certainly will be tight, especially for agentic coding. For pure chat applications this should be quite fine.

Re: Was my $48K GPU server worth it?

#337

Post by graemep »

miki123211 wrote: post #227
freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

This is, sadly, obvious and inevitable in retrospect. The two major drivers of inference costs are GPUs and electricity. You can't get cheaper GPUs, but you can make existing GPUs not sit idle, and you do that by utilizing them 24/7, processing user B's request when user A is thinking, and handling many requests in parallel, neither of which you can do as an individual. You can get cheaper electricity... by moving, a…

High usage seems to change the economics. The author of the article had a payback period of about 14 months which is excellent by any standards and an order of magnitude better than rent vs buy for a house in most places.

Re: Was my $48K GPU server worth it?

#339

Post by dgellow »

thomasmarcelis wrote: post #329
jwilliams wrote: post #286

> The mentality shift of renting vs. owning the gpus is huge. When renting, each experiment costs money and I had to ask myself is it worth it. When owning, it feels like not running experiments is costing me money. I feel like there is some very deep generalizable wisdom buried here.

Also something about subscriptions vs pay-for-usage. I feel the need to use all my weekly tokens or I'm wasting and I bet they would never get this kind of usage out of me if AI ended up being same price per token.

I always buy software/assets/dev tools for my hobbies (like CAD, music production, game dev) instead of paying subscriptions, even if that would very likely be way, way cheaper and would give me access to really cool tools. I don’t want to feel bad not using something and I know that’s the case with a subscription

Re: Was my $48K GPU server worth it?

#340

Post by jvidalv »

r0b05 wrote: post #295
kgeist wrote: post #150

I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…

How can a single 5090 serve 80 people? Something doesn't add up here.

I also call this "bollocks" there is no way this workflow is even 1/10 of what you can get with Codex/Claude Code.

A normal engineer may be running a couple of sessions with every session spawning sub agents left and right.

80 persons or even 10 having this workflow on this setup doesn't work, and this is the standard engineer workflow today.

Post reply on HN