"quit my FAANG job" as in they simultaneously worked in Facebook, Amazon, Apple, Nvidia and Google? Or did the op work at Netflix and is too ashamed to admit that :P
Was my $48K GPU server worth it?
341–350 of 480 posts
Re: Was my $48K GPU server worth it?
#342[flagged]
Re: Was my $48K GPU server worth it?
#343Earlier quoted context omitted.
I am also considering to buy 3-4x RTX 6000 Pro 96GB plus some Ryzen workstation with a grant. Is this the best general-purpose choice as of 2026 with $50k for training, fine-tuning and running large open models?
Foe multi GPU make sure you have enough PCIE lanes! That rules out consumer grade sockets like AM5, you would need Threadripper or EPYC.
Why are these sockets "ruled out"? Pipeline/layer parallelism doesn't need high bandwidth between nodes, and tensor parallelism has middling performance unless you have very fast networking and very slow compute. It all depends on what you're doing.
Re: Was my $48K GPU server worth it?
#344Earlier quoted context omitted.
How can a single 5090 serve 80 people? Something doesn't add up here.
They don't use the server all at once. In the UI, users typically ask a question, get a response, and continue with their work. In the case of autonomous agentic loops, an agent simply waits its turn until the server is ready to accept the request. Agents don't hammer the server 24/7 every second either, because they either need to be triggered or are busy doing other work, such as compiling or running tests.
It would be more interesting to know how many simultaneous users this setup can serve. Otherwise I can just say it serves 500 users but not all of them use it at the same time which doesn't communicate the right level of detail.
Re: Was my $48K GPU server worth it?
#345Just curious - What exactly are you using that rig for? I see that you said research work. Are you building a product or training models? I ask because whether something is worth it or not depends largely on what you get out oof it and how you value what you get. It's perfectly fine to leave a FANG job and go for, say, pottery hobby. What gives you happiness and your value system - these will qualify your decisions.
Re: Was my $48K GPU server worth it?
#346Earlier quoted context omitted.
Sell the machine for $4K, use it to pay for Codex Pro for everyone for a year. Everyone will be significantly more productive and happy. It's not even a real comparison if they are actually using them for coding. If you are deploying always running agents (e.g. monitoring logs and services) then sure - a QWEN local server is a good choice. But for coding the cost in productivity of using a lower performing model is w…
The 5h quota of Codex Pro on GPT 5.4 Medium lasts me for around an hour and a half, maybe 2 hours. And this is already the "savy" setup. Enable GPT 5.5 High fast and you will be beached in 30 minutes with active development. For continues all day work you definitely need a higher tier sub level. I'm actually looking into deploying a GPU at my company because we can not give out our code. Qwen 3.6 looks good
this might be true for the plus account. For the "Pro" tier ($100-$200/month) the 5h limit is never a problem.
Re: Was my $48K GPU server worth it?
#347Earlier quoted context omitted.
Sell the machine for $4K, use it to pay for Codex Pro for everyone for a year. Everyone will be significantly more productive and happy. It's not even a real comparison if they are actually using them for coding. If you are deploying always running agents (e.g. monitoring logs and services) then sure - a QWEN local server is a good choice. But for coding the cost in productivity of using a lower performing model is w…
Anyone who frivolously suggests throwing away possible independence in favor of dependence on a Silicon Valley company is either incredibly naïve or acting in bad faith.
I'll choose not to respond to your personal attack.
But in term of actually running a dev team - you are free to use QWEN or another quantized local model that can run on an RTX 5090 for coding if it makes you feel more independence. However you would struggle and spend many many more hours achieving the same thing, with a lot more debugging time, long delays before it's done, and many more prompts.
It's just not the right approach. I use QWEN and other local models all the time, but for more clearly defined monitoring and classification tasks.
Re: Was my $48K GPU server worth it?
#348Just curious - What exactly are you using that rig for? I see that you said research work. Are you building a product or training models? I ask because whether something is worth it or not depends largely on what you get out oof it and how you value what you get. It's perfectly fine to leave a FANG job and go for, say, pottery hobby. What gives you happiness and your value system - these will qualify your decisions.
In the article the author says they are doing reinforcement learning with LLMs.
Re: Was my $48K GPU server worth it?
#349Earlier quoted context omitted.
How can a single 5090 serve 80 people? Something doesn't add up here.
With parallelism of 16 you can still get around 25 to 30 tokens per user when all 16 channels are running. Not everyone will use the model at the same time but it certainly will be tight, especially for agentic coding. For pure chat applications this should be quite fine.
The problem with wide parallelism with most models is that it blows up your KV cache. There's open models with KV caches lean enough to parallelize inference or even to offload the KV cache itself to disk without immediately running into wearout concerns, but they're quite exceptional.
Re: Was my $48K GPU server worth it?
#350“Being less able to detect whether a text is AI-generated is exactly what nobody asked for — except villains.” ("—")