Can anyone recommend what types of server would it be required to run a RTX 6000 Pro?
Was my $48K GPU server worth it?
301–310 of 480 posts
Re: Was my $48K GPU server worth it?
#302In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…
How are you using the 6000 with a Mac ?
Re: Was my $48K GPU server worth it?
#303In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…
I got an RTX 6000 pro too. I like running locally, I've learned a lot more than if I had used an API and there's less worry about overspending tokens. I accidentally spent $100 on claude api in like 2 days because I didn't know what I was doing. The problem is that while one these gpus is a huge improvement over a laptop or a single 3090, you very quickly wish you had more. I would buy a second one, but I did the mat…
What kind of machine did you build around it ?
Re: Was my $48K GPU server worth it?
#304I ve seen already one question like that in the thread. But I rephrase it slightly sharper. Did you consider renting out you setup to vast.ai and if so, how much money it can generate per month deducting electricity.
Also, sorry for the noob question, is not such server generate enormous amount of heat? You did not use any special cooling system?
Re: Was my $48K GPU server worth it?
#305Earlier quoted context omitted.
It's still very contrarian to expect GPUs won't depreciate rapidly. Yes 3090s were a good investment then, but way worse than just buying Nvidia stock directly
Waiting for them to come down any day now. Been waiting since 2017.
First it was crypto, now AI. Just because the market can stay irrational for very long doesn't mean crashes don't happen. What nobody knows is when.
Re: Was my $48K GPU server worth it?
#306Earlier quoted context omitted.
This is, sadly, obvious and inevitable in retrospect. The two major drivers of inference costs are GPUs and electricity. You can't get cheaper GPUs, but you can make existing GPUs not sit idle, and you do that by utilizing them 24/7, processing user B's request when user A is thinking, and handling many requests in parallel, neither of which you can do as an individual. You can get cheaper electricity... by moving, a…
On top of that, AI providers are also eating a big loss on the service.
Are they? I only ever see unsubstantiated claims for this whereas I see many justifications that interference is comfortably profitable in isolation.
Re: Was my $48K GPU server worth it?
#307Earlier quoted context omitted.
This is, sadly, obvious and inevitable in retrospect. The two major drivers of inference costs are GPUs and electricity. You can't get cheaper GPUs, but you can make existing GPUs not sit idle, and you do that by utilizing them 24/7, processing user B's request when user A is thinking, and handling many requests in parallel, neither of which you can do as an individual. You can get cheaper electricity... by moving, a…
On top of that, AI providers are also eating a big loss on the service.
Supposedly Anthropic just reported that they’re operationally profitable. So maybe not?
Re: Was my $48K GPU server worth it?
#308Earlier quoted context omitted.
On top of that, AI providers are also eating a big loss on the service.
Are they? I only ever see unsubstantiated claims for this whereas I see many justifications that interference is comfortably profitable in isolation.
Its basic math, go calculate max sessions for a certain tps on any hardware. Session# * tps * 86400 (secs in a day) * 30 days.
You'll realize real quick its not profitible. You cant just say things you don't like to hear are unsubstantiated without verifying.
Not to mention, subscriptions.. $2mm in GPUs being given out for 5 hrs a day at a cost of $200 a month.
I could easily say that everyone who says its profitible is msking unsubstantiated claims lol.
Re: Was my $48K GPU server worth it?
#309Earlier quoted context omitted.
I have a 5090 machine sitting idle that I'm considering turning into a machine for my own small team (3 devs). Are you willing to share any lessons learned, etc. that I could make use of? We are evaluating paying for a SOTA sub or trying this, and the talk about Qwen3.6-27B makes me want to try deploying this machine.
Sell the machine for $4K, use it to pay for Codex Pro for everyone for a year. Everyone will be significantly more productive and happy. It's not even a real comparison if they are actually using them for coding. If you are deploying always running agents (e.g. monitoring logs and services) then sure - a QWEN local server is a good choice. But for coding the cost in productivity of using a lower performing model is w…
Anyone who frivolously suggests throwing away possible independence in favor of dependence on a Silicon Valley company is either incredibly naïve or acting in bad faith.
Re: Was my $48K GPU server worth it?
#310Earlier quoted context omitted.
Are they? I only ever see unsubstantiated claims for this whereas I see many justifications that interference is comfortably profitable in isolation.
Its basic math, go calculate max sessions for a certain tps on any hardware. Session# * tps * 86400 (secs in a day) * 30 days. You'll realize real quick its not profitible. You cant just say things you don't like to hear are unsubstantiated without verifying. Not to mention, subscriptions.. $2mm in GPUs being given out for 5 hrs a day at a cost of $200 a month. I could easily say that everyone who says its profitible…
You got numbers? Because it seems perfectly possible to me. OpenAI and Anthropic’s marginal cost for inference is certainly far less than their API pricing.