[deleted]
Was my $48K GPU server worth it?
271–280 of 480 posts
Re: Was my $48K GPU server worth it?
#272In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…
I’ve had the best luck selling in Craigslist. Every other platform has been sub par.
Re: Was my $48K GPU server worth it?
#273Earlier quoted context omitted.
It’s hard to know if any of these eBay listing are real or actual sales. Lots of scams.
The ones sold for $25k from established sellers are legit. Filter by "sold." The 0-reputation account in Spain selling an M3U 512GB for $4200 is 100% fraud.
oh young grasshopper, I see you dont know that money launderers love the ebay hype cycle. Its REALLY common on high dollar hot items to have phantom transactions where parties are on both sides of the transaction to clean illicit money. The high price tag and high volume amount of transactions hides the illicit signal. I have tried to buy a few of these mac studios only to have the transaction cancelled because I wasnt the dirty money on the other side.
Re: Was my $48K GPU server worth it?
#274In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…
You definitely want to get rid of your M3 Ultra before the M5 Ultra get officially announced.
Give the global memory shortage, the m5 will be both delayed and restricted to lower ram tiers, I dont think we will see a 512gb ram model until 2030
Re: Was my $48K GPU server worth it?
#275Earlier quoted context omitted.
Either I don't understand the used apple market.. or I agree this is crazy. Someone spends $25k on new hardware, waits a year, and expects to sell it for $23k? Unless the ram issues save him, and cost of new goes up, I don't see how that was going to work.
A lot of expensive things hold their value well. I have a friend who is really into telescopes and he now owns a $100,000 telescope but he didn't directly buy such an expensive scope. He started out with much cheaper ones and was able to sell them for about what he bought them for to help fund more expensive ones over 20 years. It is really interesting.
Thank god. I’ve been waiting to have a reason to tell someone I collect fountain pens.
Computers depreciate because they are obviously being supplanted by newer better models—until they become vintage and then move into collectibles.
Re: Was my $48K GPU server worth it?
#276'If you google “plugging a PC into multiple outlets”, you get lots of warnings that if you even consider such a setup you will instantly burst into flames. So I hired a professional PC builder make sure it was safe.' Not really sure how that makes it safe but OK!
Agreed, if anything an electrician was needed, not a professional PC builder.
The picture shows two power supplies. Powering what is effectively one appliance from different circuits is a definite no-no, and I can't think of any circumstance where it wouldn't be in a home.
If his mains supply was sufficient to run the server and the house in the first place then the simplest solution would be to simply upgrade one of the MCBs/RCBOs on one of the circuits to the required capacity. I am not sure a landlord would even notice something like that, and if the house is wired correctly in the first place, it's unlikely to be dangerous. So going from say, 6A to 12A, on a 20A mains supply is generally fine if the gauge of wiring is correct.
Re: Was my $48K GPU server worth it?
#277In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…
This is, sadly, obvious and inevitable in retrospect. The two major drivers of inference costs are GPUs and electricity. You can't get cheaper GPUs, but you can make existing GPUs not sit idle, and you do that by utilizing them 24/7, processing user B's request when user A is thinking, and handling many requests in parallel, neither of which you can do as an individual. You can get cheaper electricity... by moving, a…
You can definitely run many requests in parallel as a single user, you just have to be OK with a significant slowdown for any single request. Cloud inference can't reach that ratio of total throughput per hardware cost since they are heavily incented to get the most expensive hardware available and to then minimize latency (and RAM occupation over time) even at the cost of throughput. Running slower inference with cheaper hardware is just not workable in a cloud setting.
Re: Was my $48K GPU server worth it?
#278Earlier quoted context omitted.
Yes, i mean, he could rent a cage and run grumbl it there. It doesn't have to be a standard datacenter server, even though a standard datacenter server would be better and cheaper.
A cage[0] is ~100x larger than what you need to host a single server. Many data centers will colocate by the rack unit. At others you can get a quarter or half cabinet[1]. Even at the very largest enterprise datacenters you can colocate a single cabinet. [0]: https://static.cisco-eagle.com/images/category/WireCrafters/... [1]: https://www.edpeurope.com/wp-content/uploads/EDP-3-Compartme...
> A cage[0] is ~100x larger than what you need to host a single server.
Yup, but i was assuming that he wanted to experiment building gpu rigs. For sure standard GPU servers are cheaper and easy to maintain. I have two lenovos, bought them used, already EOL.. was cheap and better than any custom gpu rig.. but i was pragmatic, because my goal was to put it in production, and not to research...
Re: Was my $48K GPU server worth it?
#279Great article. I'm about to embark on a similar journey.... Doing a ton of AI development right now. Don't need a server, but a very, very high end workstation is super appealing to me right now. Looking at $50-$80k. 1TB RAM. 2x RTX Pro 6000s. 64 core Threadripper Pro. As many 4tb or 8tb nvme drives as I can stuff. I envision NixOS at the core... then everything I need virtualized on top with KVM/QEMU. Maybe a dual b…
What do you want to do with the workstation? I have a similar setup:
- 512 GB
- Epyc 9684x
- 2x RTX 6000 Pro
- 1400 W PSU x 2 but in redundant mode
Mine is in a colo where it stays nice and cool. In my case, I went with less RAM and more GPUs (bought 4). Secondarily, the Max-Q blower version of an RTX 6000 Pro Blackwell is easier to keep cool and also only needs 300 W at the cost of very little performance. The non-max-q also only really use 300 W during inference, but the good thing about a lower power use is you can put more GPUs in very safely.
I assume you want the Threadripper Pro to maximize single-core performance? So you're spending a lot of time on CPU? Interesting stuff.
I gained a lot putting the machine somewhere else. TTFT on a thing like this is between 100-800 ms depending on batching and model size and so on, and your nearest datacenter is likely Good luck with your setup. If you get around to it, and end up writing about your setup on a blog, do share. Email in profile.
Re: Was my $48K GPU server worth it?
#280I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…
> Does anyone here have experience running large models in a multi-GPU setup with several RTX 6000s in a high-concurrency regime and with large context lengths? (something like Deepseek 4 Flash, Minimax 2.7 etc.)
Join the RTX6kPRO tribe!