Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was my $48K GPU server worth it?

rosmine.ai

Post reply on HN
Print view View original

231–240 of 480 posts

Re: Was my $48K GPU server worth it?

#231

Post by margalabargala »

cronin101 wrote: post #73
freediddy wrote: post #66

Earlier quoted context omitted.

25k is definitely a lot but I did the risk analysis and I figured worst case I would lose a 1000-2000 after a year of playing around with it, so I look at it more like renting (I'm going to keep the Macbook Pro no matter what since I needed a new one).

Nitpicking, but the worst case of spending $25k is unforeseen circumstances that write off the entire asset. I don’t think -$2000 is a conservative enough figure for standard depreciation either (a lot can happen in a year)

Sure, they took a gamble that they wouldn't be able to sell it used.

If you are able to tie up $25k for a few years just for shiggles, you clearly are able to make do fine without that money and if lost it would be at worst annoying, not catastrophic.

Re: Was my $48K GPU server worth it?

#232

Post by Fabricio20 »

throwatdem12311 wrote: post #112
dekhn wrote: post #59

Earlier quoted context omitted.

I have a second computer with an RTX 4090 for gaming (running Windows). I also used the new RTX 5090 running Linux to evaluate whether Proton/Wine allow me to run Windows games on linux (yes, it works, but the compatibility and frame rate issues make me stick to native Windows for now).

If you want a GPU that has comparable performance on Linux to Windows- you want AMD. NVIDIA drivers are notoriously bad. Many of my games run better on Linux with the open source AMd drivers. (CachyOS rolling rolling rolling).

Sadly if you want a GPU with good AI performance you gotta go with NVIDIA. It might sound crazy but as a 7900XTX owner.. My 12GB 3060 on my linux server outperformed the 7900XTX by 40%. The 3060 only has half the vram of the AMD card. Proprietary drivers under Arch Linux.

On top of the significantly worse software on AMD's side (literally didn't work on windows in particular - so the "performs as good on both systems" is a nonstarter, some GGUF library dependency just doesn't work/exist under AMD on windows). Had me running the AMD card on windows under WSL (not a problem with nvidia though, that ran just fine on windows-side directly).

Aaaand also the other AMD bugs, such as the pink squares display corruption that has been an active issue for my GPU in particular (7900XTX) for over a year, maybe approaching two at this point, with no fix in sight from the AMD team (barely and ack at all - not on a single patch notes, just a bunch of reddit discussion). Really regret spending so much on an AMD gpu.

Re: Was my $48K GPU server worth it?

#233

Post by reissbaker »

kgeist wrote: post #150

I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…

Qwen 3.6 27B is fine but it's not in the same ballpark as GLM-5.1 or Kimi K2.6.

If you truly want to scale up, you should get the 8xH200 with NVLink.

Re: Was my $48K GPU server worth it?

#234

Post by dekhn »

komali2 wrote: post #211
dekhn wrote: post #59

Earlier quoted context omitted.

I have a second computer with an RTX 4090 for gaming (running Windows). I also used the new RTX 5090 running Linux to evaluate whether Proton/Wine allow me to run Windows games on linux (yes, it works, but the compatibility and frame rate issues make me stick to native Windows for now).

I have the same rig as you minus watercooling, and I assume you have AMD Ryzen 7 9800X3D? Anyway, it's my only PC now, I game, dev, run local models, edit photos, edit videos, all in Manjaro. I get ~70FPS in Cyberpunk at 4k, every setting at "Insane" or whatever goofy thing they call it, Ray tracing on path tracing off, with no framegen but with DLSS set to quality. Without DLSS I get around 40fps. Seems equivalent t…

No, it's an Alienware R51 with Intel Core Ultra 9 285K 3.2GHz Processor; NVIDIA GeForce RTX 5090 32GB GDDR7; 64GB DDR5-6400 RAM; 4TB Solid State Drive; Microsoft Windows 11 Home; 2.5GbE LAN; 2x2 Intel Killer WiFi 7 BE1750+Bluetooth 5.4; Liquid Cooler

I don't see it on the Dell site anymore, only more expensive, lesser configurations (good timing on my part?).

Yeah, I really want to put in the time to try out various games, but realistically, the whole point of getting a second computer and installing Linux was to be able to train and serve models, and switching between serving a model (that people in my house want to use at random times) and gaming didn't seem like a great choice. If I did get good results, I'd seriously consider wiping Windows 11 from my older machine (an older Alienware with a 4090), but to be honest, I'm perfectly comfortable on Windows desktop.

Re: Was my $48K GPU server worth it?

#235

Post by dekhn »

orsorna wrote: post #104
throwatdem12311 wrote: post #51

Earlier quoted context omitted.

Not even a single mention of gaming. No wonder gamers hate AI bros.

I would probably hate someone if they were buying the same hardware as me but doing something actually useful with it. Any game worth playing doesn't require high specs anyway. There is such a large catalog of old games.

I specifically got the previous model so I could play AAA games with all the settings set to Ultra, at 4K. Cyberpunk 2077 struggled even with my 4090, so I had to disable ray tracing and enable DLSS. Since I've run out of new AAA games I've been playing older ones and it's crazy how fast they are.

Re: Was my $48K GPU server worth it?

#236

Post by topaz0 »

ActorNightly wrote: post #220
jon-wood wrote: post #14

Earlier quoted context omitted.

I’m not usually one to ask this because learning to do a thing can be fun, but why exactly have you spent 25 thousand dollars on getting an LLM someone else made to answer maths exam questions?

Because buying Macs is not about performance, its about feeling like you are rich. That money could have been spent on way more bang/buck performance in the form of a set of 4 graphics cards. Also I would probably put the odds 70:30 that Apple marketing is astroturfing on HN from the amount of posts about running llms on Macbooks, because in reality, the inference speed of any decent llm is unusable on a Macbook desp…

Or it could have had way more bang/buck by feeding a family of real brains for a year or two

Re: Was my $48K GPU server worth it?

#237

Post by Craighead »

freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

I'd buy that Mac studio m3

Re: Was my $48K GPU server worth it?

#238

Post by jma24 »

freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

Better sell it fast before the M5 ones come out.

Re: Was my $48K GPU server worth it?

#239

Post by mixmastamyk »

fc417fc802 wrote: post #200
thefounder wrote: post #169

Earlier quoted context omitted.

Something is very wrong in some countries if you have to get police protection to sell a f* computer. I get it’s on the expensive side but still….

Not really. Every country has a nonzero number of criminals. It's entirely a matter of the risk/reward tradeoff. A small consumer item over $10k is well into dangerous territory.

Are we talking about a cash transaction? If so >$10k is dangerous as the police may want to steal it themselves.

If it is an electronic payment, I'm not sure how completing the transaction in front of a police station will help any. Well, it will help the buyer to see it working, but the seller gets no additional protection besides seeing "a person."

Re: Was my $48K GPU server worth it?

#240

Post by ChoGGi »

theYipster wrote: post #210
freediddy wrote: post #56

Earlier quoted context omitted.

RTX 6000 is some-what obviously my fastest card but my biggest problem with the RT 6000 is the immense heat. The GPU itself is almost 200F and the exhaust from the fans itself is over 150F. I'm worried that my hard drives are going to fail. I was told that the GDDR7 is even hotter than the GPU which is surprising to me. After my last run, I'm going to wait for the new case I ordered to come in and cannibalize my kid'…

That's about what my OC'd and watercooled 4090 runs at. The cards are designed for it. Only problem I have is when sitting next to the computer under load -- I either have to open windows or blast the AC. Too bad I don't live in a cold climate -- that 60c heat output would come in handy :)

> Too bad I don't live in a cold climate -- that 60c heat output would come in handy :)

Used to overclock back in the day during winter with an intake duct rigged to suck in outside air, best thing about -30c :)

Post reply on HN