Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was my $48K GPU server worth it?

rosmine.ai

Post reply on HN
Print view View original

131–140 of 480 posts

Re: Was my $48K GPU server worth it?

#131

Post by dekhn »

throwatdem12311 wrote: post #112
dekhn wrote: post #59

Earlier quoted context omitted.

I have a second computer with an RTX 4090 for gaming (running Windows). I also used the new RTX 5090 running Linux to evaluate whether Proton/Wine allow me to run Windows games on linux (yes, it works, but the compatibility and frame rate issues make me stick to native Windows for now).

If you want a GPU that has comparable performance on Linux to Windows- you want AMD. NVIDIA drivers are notoriously bad. Many of my games run better on Linux with the open source AMd drivers. (CachyOS rolling rolling rolling).

I have no interest in moving to AMD for video cards right now- the network effect of NVIDIA is just too high, and their peak performance is insane. I also haven't noticed any major issues with nvidia drivers, unless you mean specifically running Windows games on Linux machines with nvidia cards, where I have zero experience.

Re: Was my $48K GPU server worth it?

#132

Post by caymanjim »

This article appears to lack any reason for "needing" this beast, or any real comparison with alternatives, both of which are required to answer the question posed in the title. It's a summary of how much they spent and some light anecdotal comparison to what they might have spent on cloud services, but clearly they didn't do an exhaustive hunt for value.

The real question is whether or not they could have done whatever it is they did with less hardware. Is there a business idea here that could have been proven on cheaper hardware that could be upgraded as demand increased? Is the expected ROI there based on future earnings?

Absent any indication that this was needed in the first place, I can only conclude that it wasn't worth anything.

Re: Was my $48K GPU server worth it?

#133

Post by Atotalnoob »

freediddy wrote: post #71
jmyeet wrote: post #55

Earlier quoted context omitted.

I looked into the M3 Ultra 512GB Mac Studio before it was discontinued and the as best as I could determine it just wasn't worth it... yet. The GFLOPS and memory bandwidth just arne't there even though it can hold a much larger model in memory. But the trend here is interesting. I think by 2030 you'll be able to buy fairly cheap hardware that is currently $10k+. I don't know what this does to the trillions invested i…

I agree that all these datacenter companies like Coreweave are investing billions in technology that has a very fast depreciation curve and I don't know how they will sustain income. The same goes for datacenters in space, what happens when those chips are obsolete? Will they sent astronauts to replace them or will they let them burn up and send new ones into orbit every year? I feel that the open weight models pale…

Data centers in space aren’t realistic.

Higher radiation, space insulations, etc.

Underwater data centers provide a lot of the same benefits and can (much more) easily be hauled to the surface

Re: Was my $48K GPU server worth it?

#134

Post by dekhn »

echelon wrote: post #99
throwatdem12311 wrote: post #51

Earlier quoted context omitted.

Not even a single mention of gaming. No wonder gamers hate AI bros.

> No wonder gamers hate AI bros. Personally, playing with AI models is way more fun than getting sucked into a game loop. Game loops feel like busy work hooked to an engineered dopamine drip. AI models are new frontiers and are exciting to build with, modify, lobotomize, and hack around with.

I remember playing Quake III which had user-programmable bots and thinking "wow, this is a really hard computer vision and reasoning problem". And then realizing "huh, that's a major research area, I should work on that". Later I learned that the bots were fairly simple and worked on far simpler world representations (nav meshes).

It looks like DM took a crack at it: https://deepmind.google/blog/capture-the-flag-the-emergence-...

Re: Was my $48K GPU server worth it?

#135

Post by [deleted] »

freediddy wrote: post #71
jmyeet wrote: post #55

Earlier quoted context omitted.

I looked into the M3 Ultra 512GB Mac Studio before it was discontinued and the as best as I could determine it just wasn't worth it... yet. The GFLOPS and memory bandwidth just arne't there even though it can hold a much larger model in memory. But the trend here is interesting. I think by 2030 you'll be able to buy fairly cheap hardware that is currently $10k+. I don't know what this does to the trillions invested i…

I agree that all these datacenter companies like Coreweave are investing billions in technology that has a very fast depreciation curve and I don't know how they will sustain income. The same goes for datacenters in space, what happens when those chips are obsolete? Will they sent astronauts to replace them or will they let them burn up and send new ones into orbit every year? I feel that the open weight models pale…

[deleted]

Re: Was my $48K GPU server worth it?

#136

Post by tencentshill »

freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

No harm in listing it for $20k, and if it sells, that's an easy $5-10k for you.

Re: Was my $48K GPU server worth it?

#137

Post by hnav »

throwawaytea wrote: post #116
cronin101 wrote: post #73

Earlier quoted context omitted.

Nitpicking, but the worst case of spending $25k is unforeseen circumstances that write off the entire asset. I don’t think -$2000 is a conservative enough figure for standard depreciation either (a lot can happen in a year)

Either I don't understand the used apple market.. or I agree this is crazy. Someone spends $25k on new hardware, waits a year, and expects to sell it for $23k? Unless the ram issues save him, and cost of new goes up, I don't see how that was going to work.

This is the case in lots of markets, e.g. look at used cars, luxury goods and more. Some of it is driven by inflation/the rapid devaluation of the dollar. General and AI-adjacent compute in particular hasn't come down in price in a long while.

Re: Was my $48K GPU server worth it?

#138

Post by dylan604 »

hnuser123456 wrote: post #33
jon-wood wrote: post #14

Earlier quoted context omitted.

I’m not usually one to ask this because learning to do a thing can be fun, but why exactly have you spent 25 thousand dollars on getting an LLM someone else made to answer maths exam questions?

Privacy and offline operation are valuable or non-negotiable in some cases, but the difference is pretty categorical between what can run on a single card and what can run on a DGX GB200 NVL72 cabinet. Doesn't mean it's not worth seeing how far local models can be pushed. Not every problem needs a senior engineer.

I know it's one of those "if you have to ask" situations, but curiosity got the better part of me. Here's the search assist response:

"The DGX GB200 NVL72 AI server costs approximately $3 million per unit. This system includes 72 Blackwell GPUs and 36 Grace CPUs, making it one of the most powerful AI servers available."

The search assist actually credited a source used with: https://www.tweaktown.com/news/98292/nvidias-new-gb200-super...

That $25k spend by GGGP seems like nothing in comparison. That's ~1/3 of one chip in that cabinet. God gawd I'm old and out of touch with modern AI data centers.

Re: Was my $48K GPU server worth it?

#139

Post by dylan604 »

freediddy wrote: post #56
bethekind wrote: post #37

Earlier quoted context omitted.

Which of these has been the most productive for you? Sounds like you've enjoyed the RTX6000 the most?

RTX 6000 is some-what obviously my fastest card but my biggest problem with the RT 6000 is the immense heat. The GPU itself is almost 200F and the exhaust from the fans itself is over 150F. I'm worried that my hard drives are going to fail. I was told that the GDDR7 is even hotter than the GPU which is surprising to me. After my last run, I'm going to wait for the new case I ordered to come in and cannibalize my kid'…

Since you are not running realtime 3d grafix, could you put the card in an external chassis so the heat is not in the same box as the SSDs?

Re: Was my $48K GPU server worth it?

#140

Post by dylan604 »

m463 wrote: post #100

Other things people spend "too much money" on: - muscle cars, with all the stuff, driven occasionally. - boats, that don't get taken out much - gamer x, where x=system or laptop or keyboard or mouse or desk or glasses or mousepad or speakers or ... usually with "> too much RGB" - children $48k for something constructive even if ai related? no problem, refreshing even.

One of these things is not like the others. If you don't spend the money on one of them, you can get a visit from government officials that might decide to take that "item" from you. You'd also be a worthless human to spend that money on the other 3 while not on the one.
Post reply on HN