Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was my $48K GPU server worth it?

rosmine.ai

Post reply on HN
Print view View original

191–200 of 480 posts

Re: Was my $48K GPU server worth it?

#191

Post by TheOtherHobbes »

dylan604 wrote: post #138
hnuser123456 wrote: post #33

Earlier quoted context omitted.

Privacy and offline operation are valuable or non-negotiable in some cases, but the difference is pretty categorical between what can run on a single card and what can run on a DGX GB200 NVL72 cabinet. Doesn't mean it's not worth seeing how far local models can be pushed. Not every problem needs a senior engineer.

I know it's one of those "if you have to ask" situations, but curiosity got the better part of me. Here's the search assist response: "The DGX GB200 NVL72 AI server costs approximately $3 million per unit. This system includes 72 Blackwell GPUs and 36 Grace CPUs, making it one of the most powerful AI servers available." The search assist actually credited a source used with: https://www.tweaktown.com/news/98292/nvidi…

It's The Circle of Computing Life. The pendulum swings between centralised mainframe timesharing-for-hire and desktop individuality.

We've been in a centralised phase for longer than usual - first cloud everything, then AI - but at some point in the next decade prices will crash and a market will appear for personal, local intelligence.

Re: Was my $48K GPU server worth it?

#192

Post by jumploops »

caymanjim wrote: post #132

This article appears to lack any reason for "needing" this beast, or any real comparison with alternatives, both of which are required to answer the question posed in the title. It's a summary of how much they spent and some light anecdotal comparison to what they might have spent on cloud services, but clearly they didn't do an exhaustive hunt for value. The real question is whether or not they could have done whate…

At the end of the article, the author has this to say:

> UPDATE: Launch was a success! 400K+ views, and multiple companies reached to use my IP. Read more here[0]

[0]https://rosmine.ai/2026/05/18/fixing-llm-writing-with-distri...

Re: Was my $48K GPU server worth it?

#193

Post by michaelt »

thefounder wrote: post #169
tracker1 wrote: post #147

Earlier quoted context omitted.

If you're in a decent sized city, you should be able to find a local buyer on Craigslist or FB Marketplace... Beyond that, for higher value, smaller items like your M3 Ultra, I would talk to your local police department and/or library to see if you can do the exchange there. Larger libraries usually have a police officer on site or nearby, and the PD office near you may also provide a "safe" exchange location... I'd…

Something is very wrong in some countries if you have to get police protection to sell a f* computer. I get it’s on the expensive side but still….

See e.g. https://www.murphytx.org/843/Safe-Exchange https://www.ottawapolice.ca/en/community-safety-and-crime-pr...

Police "safe trade zones" are basically a parking space outside a police station, with a sign.

Re: Was my $48K GPU server worth it?

#194

Post by CamperBob2 »

kgeist wrote: post #150

I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…

Does anyone here have experience running large models in a multi-GPU setup with several RTX 6000s in a high-concurrency regime and with large context lengths? (something like Deepseek 4 Flash, Minimax 2.7 etc.)

For what it's worth, I've been seeing ~100 tps with 4-bit MiniMax 2.7 on two RTX 6000 boards, just running under llama-server without any optimization effort at all. I have no serious long-context experience with that setup, but at 30K context it's still above 90 tps.

If you are happy with Qwen 3.6 27B, I would personally switch the 5090 out for 2x RTX 6000s and keep running 27B. That will give you ~2x your current throughput with a lot more headroom for multiple users. More important, it would buy time to see how things develop over the next few months before you spend a whole lot of money.

Re: Was my $48K GPU server worth it?

#195

Post by dekhn »

throwatdem12311 wrote: post #181
dekhn wrote: post #131

Earlier quoted context omitted.

I have no interest in moving to AMD for video cards right now- the network effect of NVIDIA is just too high, and their peak performance is insane. I also haven't noticed any major issues with nvidia drivers, unless you mean specifically running Windows games on Linux machines with nvidia cards, where I have zero experience.

Network effect for graphics cards? Literally what? Your friends don’t care what GPU you run my guy and there is not much benefit of having brand loyalty to a company like Nvidia that gives absolutely zero fucks about people that aren’t their enterprise customers buying GPUs by the thousands. If there’s any “network effect” for gaming GPUs on Linux it’s in favor of AMD because of the immense amount of work Valve has b…

I am also an enterprise customer that buys GPUs by the thousands, you can see a bit more about my work here: https://www.gene.com/media/press-releases/15010/2023-11-21/g... and https://blogs.nvidia.com/blog/roche-ai-factories-omniverse/ and have worked with nvidia since the mid-2000s on high performance computing for scientific research (in addition to having nvidia graphics cards since the Riva TNT, running both Windows and Linux). So having a blackwell graphics card I can evalute with linux and windows, both for ML training, inference, and gaming, is a huge network effect.

Re: Was my $48K GPU server worth it?

#196

Post by arjie »

skiing_crawling wrote: post #148
freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

I got an RTX 6000 pro too. I like running locally, I've learned a lot more than if I had used an API and there's less worry about overspending tokens. I accidentally spent $100 on claude api in like 2 days because I didn't know what I was doing. The problem is that while one these gpus is a huge improvement over a laptop or a single 3090, you very quickly wish you had more. I would buy a second one, but I did the mat…

You can fit Deepseek 4 Flash on two with TP 2 and 6 different streams at 65k context. 150 tok/s

Re: Was my $48K GPU server worth it?

#197

Post by vidarh »

Quarrel wrote: post #52
vidarh wrote: post #44

Earlier quoted context omitted.

Giving an LLM samples and tell it to apply the style in the sample works a lot better than just telling it to copy a style it may have seen, or a list of rules. They do it well enough that it'd take really good output to beat.

They really don't. If your goal is to say, write science fiction, their reversion to classic LLM-isms, is really distracting and is what makes people say from a glance that it was written by an LLM. You basically can't use them at the moment in any real "natural" long-form writing. Everyone will call "slop" pretty quickly on the current frontier models. Rosmin's DFT paper is worth a read.

I have seen examples that shows otherwise, including from a client that tested it extensively by paying people who thought they were paid to help detect AI generated content. They did little more than what I described. It works very well. Some people still insist they are able to tell the difference, but in the tests I saw, people did little better than random chance.

Some of it you could probably tell with statistical analysis, but actualy people are far worse at judging whether content is AI generated than they think they are.

If you need to beat an AI testing tool, you need to do marginally more work than to stop people from recognising it, but not all that much.

The nature of it is that you don't "see" most of the stuff that is well done because few people want to talk about it.

Re: Was my $48K GPU server worth it?

#198

Post by mschuster91 »

beachy wrote: post #157
mschuster91 wrote: post #93

Earlier quoted context omitted.

> Proper protection costs thousands Frankly that's something a landlord should provide. And there's insurance against losses from electrical issues.

Why should a domestic landlord provide you with data center-level power protection instead of just the normal household utility connection?

I'm talking about standard surge protectors. Properly installed they are enough except for direct lightning strikes, these will fry everything. But unfortunately, even in code-obsessed Germany landlords are not required to retrofit SPDs.

Re: Was my $48K GPU server worth it?

#199

Post by rnxrx »

The $48K also isn't fully sunk cost - there's a non-trivial residual value for those GPUs at the moment and likely for a few years yet. The server has a depreciation curve that's pretty enviable, actually!

Re: Was my $48K GPU server worth it?

#200

Post by fc417fc802 »

thefounder wrote: post #169
tracker1 wrote: post #147

Earlier quoted context omitted.

If you're in a decent sized city, you should be able to find a local buyer on Craigslist or FB Marketplace... Beyond that, for higher value, smaller items like your M3 Ultra, I would talk to your local police department and/or library to see if you can do the exchange there. Larger libraries usually have a police officer on site or nearby, and the PD office near you may also provide a "safe" exchange location... I'd…

Something is very wrong in some countries if you have to get police protection to sell a f* computer. I get it’s on the expensive side but still….

Not really. Every country has a nonzero number of criminals. It's entirely a matter of the risk/reward tradeoff. A small consumer item over $10k is well into dangerous territory.
Post reply on HN