Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was my $48K GPU server worth it?

rosmine.ai

Post reply on HN
Print view View original

251–260 of 480 posts

Re: Was my $48K GPU server worth it?

#251

Post by 0xbadcafebee »

mschuster91 wrote: post #198
beachy wrote: post #157

Earlier quoted context omitted.

Why should a domestic landlord provide you with data center-level power protection instead of just the normal household utility connection?

I'm talking about standard surge protectors. Properly installed they are enough except for direct lightning strikes, these will fry everything. But unfortunately, even in code-obsessed Germany landlords are not required to retrofit SPDs.

To protect a large electrical device investment, you would want an EMP shield whole-home SPD, in addition to an SPD right at the electrical device. The first one shields exterior surges (including non-terrestrial), but the second shields against internal surges. And yeah lightning will blast through both of them. So the best bet is probably a lightning strike detector combined with renters insurance.

Re: Was my $48K GPU server worth it?

#252

Post by fnordpiglet »

conductr wrote: post #241
selicos wrote: post #219

Earlier quoted context omitted.

Their more recent post seems to suggest it was worthwhile. https://rosmine.ai/2026/05/18/fixing-llm-writing-with-distri... Abstract/TLDR: LLMs are notoriously formulaic at writing, overusing certain tokens or phrases. I show that models trained with SFT fail to match the distribution of the training data by using Maximum Mean Discrepancy (MMD), Judge Model Quality (JMQ), and L2 Token Distribution.

Idk if this turns into revenue or some financial metric but even if it does and it was a good outcome for author, it still says nothing of risk. What if he loses his timing opportunity / gets beat to market because he's unnecessarily futzing around with hardware? AI is rapidly advancing and he spent 2 years on this to save what was probably <2 months of faang income. There's multiple other angles I could dissect this…

Let’s be clear, though, FAANG (as someone who has spent an awful lot of my life working at FAANG) was pays well but crushes your soul. There was a time, a very long time ago, where it didn’t, and there are the soulless soul crushers that love it there, but I would rather futz around with a mid range cars worth of hardware and be happy than spend a moment longer prostituting my soul for their money.

There was a time in this industry that it paid about as well as an accountant and people did it because they loved what they did. Then the money flooded in, a bunch of people switched majors from business to CS, washed out in industry, got their MBA, and became product managers and engineering managers and sucked all joy from it. God bless those that find that joy again.

Re: Was my $48K GPU server worth it?

#253

Post by jumploops »

komali2 wrote: post #208
jumploops wrote: post #192

Earlier quoted context omitted.

At the end of the article, the author has this to say: > UPDATE: Launch was a success! 400K+ views, and multiple companies reached to use my IP. Read more here[0] [0] https://rosmine.ai/2026/05/18/fixing-llm-writing-with-distri...

Post-hoc justification. There's no analysis of whether that level of hardware was necessary to launch, only that they did get that hardware and did launch.

Looking at the GPU utilization graph, it certainly seems like the hardware was saturated for many days/weeks on end.

Was it worth it to spend that amount up front, yak shave while building the system, etc. vs. pay for cloud GPUs? Probably not in terms of dollars, when their time is also valued in dollars.

Was it worth it for this person? It seems, unequivocally, yes.

Re: Was my $48K GPU server worth it?

#254

Post by dnnddidiej »

selicos wrote: post #219
caymanjim wrote: post #132

This article appears to lack any reason for "needing" this beast, or any real comparison with alternatives, both of which are required to answer the question posed in the title. It's a summary of how much they spent and some light anecdotal comparison to what they might have spent on cloud services, but clearly they didn't do an exhaustive hunt for value. The real question is whether or not they could have done whate…

Their more recent post seems to suggest it was worthwhile. https://rosmine.ai/2026/05/18/fixing-llm-writing-with-distri... Abstract/TLDR: LLMs are notoriously formulaic at writing, overusing certain tokens or phrases. I show that models trained with SFT fail to match the distribution of the training data by using Maximum Mean Discrepancy (MMD), Judge Model Quality (JMQ), and L2 Token Distribution.

That is like saying my new restaurant was a success, therefore powering it with a generator was better than connecting it to the grid.

The raw infra being local didn't enable any of that. Now if was building ASICs at TMSC that would a different thing because you'd then be using something different locally.

Re: Was my $48K GPU server worth it?

#255

Post by SJMG »

NiloCK wrote: post #247
HWR_14 wrote: post #246

The other advantage of the local GPU is that you are not feeding your data into cloud providers. I'm not sure how much you can really trust Anthropic and OpenAI not be improving their models based on your input.

Doesn't it benefit me if the models I use improve?

Do you value the infinitesimal improvement of model quality more than your privacy?

Re: Was my $48K GPU server worth it?

#256

Post by andoando »

girvo wrote: post #163
jon-wood wrote: post #14

Earlier quoted context omitted.

I’m not usually one to ask this because learning to do a thing can be fun, but why exactly have you spent 25 thousand dollars on getting an LLM someone else made to answer maths exam questions?

I didn't spend that much, only $6500 AUD for a GB10 based Asus GX10 which is even slower than OPs, but I spent that because it makes for a great learning platform. Theres not much else that lets me fiddle with 128GB of RAM for my graphics processor, and it's quite lovely to be able to run things as long as I like without worrying about my cloud instance being shut down. It's not financially a good idea: renting reall…

You could just rent a bare metal server with those specs

Re: Was my $48K GPU server worth it?

#257

Post by adrianN »

miki123211 wrote: post #227
freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

This is, sadly, obvious and inevitable in retrospect. The two major drivers of inference costs are GPUs and electricity. You can't get cheaper GPUs, but you can make existing GPUs not sit idle, and you do that by utilizing them 24/7, processing user B's request when user A is thinking, and handling many requests in parallel, neither of which you can do as an individual. You can get cheaper electricity... by moving, a…

Historically it was not uncommon for beds to be rented out to multiple people.

Re: Was my $48K GPU server worth it?

#258

Post by throwaway2037 »

freediddy wrote: post #10

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…

    > if I can find a reliable place to sell it without getting ripped off by scammers.
I don't follow this last part. What is the scam they try to run?

Re: Was my $48K GPU server worth it?

#260

Post by nullsanity »

andoando wrote: post #256
girvo wrote: post #163

Earlier quoted context omitted.

I didn't spend that much, only $6500 AUD for a GB10 based Asus GX10 which is even slower than OPs, but I spent that because it makes for a great learning platform. Theres not much else that lets me fiddle with 128GB of RAM for my graphics processor, and it's quite lovely to be able to run things as long as I like without worrying about my cloud instance being shut down. It's not financially a good idea: renting reall…

You could just rent a bare metal server with those specs

[dead]
Post reply on HN