Questions:
1) Was the energy bill factored in? 2) Have you extracted any comparable value out of this?
Was my $48K GPU server worth it?
361–370 of 480 posts
Re: Was my $48K GPU server worth it?
#362I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…
> don't seem to outperform Qwen3.6 that much in agentic coding/tasks
idk i imagine you'll hit less edges with a larger model just because.. more data
if you think of them as a kind of NN compression, it's ~obvious that the larger model can have more stuff encoded in it and hopefully accessible
i don't use LLMs much right now but using midrange models seems like an unnecessary compromise in most cases, especially since the big open models sound to be rivaling opus and not just sonnet :p
Re: Was my $48K GPU server worth it?
#363Earlier quoted context omitted.
The ones sold for $25k from established sellers are legit. Filter by "sold." The 0-reputation account in Spain selling an M3U 512GB for $4200 is 100% fraud.
oh young grasshopper, I see you dont know that money launderers love the ebay hype cycle. Its REALLY common on high dollar hot items to have phantom transactions where parties are on both sides of the transaction to clean illicit money. The high price tag and high volume amount of transactions hides the illicit signal. I have tried to buy a few of these mac studios only to have the transaction cancelled because I was…
Funny, they have the "honesty" to cancel the transaction and not take your money, just to keep their ebay reputation high ?
Re: Was my $48K GPU server worth it?
#364Earlier quoted context omitted.
Either I don't understand the used apple market.. or I agree this is crazy. Someone spends $25k on new hardware, waits a year, and expects to sell it for $23k? Unless the ram issues save him, and cost of new goes up, I don't see how that was going to work.
Apple products have had relatively high resale for a while. Only losing 8% in a year is probably extra unusual, and 1-year-old wasn't really ever the sweet spot, but a "sell used privately after a few years, roll onto the new one" has been a relatively common play. Doing this particular one is definitely expecting the market squeeze to continue. "Worst case" is back to more "normal" depreciation. Where I'd expect to…
> "Worst case" is back to more "normal" depreciation.
I would absolutely not count on that, if and when it drops it will drop hard.
Re: Was my $48K GPU server worth it?
#365Re: Was my $48K GPU server worth it?
#366Earlier quoted context omitted.
First it was crypto, now AI. Just because the market can stay irrational for very long doesn't mean crashes don't happen. What nobody knows is when.
at some point you have to accept that the market is actually rational
Yup, just like crypto and tulips were rational behaviors for the global economy. Or investing 5% of world GDP in half-baked AI is.
Re: Was my $48K GPU server worth it?
#367I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…
> our infosec department doesn't buy the "zero retention" promise They are wise to be skeptical! It is neither a promise nor zero data retention. Look at Anthropic's Zero Data Retention policy -- and remember, this is the policy that applies to the exclusively eligible enterprise partners who can even qualify for a ZDR agreement with Anthropic: > When ZDR is enabled, prompts and model responses generated during Claud…
I wonder how aws handles this in bedrock. Do they use Anthropics classifiers? Or their own? Or none? Would their data policing be different in bedrock than their other services?
Re: Was my $48K GPU server worth it?
#368I can't imagine spending $48K on a home GPU server, but I did just splurge and buy a PC with an RTX 5090, specifically to hold the largest models you can fit in 32GB. It's a top of the line PC with water cooled high end CPUs, 64GB RAM, RTX 5090 for $5K. To me the jury is still out whether this was a worthwhile investment, but I do expect to use this machine for a decade. I don't run it at 100% power (it's mostly idle…
Yes! It scared me too. I tried to insure it under my renter's insurance policy, but they not surprisingly refused. I had to get business insurance to cover it
You showed this setup to a business insurance underwriter and they gave you a policy? Can I ask how much the premium is? Or is this just theft insurance?
Re: Was my $48K GPU server worth it?
#369Earlier quoted context omitted.
at some point you have to accept that the market is actually rational
Yup, just like crypto and tulips were rational behaviors for the global economy. Or investing 5% of world GDP in half-baked AI is.
wanting to get rich quick is completely rational if you ask me ;)
Re: Was my $48K GPU server worth it?
#370In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…
I’m not usually one to ask this because learning to do a thing can be fun, but why exactly have you spent 25 thousand dollars on getting an LLM someone else made to answer maths exam questions?
This is making me feel a lot better about my plan to lease a $25k EV simply because it's available at a massive discount. I'll probably end up using less electricity, too.