People doing economics with the cloud GPUs, of course cloud GPUs are going cheaper. But also, is generating tokens all you do with your computer? I can play games on DGX spark and also do LLM inference, so sometimes the economics work out, apart from having fun with it.
Was my $48K GPU server worth it?
261–270 of 480 posts
Re: Was my $48K GPU server worth it?
#262Earlier quoted context omitted.
Well, Apple is literally not offering the M3 512GB studios currently. You can’t even back order one. They are selling on EBay for over $20k, used.
It’s hard to know if any of these eBay listing are real or actual sales. Lots of scams.
The ones sold for $25k from established sellers are legit. Filter by "sold."
The 0-reputation account in Spain selling an M3U 512GB for $4200 is 100% fraud.
Re: Was my $48K GPU server worth it?
#263Earlier quoted context omitted.
I’m not usually one to ask this because learning to do a thing can be fun, but why exactly have you spent 25 thousand dollars on getting an LLM someone else made to answer maths exam questions?
Because buying Macs is not about performance, its about feeling like you are rich. That money could have been spent on way more bang/buck performance in the form of a set of 4 graphics cards. Also I would probably put the odds 70:30 that Apple marketing is astroturfing on HN from the amount of posts about running llms on Macbooks, because in reality, the inference speed of any decent llm is unusable on a Macbook desp…
40-80 tok/s is unusable to you? Ok.
If you like having a box with 8-12 fans blasting hot air and noise into your office all day, nobody's stopping you.
Re: Was my $48K GPU server worth it?
#264In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…
> if I can find a reliable place to sell it without getting ripped off by scammers. I don't follow this last part. What is the scam they try to run?
A buyer can claim they never received it or that the box was empty, thus receiving a full refund.
For something listed at $25k I would not list on eBay at all. eBay corporate will pocket $3400 in fees and will also dock you local taxes on the $25k.
Re: Was my $48K GPU server worth it?
#265Earlier quoted context omitted.
Actually that's the best part of AI. It has access to experience with way more than the select sample size here.
I'm not entirely sure what your point is here; me asking for humans to give an opinion does not preclude me from also asking AI.
I was just making a correction based on what you said.
"AI is cool but it's not going to have all the good and bad experiences that humans have had with different motherboards."
AI will have more access to experiences than you'll find here.
Re: Was my $48K GPU server worth it?
#266I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…
> our infosec department doesn't buy the "zero retention" promise They are wise to be skeptical! It is neither a promise nor zero data retention. Look at Anthropic's Zero Data Retention policy -- and remember, this is the policy that applies to the exclusively eligible enterprise partners who can even qualify for a ZDR agreement with Anthropic: > When ZDR is enabled, prompts and model responses generated during Claud…
I think it’s a cost/opportunity tradeoff at best with any agreement, regardless. The rest of the contract may make it difficult to impossible to do anything about it, starting with basic arbitration clauses and ending in a ton of other provisions that can make any legal action futile. I doubt there’s much room to negotiate too.
Given that all labs need to diversify to become profitable, they’ll end up competing with their customers and theres nothing that exposes a business more than having AI offload every job function for every account, every mail etc.
Assuming this won’t be an issue is naive at best.
Re: Was my $48K GPU server worth it?
#267Quick tip for people who want to experiment with local models: A lot of the common smaller models are also available on openrouter or other services. Dirt cheap. I know it's not the same. But a lot of people buy expensive GPUs, just to find out they have no real use for smaller models.
Openrouter is great for experimenting with models. I did exactly what you're saying to test smaller models that will run on commodity hardware and determine if it might be worth it to drop $10k on hardware. For me the answer was no, but it's close. I'm very excited for the next few innovation cycles to arrive.
Re: Was my $48K GPU server worth it?
#268I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I'm impressed by Qwen3.6-27b's capabilities in agentic coding/tasks so far. Devs say it's not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok/sec, up to 260k context. The server cost about $10k with all the bells and whistles. I spent a lot of…
I have a 5090 machine sitting idle that I'm considering turning into a machine for my own small team (3 devs). Are you willing to share any lessons learned, etc. that I could make use of? We are evaluating paying for a SOTA sub or trying this, and the talk about Qwen3.6-27B makes me want to try deploying this machine.
Sell the machine for $4K, use it to pay for Codex Pro for everyone for a year. Everyone will be significantly more productive and happy.
It's not even a real comparison if they are actually using them for coding.
If you are deploying always running agents (e.g. monitoring logs and services) then sure - a QWEN local server is a good choice. But for coding the cost in productivity of using a lower performing model is way too high.
Re: Was my $48K GPU server worth it?
#269Earlier quoted context omitted.
Idk if this turns into revenue or some financial metric but even if it does and it was a good outcome for author, it still says nothing of risk. What if he loses his timing opportunity / gets beat to market because he's unnecessarily futzing around with hardware? AI is rapidly advancing and he spent 2 years on this to save what was probably <2 months of faang income. There's multiple other angles I could dissect this…
Let’s be clear, though, FAANG (as someone who has spent an awful lot of my life working at FAANG) was pays well but crushes your soul. There was a time, a very long time ago, where it didn’t, and there are the soulless soul crushers that love it there, but I would rather futz around with a mid range cars worth of hardware and be happy than spend a moment longer prostituting my soul for their money. There was a time i…
> would rather futz around with a mid range cars worth of hardware and be happy than spend a moment longer prostituting my soul for their money.
So only 2 options in this profession are 1) sell your soul to 1 of 5 evil corporations that just so happen to also pay excessively well or 2) choose to be unemployed for years while spending a significant amount of money on hardware trying to turn a hobby into a business
Also by your reasoning, these GPUs are blood diamonds and the authors future product/business should warrant preemptive boycott by all the perfect people like you
Re: Was my $48K GPU server worth it?
#270Earlier quoted context omitted.
Not really. Every country has a nonzero number of criminals. It's entirely a matter of the risk/reward tradeoff. A small consumer item over $10k is well into dangerous territory.
Are we talking about a cash transaction? If so >$10k is dangerous as the police may want to steal it themselves. If it is an electronic payment, I'm not sure how completing the transaction in front of a police station will help any. Well, it will help the buyer to see it working, but the seller gets no additional protection besides seeing "a person."
Why would the seller worry about ... himself? The seller worries that the buyer might not have any intention to pay for the small, expensive, easy to fence item to begin with. Conversely, the buyer worries that the seller might not have brought an item at all.