missing from most of these cost discussions: privacy. for some workloads the entire value of local is zero data leaving the network, and cloud cost is irrelevant
Was my $48K GPU server worth it?
401–410 of 480 posts
Re: Was my $48K GPU server worth it?
#402out of curiosity, did you check how much would cost to rent a cage in a colocation space? Having to power your computer from two different outlets sounds wild..
The cheaper, easier solution would probably be just to get an electrician to wire up a high amperage 240V outlet just like your electric stove or dryer has, and then get a PSU that connects to that. Would probably cost you $500-1000 depending on how difficult your home is.
The article stated that this was dismissed because the author lives in a rental apartment and he was not certain the landlord would agree to making this change.
I did not see any indication that the landlord was ever actually asked, it appeared to be the author's "sense" that any answer would be "no" from the landlord.
Re: Was my $48K GPU server worth it?
#403In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment. In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I'v…
I'm not really asking this from the perspective of whether I should buy hardware. I'm trying to understand the economics. The AI space is moving so fast that it is hard to know which conclusions are stable. After all the discussion around local models, is the practical conclusion still that API/frontier providers have a huge structural advantage because of datacenter hardware, high utilization, batching, optimized in…
The speed of results for an API call to ChatGPT is 10-100x faster than my local LLM. I haven't exactly quantified the results but I was getting results in a few seconds vs 10+ minutes for my local LLM. I'm going to do a deep dive this weekend and try to get better results, but it was staggering. I'll also do a deep dive on how to optimize my setup and see if I can get things to perform much quicker.
Re: Was my $48K GPU server worth it?
#404Earlier quoted context omitted.
Are they? I only ever see unsubstantiated claims for this whereas I see many justifications that interference is comfortably profitable in isolation.
Its basic math, go calculate max sessions for a certain tps on any hardware. Session# * tps * 86400 (secs in a day) * 30 days. You'll realize real quick its not profitible. You cant just say things you don't like to hear are unsubstantiated without verifying. Not to mention, subscriptions.. $2mm in GPUs being given out for 5 hrs a day at a cost of $200 a month. I could easily say that everyone who says its profitible…
To some degree I think there's a hope that it becomes like a gym membership. If everybody used their membership, the gym would be too crowded. It's all of those memberships that people feel like they need to have but don't use where the extra profit comes in.
As long as the power users are paying per token, everything is good.
Re: Was my $48K GPU server worth it?
#405> I thought that I could not get a standard datacenter server because my apartment wouldn’t let me upgrade the circuits, so I needed to have 2 power supplies plugged into different circuits. Why didn't they just put a higher amp breaker in the box?
> Why didn't they just put a higher amp breaker in the box?
1) note the word "apartment" -- they rent, not own, and doing so not only would likely be illegal, but might also get them kicked out of the apartment.
2) Unless the wiring on the circuit drop, and all the end points are rated to handle the higher current, doing so would be an electrical code violation (and therefore trip into that "illegal" arena that might result in getting kicked out of the apartment).
Most residences are wired using the minimum size wire rated for the installed breaker (because doing so saves costs). So a 15a breaker in the box would mean 14gauge (the US NEC minimum size for 15a circuits) wiring in the walls and 15a rated outlets/switches. Installing a 20a breaker in the box would be a code violation, and in many jurisdictions also illegal.
And all the above is without considering that installing a 20a breaker on wires rated for 15a increases the fire risk tremendously if those wires are now asked to actually carry 20a for any length of time.
Re: Was my $48K GPU server worth it?
#406Earlier quoted context omitted.
On top of that, AI providers are also eating a big loss on the service.
Are they? I only ever see unsubstantiated claims for this whereas I see many justifications that interference is comfortably profitable in isolation.
Especially since their costs might be multi-year investments. It's too early to judge the quality of those investments.
Re: Was my $48K GPU server worth it?
#407Earlier quoted context omitted.
Not really. Every country has a nonzero number of criminals. It's entirely a matter of the risk/reward tradeoff. A small consumer item over $10k is well into dangerous territory.
Are we talking about a cash transaction? If so >$10k is dangerous as the police may want to steal it themselves. If it is an electronic payment, I'm not sure how completing the transaction in front of a police station will help any. Well, it will help the buyer to see it working, but the seller gets no additional protection besides seeing "a person."
I love that it never occurred to you that the "buyer" could just steal the item. Be safe out there.
Re: Was my $48K GPU server worth it?
#408Earlier quoted context omitted.
A cage[0] is ~100x larger than what you need to host a single server. Many data centers will colocate by the rack unit. At others you can get a quarter or half cabinet[1]. Even at the very largest enterprise datacenters you can colocate a single cabinet. [0]: https://static.cisco-eagle.com/images/category/WireCrafters/... [1]: https://www.edpeurope.com/wp-content/uploads/EDP-3-Compartme...
> A cage[0] is ~100x larger than what you need to host a single server. Yup, but i was assuming that he wanted to experiment building gpu rigs. For sure standard GPU servers are cheaper and easy to maintain. I have two lenovos, bought them used, already EOL.. was cheap and better than any custom gpu rig.. but i was pragmatic, because my goal was to put it in production, and not to research...
You could fit 10 million dollars of GPU rigs in the smallest of cages. A cage is an entire room. You don't need that to run a few servers.
Re: Was my $48K GPU server worth it?
#409[dead]
Re: Was my $48K GPU server worth it?
#410Earlier quoted context omitted.
Because buying Macs is not about performance, its about feeling like you are rich. That money could have been spent on way more bang/buck performance in the form of a set of 4 graphics cards. Also I would probably put the odds 70:30 that Apple marketing is astroturfing on HN from the amount of posts about running llms on Macbooks, because in reality, the inference speed of any decent llm is unusable on a Macbook desp…
Or it could have had way more bang/buck by feeding a family of real brains for a year or two
Excuse me for this comment, really, but I can't comprehend the absurdity, some people are buying GPUs when other people have no money for insulin so they literally die. I don't mean anything towards op or gp, quite the opposite I'm truly happy they have this kind of freedom, it must feel really nice, I just hate this game so much.