It’s worth every cent. Better than wasting the same amount on Bitcoin mining.
ChatGPT could cost over $700k per day to operate
31–40 of 223 posts
Re: ChatGPT could cost over $700k per day to operate
#32Isn't Microsoft too late to the chip game? Both their competitors AWS and GCP have their own ML training chips, while Azure has nothing. The article seems like they may have something by next year, but by that time the competition would have evolved even more. Nadella has executed on strategy brilliantly but on the cloud seems it seems like the tech innovation is a bit lagging. It hasn't mattered that much so far tho…
It feels like Microsoft realized that sales/support matters way more than technology when it comes to cloud services. See how Azure has a ton of users despite being much worse than AWS or GCP by every objective metric.
I would imagine that Google's reputation as a company where it is impossible to ever talk to a human (even when you have a 7 figure annual spend) hurts them in this space.
Re: ChatGPT could cost over $700k per day to operate
#33Interesting numbers. That roughly equates to about $250 million per year plus I don't know how much training is costing them to keep the model up to date and suchlike. The company also has about 375 employees. I've no idea how much they get paid but I used $200k as a yearly cost and that comes to $75 million. That's about 3:1 cost of operating the services to paying employees. That seems quite high as I've never been…
Pretty sure he said the days of models getting larger were over. Not that LLMs we’re over
Re: ChatGPT could cost over $700k per day to operate
#34Earlier quoted context omitted.
Moore's law-ish like optimization. You Z80 computer cost $700 in the lat 70's...they're now in sub-$1 embedded controllers.
But what is being optimized? Hardware sure isn't getting faster in a hurry, and I don't see anything on the horizon that will aid in optimizing software.
The various open source LLMs are doing things like reducing bits-per-parameter to reduce hardware requirements; if they're using COTS hardware it almost certainly isn't optimised for their specific models; Moore's Law is pretty heavily reinterpreted, so although we normally care about "operations per second at a fixed number of monies" what matters here is "joules per operation" which can improve a by a huge margin even before human level, which itself appears to be a long way from the limits of the laws of physics; and even if we were near the end of Moore's Law and there was only a 10% total improvement available, that's 10% of a big number.
Re: ChatGPT could cost over $700k per day to operate
#35Isn't Microsoft too late to the chip game? Both their competitors AWS and GCP have their own ML training chips, while Azure has nothing. The article seems like they may have something by next year, but by that time the competition would have evolved even more. Nadella has executed on strategy brilliantly but on the cloud seems it seems like the tech innovation is a bit lagging. It hasn't mattered that much so far tho…
Microsoft have a pretty considerable advantage - a valuable production workload to optimise and strong knowledge of what that looks like.
Sometimes it's better to be the fast follower.
Re: ChatGPT could cost over $700k per day to operate
#36This might sound tone deaf but thats seems extremely cheap to me. It means the cost range for chatGPT is between $300m - $500m per year. If the 100m users is accurate, it means they only need to convert low single digit percentage to paying users to break even. Now it makes sense why they chose to charge $20/m, I predicted much higher.
What does it cost to run per day versus what did it cost to train?
Re: ChatGPT could cost over $700k per day to operate
#37My main concern with AI is that I'm not so sure if there is a market for any kind of intelligence anymore. So much human talent and intelligence has been wasted over the past few years; so many potentially useful projects went unfunded, I struggle to think of what purpose AI would serve in such an economic system as ours. I can see its potential, but I can't see our current system facilitating that potential. It seem…
> these days Wasn't it always more or less like this?
No.
Re: ChatGPT could cost over $700k per day to operate
#38This might sound tone deaf but thats seems extremely cheap to me. It means the cost range for chatGPT is between $300m - $500m per year. If the 100m users is accurate, it means they only need to convert low single digit percentage to paying users to break even. Now it makes sense why they chose to charge $20/m, I predicted much higher.
That's just server costs, not labor hours for support, management & innovation.
Re: ChatGPT could cost over $700k per day to operate
#39It’s worth every cent. Better than wasting the same amount on Bitcoin mining.
[flagged]
Re: ChatGPT could cost over $700k per day to operate
#40Earlier quoted context omitted.
They're interesting numbers, but the linked article's cite amounts to: > ChatGPT could cost OpenAI up to $700,000 a day to run due to "expensive servers," an analyst told The Information. which, pardon me, but no shit. Before I break out my back of the envelope calculator, on how many biggest GPU instances in Azure that is, the real question is what their underlying assumptions are, and where they're getting them fro…
Considering that Microsoft is a huge investor in OpenAI, I'd be surprised they pay anything at all in reality.
that's ridiculous, OpenAI is paying. Granted Microsoft invested heavily into OpenAI but those are two separate financial transactions. Sure you can rationalize in your head that IN-OUT=DIFF but that's not how books are kept.