This might sound tone deaf but thats seems extremely cheap to me. It means the cost range for chatGPT is between $300m - $500m per year. If the 100m users is accurate, it means they only need to convert low single digit percentage to paying users to break even. Now it makes sense why they chose to charge $20/m, I predicted much higher.
interesting that the article mentions Microsoft is trying to build their own AI chips to help reduce cost, they are years behind Google in that effort. Google still has the advantage in both research and hardware, they just fumbled the execution but it's still very early in the game
ChatGPT could cost over $700k per day to operate
111–120 of 223 posts
Re: ChatGPT could cost over $700k per day to operate
#112Earlier quoted context omitted.
You can use bing chat for free. Supposedly powered by gpt4.
Bing gives you much shorter answers, often without code examples
Re: ChatGPT could cost over $700k per day to operate
#113This sounds like a lot but I feel like optimizations are going to chop a zero off of that figure pretty quickly. For example, why not cache user prompt/response pairs and use cosine distance for key lookup? You could probably find a way to do this at the edge without a whole lot of suffering. I suspect many of the prompts that hit the public API are effectively the same thing over and over every day. Why let that kin…
because of context. You can cache it if it is indeed the first sentence of a fresh dialogue, but that's it.
Re: ChatGPT could cost over $700k per day to operate
#114Isn't Microsoft too late to the chip game? Both their competitors AWS and GCP have their own ML training chips, while Azure has nothing. The article seems like they may have something by next year, but by that time the competition would have evolved even more. Nadella has executed on strategy brilliantly but on the cloud seems it seems like the tech innovation is a bit lagging. It hasn't mattered that much so far tho…
It feels like Microsoft realized that sales/support matters way more than technology when it comes to cloud services. See how Azure has a ton of users despite being much worse than AWS or GCP by every objective metric. I would imagine that Google's reputation as a company where it is impossible to ever talk to a human (even when you have a 7 figure annual spend) hurts them in this space.
The problem for both of them is AWS, which somehow manages to give you your cake and let you eat it too, even if it's a little more expensive.
Re: ChatGPT could cost over $700k per day to operate
#115Earlier quoted context omitted.
No.
Agreed. I entered the workforce as a developer in 2012; it felt like the tail-end of the tech boom but intelligence was still valued. Nowadays, most people don't even recognize intelligence, let alone value it.
Funny to think all that was a generation ago, that the economical conditions for our artifice are now history and it's once again economical to study the much more all-encompassing art of not thinking. (It's very cyclical but, since it involves not thinking, i.e. the termination of a coherent thread of thought, you think it's for the first time every time. But sure, plug in to the simulation at 110% and get 10 years of life for free, you won't even notice the difference, after all, you're already not noticing it.)
This is how I, a unit of human capital from the former global East, see the matter of intelligence and automating it at scale: "intelligence" is already something that was forced onto my society in the 1950s, and once again on the 1990s, so that we could be interfaced with the same kinds of systems that the conquering populations exported, under conditions of forcible termination of local culture at the cost of great atrocities. By the time we started building our own Borg cube we'd already been half Borged by the Western one. If the word in some cases denotes another kind of abstract human "intelligence", I have rarely if ever seen such cases exemplified in public. So now it's a bit distasteful to equate "intelligence" with "savoir faire". Having neuromorphic computing in the first place would've saved a lot of suffering in comparison with how we were "civilized" to support only the established market protocols and nothing of our own.
Which is something someone will eventually have to admit - probably after a Pontypool-class scenario wipes out their investment in predictable outcomes for a region and they're left with a huge mess on their hands to mop up. And then it turns out the consequences themselves have been thought in such inhumanly fine detail as to possess a rudimentary intelligence of their own and the mop up ends up embedding a sentient malevolent ghost into the nature of that branch of reality. Shit happens, at a certain saturation with Internet stream of thought an omnipresent autoritarian AI even becomes fun to imagine. Of course if one existed its cruelty would be much more banal, as is always the difference between mentalization and reality until the next brief Golden Age (for some of the people, some of the time) and then it's "oh no this time we're fucked" again (for some other people).
So sad because there are so many great ideas worth building and fighting for, even in the current limits of human knowledge, but in practice any form of innovation is only tolerated if it's backed by a major financial effort by the upholders of the status quo. For some reason so many times this has ended to the detriment of nonparticipants that we've basically stopped noticing or keeping count. As regulatory capture, so learned helplessness. The economic reality of already existing in a world of inscrutable mechanisms as that most fragile of creatures, the independent knowledge worker, is a psychological stressor, to the extent that everyone's so burnt by the effort on giving up on things that in this new ecosystem are "too smart to work". Help, the AI stole my role in the food chain and now all my thoughts lead nowhere.
If that's you, look for a bigger room.
Re: ChatGPT could cost over $700k per day to operate
#116Re: ChatGPT could cost over $700k per day to operate
#117Earlier quoted context omitted.
Don't generalize, I've never used it and never seen any reason to use it.
You should give it a shot, keep an open mind. Yes, it will get things wrong and yes, it will disappoint you. But every conversation I've had with it holds some surprises for me. It's a tool you'll need to learn to use before it actually becomes useful , but give it a couple minutes a few times a week and you'll start to see the diamonds in the rough.
Re: ChatGPT could cost over $700k per day to operate
#118Earlier quoted context omitted.
Don't generalize, I've never used it and never seen any reason to use it.
You should give it a shot, keep an open mind. Yes, it will get things wrong and yes, it will disappoint you. But every conversation I've had with it holds some surprises for me. It's a tool you'll need to learn to use before it actually becomes useful , but give it a couple minutes a few times a week and you'll start to see the diamonds in the rough.
#include
then it is a must have.
Re: ChatGPT could cost over $700k per day to operate
#119Now I wonder what are the emissions. Is this technology net positive in long run for things like climate change?
Re: ChatGPT could cost over $700k per day to operate
#120Now I wonder what are the emissions. Is this technology net positive in long run for things like climate change?
I have used it to help people eliminate food waste. My reach is about ~100k people max. I imagine people will be using this to create wasteful products, but also green solutions.