Earlier quoted context omitted.
I like how you abbreviate "Microsoft" as "M$". It means you think of them without bias and with a clear head. /s
M$ has been a rather common abbreviation for Microsoft since at least slashdot days.
Bill Gates as Borg.
ChatGPT could cost over $700k per day to operate
181–190 of 223 posts
Re: ChatGPT could cost over $700k per day to operate
#182Interesting numbers. That roughly equates to about $250 million per year plus I don't know how much training is costing them to keep the model up to date and suchlike. The company also has about 375 employees. I've no idea how much they get paid but I used $200k as a yearly cost and that comes to $75 million. That's about 3:1 cost of operating the services to paying employees. That seems quite high as I've never been…
200k is too small, strong sde1s at Amazon get paid that much in hcol areas. Closer to 500k.
These numbers are insane to me.
I'm 20 years into programming and a senior architect and lead on an enterprise project.
I don't even make that first number.
But I value certain things way more than other things, and my current job provides it. Fully remote, leaves me completely alone to accomplish what they need done (and I get it done), unlimited vacation, great benefits, zero pointless meetings (almost an empty calendar).
I'm sure these other companies offer some of that but 500k?! That is absurd.
Re: ChatGPT could cost over $700k per day to operate
#183Earlier quoted context omitted.
Moore's law-ish like optimization. You Z80 computer cost $700 in the lat 70's...they're now in sub-$1 embedded controllers.
But what is being optimized? Hardware sure isn't getting faster in a hurry, and I don't see anything on the horizon that will aid in optimizing software.
> Hardware sure isn't getting faster in a hurry
How is it not?
These LLMs were recently trained using NVidia A100 GPUs.
Now NVidia has H100 GPUs.
The H100 is up to nine times faster for AI training and 30 times faster for inference than the A100.
Re: ChatGPT could cost over $700k per day to operate
#184Earlier quoted context omitted.
I have started incorporating ChatGPT into my side project. I'm not integrating it into software or anything like that, but using it as a tool for product ideas, product photo ideas, funny social media captions, figuring out the best hashtags to use, product tagging and categorization. It has proven to be a great way to get everything to around 70%, then send off to my assistant for the remaining 30% of polish. So at…
I paid for it, then stopped paying for it. It seemed to be completely novel and fun to use but after a while it just became a pain. I tried writing a library to act as a code assistant for data science but trying to get it to write the codes that were useful to me took around the same amount of time as writing them myself. What you say is completely correct, it gets to around 70% of whatever you ask it to do then you…
Yes, some of the transformation mechanisms are nice if you need to dump info into it and want it reorganized or processed into something else as well. But outside of that it's hard to justify spending $20 month so you can be bullshitted by some know-it-all toddling proto-skynet when you need factual information.
Re: ChatGPT could cost over $700k per day to operate
#185My main concern with AI is that I'm not so sure if there is a market for any kind of intelligence anymore. So much human talent and intelligence has been wasted over the past few years; so many potentially useful projects went unfunded, I struggle to think of what purpose AI would serve in such an economic system as ours. I can see its potential, but I can't see our current system facilitating that potential. It seem…
There is definitely a market for an intelligence crutch or assisted critical-thinking aid. Like people slowly getting more obese, the general IQ and CT have been sliding for the last few decades. Not sure if it’s the food, media or zanax but society needs something. I’d wager the Altmans of the world know this and figured out a way to monetize it. Necessity is the mother of invention after all.
What is "CT" ?
Re: ChatGPT could cost over $700k per day to operate
#186Earlier quoted context omitted.
We're actually at an inflection point where this isn't the case anymore. For a long time, GPU hardware basically became more powerful with each generation, but prices stayed roughly the same plus minus inflation. Last couple of years, this trend has broken. You pay double or even quadruple the price for a relatively tenuous increase in performance.
We said that in 1982, and 1987, and 1993, and 1995, and 2001, 2003, 2003.5 You get the point. There's always local optimization that leads to improvements. Look at the Apple M1 chip rollout as a prime example of that. Big/Little processors, on die RAM, shared memory with the GPU and Neural Engine, power integration with the OS. LOTS of things that led to a big leap forward.
Big difference now is that we have a clear inflection point. Die processes aren't getting much smaller than they are. A sub-nanometer process would involve arranging single digit counts of atoms into a transistor. A sub-Å process would involve single atom transistors. A sub 0.5Å process would mean making them out of subatomic particles. This isn't even possible in sci-fi.
You can re-arrange them for minor boosts, double the performance a few times sure, but that's not a sustained improvement month upon month like we have in the past.
As anyone who has ever optimized code will attest, optimization within fixed constraints typically hits diminishing returns very quickly. You have to work harder and harder for every win, and the wins get smaller and smaller.
Re: ChatGPT could cost over $700k per day to operate
#187Earlier quoted context omitted.
This would be optimizations, which is not really the same thing as moore's law-like growth which was absolutely mind-boggling, like it's hard to even wrap your head around how fast tech was moving in that period since humans don't really grok exponentials too well, we just think they look like second degree polynomials.
Probabilistic computing offers the potential of a return to that pace of progress. We spend a lot of silicon on squashing things to 0/1 with error correction, but using analog voltages to carry information and relying on parameter redundancy for error correction could lead to much greater efficiency both in terms of OPS/mm^2 and OPS/watt.
Seems very unrealistic when considering how electromagnetic interference works. Clamping the voltages to high and low goes some way to mitigate that problem.
Re: ChatGPT could cost over $700k per day to operate
#188Earlier quoted context omitted.
My habit is to cancel right away when I start a subscription. You can still use the service for the month you paid for, and it's easy to restart it if you do still find yourself attempting to use it a month later.
Substack disappointed me because you can't do this. They remove content access if you do not have an "active" subscription. You don't have to pay again to "reactivate", but the paid content is locked until you do. This is obviously intended dark behavior because when I went to fully unsub later in the month they gave me a special dialog and a free month hoping for me to forget again this month. Very lame.
Oh, you haven't seen darker behavior of many Japan based services:
- Payment cycle closes always on the 1st of the month (which means, if you sign up on the last day of the month, you get 1 day of service for full month of payment. No proration, or anything.)
- If you cancel the service, you'll immediately lose access, doesn't matter if it's 1st day of the month, or very end of the month, it's gone. To reenroll, you often have to pay again.
Re: ChatGPT could cost over $700k per day to operate
#189Earlier quoted context omitted.
We're actually at an inflection point where this isn't the case anymore. For a long time, GPU hardware basically became more powerful with each generation, but prices stayed roughly the same plus minus inflation. Last couple of years, this trend has broken. You pay double or even quadruple the price for a relatively tenuous increase in performance.
That's not true. You can buy Raspberry PI, which is 10x cheaper and 10x more powerful than the computers at the beginning of 2000s. Ditto with mobile phones. iPhone may be more expensive than when it launched, but you can buy dirt-cheap chinese smartphones that have similar performance - if not higher to the first iPhones.
I don't think this contradicts what I'm saying. This is happening now. Not 15 years ago.
Re: ChatGPT could cost over $700k per day to operate
#190This might sound tone deaf but thats seems extremely cheap to me. It means the cost range for chatGPT is between $300m - $500m per year. If the 100m users is accurate, it means they only need to convert low single digit percentage to paying users to break even. Now it makes sense why they chose to charge $20/m, I predicted much higher.
This is cheaper than running Twitter which is a CRUD app
[flagged]