Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT could cost over $700k per day to operate

businessinsider.com

Post reply on HN
Print view View original

191–200 of 223 posts

Re: ChatGPT could cost over $700k per day to operate

#191

Post by CuriouslyC »

marginalia_nu wrote: post #187
CuriouslyC wrote: post #102

Earlier quoted context omitted.

Probabilistic computing offers the potential of a return to that pace of progress. We spend a lot of silicon on squashing things to 0/1 with error correction, but using analog voltages to carry information and relying on parameter redundancy for error correction could lead to much greater efficiency both in terms of OPS/mm^2 and OPS/watt.

Seems very unrealistic when considering how electromagnetic interference works. Clamping the voltages to high and low goes some way to mitigate that problem.

That's only an issue if the interference is correlated.

Re: ChatGPT could cost over $700k per day to operate

#192

Post by samspenc »

miohtama wrote: post #3
aero-glide2 wrote: post #2

I would have never paid $100 per month for software but here i am doing the same with api costs.

Only 240,000 paying customers needed to cover $24M/mo server costs. I would assume personnel costs could be in the same class as well.

Around 375 employees per https://en.wikipedia.org/wiki/OpenAI

Let's say 500 employees, earning above Bay Area average, along with benefits etc at $400K average per year. That's $200 million a year in wages, so in the same ballpark as the $200 million in infrastructure costs cited in the article, for a total of $400 million in expenses per year.

Re: ChatGPT could cost over $700k per day to operate

#193

Post by samspenc »

twosdayz wrote: post #76
Dave3of5 wrote: post #9

Interesting numbers. That roughly equates to about $250 million per year plus I don't know how much training is costing them to keep the model up to date and suchlike. The company also has about 375 employees. I've no idea how much they get paid but I used $200k as a yearly cost and that comes to $75 million. That's about 3:1 cost of operating the services to paying employees. That seems quite high as I've never been…

Someone had posted their tax filings in a different OpenAI thread. Although this only starts at 2020, this may give some insight into their employee costs https://projects.propublica.org/nonprofits/organizations/810...

Interesting, the 2020 revenue and costs are significantly lower than previous years. Actually the prior years give a much better insight into salaries there. I wonder if this is because they switched from the non-profit model to the for-profit subsidiary at that time?

Re: ChatGPT could cost over $700k per day to operate

#194

Post by mcny »

rabuse wrote: post #130
rvnx wrote: post #77

Earlier quoted context omitted.

This is cheaper than running Twitter which is a CRUD app

Everything highly profitable is basically a CRUD app

You're absolutely right.

At the end of the day, all we are doing is create, read, update, and delete data.

I mean if you take away all the complexity associated with ranking and scale (not everything has to be Google scale), that is exactly what a web search is as well, right?

I remember reading a post by a maintainer saying their job is to manipulate strings or something to that effect. iirc it was a gofmt maintainer who said that but I can't find that post now.

Re: ChatGPT could cost over $700k per day to operate

#195

Post by JohnFen »

consp wrote: post #64
javajosh wrote: post #52

Earlier quoted context omitted.

It's almost like ChatGPT is nerd-sniping us all. But we like it and it's useful to us.

Don't generalize, I've never used it and never seen any reason to use it.

I've used it out of curiosity, but have no real use case for it.

Re: ChatGPT could cost over $700k per day to operate

#196

Post by 988747 »

kolinko wrote: post #169
pluijzer wrote: post #51

Earlier quoted context omitted.

As a side, I am a bit shocked by these numbers. Is this an American thing? I understand myself to be good software engineer with good well rounded experience of 14+ years. Yet my income, in Europe, is really above 100k. What I am wondering, for those earning 500k, how big is your work load/stress. Would this be a 9-5 job you leave at the office when going home. Or does a job that earns so much consume your life?

Taxes in the bay area can be insane - ~40% if I remember correctly. On top of that you have crazy-expensive healthcare, and crazy expensive housing costs. ~100k€ in (western) Europe may be comparable to ~200k€ in Bay Area.

[dead]

Re: ChatGPT could cost over $700k per day to operate

#198

Post by ben_w »

marginalia_nu wrote: post #186
Damogran6 wrote: post #92

Earlier quoted context omitted.

We said that in 1982, and 1987, and 1993, and 1995, and 2001, 2003, 2003.5 You get the point. There's always local optimization that leads to improvements. Look at the Apple M1 chip rollout as a prime example of that. Big/Little processors, on die RAM, shared memory with the GPU and Neural Engine, power integration with the OS. LOTS of things that led to a big leap forward.

Big difference now is that we have a clear inflection point. Die processes aren't getting much smaller than they are. A sub-nanometer process would involve arranging single digit counts of atoms into a transistor. A sub-Å process would involve single atom transistors. A sub 0.5Å process would mean making them out of subatomic particles. This isn't even possible in sci-fi. You can re-arrange them for minor boosts, dou…

Current process nodes are mostly 5nm, with 3nm getting rolled out. Atomic is ~0.1nm, which is x30 linear and x900 by area.

However, none of that is actually important when the thing people care about most right now is energy consumed per operation.

This metric dominates for anything battery powered for obvious reasons; less obvious to most is that it's also important for data centres where all the components need to be spread out so the air con can keep them from being damaged by their own heat.

I've noticed a few times where people have made unflattering comparisons between AI and cryptocurrency. One of the few that I would agree with is the power requirements are basically "as much as you can".

Because of that:

> double the performance a few times sure, but that's not a sustained improvement month upon month like we have in the past.

"Doubling a few times" is still huge, even if energy efficiency was perfectly tied to feature size.

But as I said before, the maximum limit for energy efficiency is in the order of a billion-fold, not the x900 limit in areal density, and even our own brains (which have the extra cost of being made of living cells that need to stay that way) are an existential proof it's possible to be tens of thousands of times more energy efficient.

Re: ChatGPT could cost over $700k per day to operate

#199

Post by hospitalJail »

rideontime wrote: post #153
hospitalJail wrote: post #88

Earlier quoted context omitted.

I have used it to help people eliminate food waste. My reach is about ~100k people max. I imagine people will be using this to create wasteful products, but also green solutions.

I, too, am curious how you used a fancier autocomplete to eliminate food waste.

It taught me about the '2 bin system' in ISE. Basically kitchen logistics.

Re: ChatGPT could cost over $700k per day to operate

#200

Post by hospitalJail »

Taywee wrote: post #120
hospitalJail wrote: post #88

Earlier quoted context omitted.

I have used it to help people eliminate food waste. My reach is about ~100k people max. I imagine people will be using this to create wasteful products, but also green solutions.

How have you used it to help people eliminate food waste?

It taught me about the '2 bin system' in ISE. Basically kitchen logistics.
Post reply on HN