Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT could cost over $700k per day to operate

businessinsider.com

Post reply on HN
Print view View original

61–70 of 223 posts

Re: ChatGPT could cost over $700k per day to operate

#61

Post by crop_rotation »

sebzim4500 wrote: post #32
crop_rotation wrote: post #5

Isn't Microsoft too late to the chip game? Both their competitors AWS and GCP have their own ML training chips, while Azure has nothing. The article seems like they may have something by next year, but by that time the competition would have evolved even more. Nadella has executed on strategy brilliantly but on the cloud seems it seems like the tech innovation is a bit lagging. It hasn't mattered that much so far tho…

It feels like Microsoft realized that sales/support matters way more than technology when it comes to cloud services. See how Azure has a ton of users despite being much worse than AWS or GCP by every objective metric. I would imagine that Google's reputation as a company where it is impossible to ever talk to a human (even when you have a 7 figure annual spend) hurts them in this space.

Yes, but that strategy works due to how entrenched they were in the enterprise already. It won't work for a brand new vendor. Almost every big company uses some enterprise Windows product (Windows,Office,AD, SQLServer).

Once people already are buying 20 products from you and have a good sales relationship of decades, selling them some cloud services is easy and might even lead to better deals on something else (Windows/Office)

Re: ChatGPT could cost over $700k per day to operate

#62

Post by capableweb »

gabereiser wrote: post #40
capableweb wrote: post #20

Earlier quoted context omitted.

Considering that Microsoft is a huge investor in OpenAI, I'd be surprised they pay anything at all in reality.

that's ridiculous, OpenAI is paying. Granted Microsoft invested heavily into OpenAI but those are two separate financial transactions. Sure you can rationalize in your head that IN-OUT=DIFF but that's not how books are kept.

Why is that ridiculous? Cloud services gives companies "coupons" and free usage for X hours for a bunch of other companies, why wouldn't they do that for a company they invested heavily in?

Re: ChatGPT could cost over $700k per day to operate

#63

Post by darkwizard42 »

pluijzer wrote: post #51
pdksam wrote: post #12

Earlier quoted context omitted.

200k is too small, strong sde1s at Amazon get paid that much in hcol areas. Closer to 500k.

As a side, I am a bit shocked by these numbers. Is this an American thing? I understand myself to be good software engineer with good well rounded experience of 14+ years. Yet my income, in Europe, is really above 100k. What I am wondering, for those earning 500k, how big is your work load/stress. Would this be a 9-5 job you leave at the office when going home. Or does a job that earns so much consume your life?

Honestly, depends. Some teams at FAAMNG are really stressful and if you work on a Tier 1 service even with loads of SRE support you have to be working a fair bit. That being said, the pay is for design decisions at the higher IC level (senior or staff) and most people at that level are very smart. I’m not saying this salary is for 10x engineers or anything.

I would say 50% the work is harder and consuming and then 50% they can just afford to pay you more and lock up talent because of the wild margins on their products.

Re: ChatGPT could cost over $700k per day to operate

#64

Post by consp »

javajosh wrote: post #52
jermaustin1 wrote: post #30

Earlier quoted context omitted.

I have started incorporating ChatGPT into my side project. I'm not integrating it into software or anything like that, but using it as a tool for product ideas, product photo ideas, funny social media captions, figuring out the best hashtags to use, product tagging and categorization. It has proven to be a great way to get everything to around 70%, then send off to my assistant for the remaining 30% of polish. So at…

It's almost like ChatGPT is nerd-sniping us all. But we like it and it's useful to us.

Don't generalize, I've never used it and never seen any reason to use it.

Re: ChatGPT could cost over $700k per day to operate

#65

Post by justinclift »

pluijzer wrote: post #51
pdksam wrote: post #12

Earlier quoted context omitted.

200k is too small, strong sde1s at Amazon get paid that much in hcol areas. Closer to 500k.

As a side, I am a bit shocked by these numbers. Is this an American thing? I understand myself to be good software engineer with good well rounded experience of 14+ years. Yet my income, in Europe, is really above 100k. What I am wondering, for those earning 500k, how big is your work load/stress. Would this be a 9-5 job you leave at the office when going home. Or does a job that earns so much consume your life?

Amazon has a terrible reputation for internal infrastructure issues, with "on call" being a truly shitty experience for employees. aka burn out over a year is common

Note that there's likely to be some variation per team, but Amazon is famously bad, so ... ;)

Re: ChatGPT could cost over $700k per day to operate

#66

Post by ben_w »

marginalia_nu wrote: post #42
ben_w wrote: post #34

Earlier quoted context omitted.

The various open source LLMs are doing things like reducing bits-per-parameter to reduce hardware requirements; if they're using COTS hardware it almost certainly isn't optimised for their specific models; Moore's Law is pretty heavily reinterpreted, so although we normally care about "operations per second at a fixed number of monies" what matters here is "joules per operation" which can improve a by a huge margin e…

Moore's law was an effect that stemmed from the locally exponential efficiency increase from designing computers using computers, each iteration growing more powerful and capable of designing still more powerful hardware. 10% here and there is very small compared to the literal orders magnitude improvements during the reign of Moore's Law. I don't really see anything like that here.

> 10% here and there is very small compared to the literal orders magnitude improvements during the reign of Moore's Law.

Missing the point, despite being internally correct: 10% of $700k/day is still $25M/y.

If you'd instead looked at my point about energy cost per operation, there's room for something like 46,000 improvement just to human level, and 5.3e9 to the Landauer limit.

Re: ChatGPT could cost over $700k per day to operate

#67

Post by mnky9800n »

jermaustin1 wrote: post #30
pc_edwin wrote: post #15

This might sound tone deaf but thats seems extremely cheap to me. It means the cost range for chatGPT is between $300m - $500m per year. If the 100m users is accurate, it means they only need to convert low single digit percentage to paying users to break even. Now it makes sense why they chose to charge $20/m, I predicted much higher.

I have started incorporating ChatGPT into my side project. I'm not integrating it into software or anything like that, but using it as a tool for product ideas, product photo ideas, funny social media captions, figuring out the best hashtags to use, product tagging and categorization. It has proven to be a great way to get everything to around 70%, then send off to my assistant for the remaining 30% of polish. So at…

I paid for it, then stopped paying for it. It seemed to be completely novel and fun to use but after a while it just became a pain. I tried writing a library to act as a code assistant for data science but trying to get it to write the codes that were useful to me took around the same amount of time as writing them myself. What you say is completely correct, it gets to around 70% of whatever you ask it to do then you have to finish the job for it. Which I guess is nice. But I'm not going to pay 20$/month for something that i can just use for free that amounts to slightly better intellisense.

For text generation, it is much better for tasks like

* letter writing * rewriting my writing so i don't plaigerize myself * summarizing several paragraphs into one

but all of that is available in the free version.

Re: ChatGPT could cost over $700k per day to operate

#68

Post by ElFitz »

warent wrote: post #49
nathancahill wrote: post #41

Earlier quoted context omitted.

Yeah think of all the labor hours, how many people do they have back there typing up the answers?

And who pays the tiny people in my magic light box to draw the answers on the window?!

Those don’t count. They’re fairies, not people. Why on Earth would you pay fairies?

Re: ChatGPT could cost over $700k per day to operate

#69

Post by tilne »

warent wrote: post #49
nathancahill wrote: post #41

Earlier quoted context omitted.

Yeah think of all the labor hours, how many people do they have back there typing up the answers?

And who pays the tiny people in my magic light box to draw the answers on the window?!

Do you know the answer?

Re: ChatGPT could cost over $700k per day to operate

#70

Post by Swizec »

javajosh wrote: post #52
jermaustin1 wrote: post #30

Earlier quoted context omitted.

I have started incorporating ChatGPT into my side project. I'm not integrating it into software or anything like that, but using it as a tool for product ideas, product photo ideas, funny social media captions, figuring out the best hashtags to use, product tagging and categorization. It has proven to be a great way to get everything to around 70%, then send off to my assistant for the remaining 30% of polish. So at…

It's almost like ChatGPT is nerd-sniping us all. But we like it and it's useful to us.

Nerd-snipe and useful? That’s the holy grail! We’re having fun and we’re solving real problems that we have. Amazing.

I’ve already incorporated it into my publishing process. My home-grown “cms” uses ChatGPT (via API) to write my article description, draft a twitter thread, and craft a “viral insight”. The latter is mostly useful to make sure my article even makes a point.

Hoping to use it for a related articles feature next. I’m also building a chatbot based on my content.

Post reply on HN