ChatGPT could cost over $700k per day to operate

Post by jerpint » Sat, Apr 22, 2023, 12:28 AM UTC

mnky9800n wrote: post #67

jermaustin1 wrote: post #30

Earlier quoted context omitted.

I have started incorporating ChatGPT into my side project. I'm not integrating it into software or anything like that, but using it as a tool for product ideas, product photo ideas, funny social media captions, figuring out the best hashtags to use, product tagging and categorization. It has proven to be a great way to get everything to around 70%, then send off to my assistant for the remaining 30% of polish. So at…

I paid for it, then stopped paying for it. It seemed to be completely novel and fun to use but after a while it just became a pain. I tried writing a library to act as a code assistant for data science but trying to get it to write the codes that were useful to me took around the same amount of time as writing them myself. What you say is completely correct, it gets to around 70% of whatever you ask it to do then you…

Post by dmix » Sat, Apr 22, 2023, 12:54 AM UTC

bob1029 wrote: post #179

dmix wrote: post #100

Earlier quoted context omitted.

It's not like a database you can cache though, the responses are non-deterministic, the responses you get can be different the next time you query it with the exact same prompt. That's part of the point of it being generative AI (vs a question/answer system that people imagine it is).

> That's part of the point of it being generative AI (vs a question/answer system that people imagine it is). The point I am trying to make is that not all use cases for ChatGPT are generative. There are a lot of Q&A use cases today despite the fact that these are so far beneath its true capabilities. These items could be dealt with using more economical means. "give me a recipe for XYZ" should not require a GPU for…

Post by gg82 » Sat, Apr 22, 2023, 2:10 AM UTC

marginalia_nu wrote: post #59

reitanqild wrote: post #55

Earlier quoted context omitted.

> 10% here and there is very small compared to the literal orders magnitude improvements during the reign of Moore's Law. I can't confirm it, but I noticed this comment says "gpu tech has beat Moore’s law for DNNs the last several years": https://news.ycombinator.com/item?id=35653231

We're actually at an inflection point where this isn't the case anymore. For a long time, GPU hardware basically became more powerful with each generation, but prices stayed roughly the same plus minus inflation. Last couple of years, this trend has broken. You pay double or even quadruple the price for a relatively tenuous increase in performance.

Post by kristianp » Sat, Apr 22, 2023, 3:36 AM UTC

Post by mnky9800n » Sat, Apr 22, 2023, 5:22 PM UTC

qumpis wrote: post #218

mnky9800n wrote: post #208

Earlier quoted context omitted.

I didn't see any difference between 3.5 turbo and 4 for the data science app that I wrote. Another thing is that it's stochastic. So you could actually get a prompt that works well then you run it again and the identical prompt that generated codes that work now has generated codes that don't work. Of course, I could have my data science app cache codes that worked so if the prompt doesn't change it already has the g…

In my experience 3.5 is so far away from 4 that it makes it makes 3.5 useless. I'd give another shot to gpt 4 if 3.5 was almost working for you (what did you use it for, anything for specific?)