Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT could cost over $700k per day to operate

businessinsider.com

Post reply on HN
Print view View original

131–140 of 223 posts

Re: ChatGPT could cost over $700k per day to operate

#131

Post by alex_sf »

kylecazar wrote: post #90
BlackSwanMan wrote: post #81

Earlier quoted context omitted.

The cheap version does all of that much worse. I don't understand how people can be so cheap.

There are so many subscription products these days that it makes sense to assess the value of each carefully before committing. If you're like me you tend to forget you're paying for something, and have to do a yearly purge. It's very easy to slowly but surely rack up hundreds of dollars a month in subscription services.

I've been using Privacy.com cards for years to avoid this. Set a limit of 1 month on the card, and if I'm actually using the sub I'll fix/update it when the sub runs out.

Re: ChatGPT could cost over $700k per day to operate

#132

Post by vitus »

reaperman wrote: post #101
zamnos wrote: post #43

Earlier quoted context omitted.

Why wouldn't inference follow diurnal patterns?

Agreed. That seems backwards. Training would not follow circadian rhythms, inference would.

I should clarify: training is not latency sensitive, so you can run your workloads at off-peak hours when people are asleep. Inference means you need to run your workloads at peak when people are using your service.

(Looking back, I'm happy that I was careful in my wording in that I didn't say diurnal cycles aren't relevant, just that they aren't as useful in this case)

That said, I suppose I misread the specific suggestion about spinning down servers off-peak and was thinking more about spot pricing at peak vs trough.

Re: ChatGPT could cost over $700k per day to operate

#133

Post by bobsmooth »

djschnei wrote: post #39
xyzzy4747 wrote: post #31

It’s worth every cent. Better than wasting the same amount on Bitcoin mining.

[flagged]

Privilege is every second not hunting and gathering for our food. Fun stuff like GPT or bitcoin is what keeps life interesting.

Re: ChatGPT could cost over $700k per day to operate

#135

Post by edgyquant »

behnamoh wrote: post #58
ren_engineer wrote: post #46

Earlier quoted context omitted.

interesting that the article mentions Microsoft is trying to build their own AI chips to help reduce cost, they are years behind Google in that effort. Google still has the advantage in both research and hardware, they just fumbled the execution but it's still very early in the game

The past 3 years showed that Google is not as ahead as people thought they were.

In what way?

Re: ChatGPT could cost over $700k per day to operate

#136

Post by Workaccount2 »

maxfurman wrote: post #72
pluijzer wrote: post #51

Earlier quoted context omitted.

As a side, I am a bit shocked by these numbers. Is this an American thing? I understand myself to be good software engineer with good well rounded experience of 14+ years. Yet my income, in Europe, is really above 100k. What I am wondering, for those earning 500k, how big is your work load/stress. Would this be a 9-5 job you leave at the office when going home. Or does a job that earns so much consume your life?

FAANG salaries are so bloated because Bay Area housing costs are insane. Someone making 500k could put half or more of into their mortgage. I've said it before on here, but I live very comfortably in Philly for a lot less than that.

I'd argue it's the opposite. We're coming off a decade of free money driving a second tech boom.

If interest rates stay elevated, and value investing becomes valuable again, it will be interesting to see how the tech space transforms. When start-ups have to compete with money market funds or even treasuries for investor cash, things become orders of magnitude tighter and more challenging.

Re: ChatGPT could cost over $700k per day to operate

#138

Post by moffkalast »

jongjong wrote: post #56
pohl wrote: post #37

Earlier quoted context omitted.

No.

Agreed. I entered the workforce as a developer in 2012; it felt like the tail-end of the tech boom but intelligence was still valued. Nowadays, most people don't even recognize intelligence, let alone value it.

Well you should be just fine then /s

Re: ChatGPT could cost over $700k per day to operate

#139

Post by phkahler »

octopoc wrote: post #71
jermaustin1 wrote: post #30

Earlier quoted context omitted.

I have started incorporating ChatGPT into my side project. I'm not integrating it into software or anything like that, but using it as a tool for product ideas, product photo ideas, funny social media captions, figuring out the best hashtags to use, product tagging and categorization. It has proven to be a great way to get everything to around 70%, then send off to my assistant for the remaining 30% of polish. So at…

I've been using it for product ideas as well. Occasionally it's brilliant but sometimes it's so bad it's funny, like when it suggested Seven-Year Itch for a perfume brand.

>> it's so bad it's funny, like when it suggested Seven-Year Itch for a perfume brand.

Yeah, what's wrong with that? I'm sure it would be catchy to a particular demographic. No need for a brand to cater to everyone.

Re: ChatGPT could cost over $700k per day to operate

#140

Post by kolinko »

barkerja wrote: post #57
ren_engineer wrote: post #46

Earlier quoted context omitted.

interesting that the article mentions Microsoft is trying to build their own AI chips to help reduce cost, they are years behind Google in that effort. Google still has the advantage in both research and hardware, they just fumbled the execution but it's still very early in the game

> they are years behind Google in that effort. Google still has the advantage in both research and hardware, they just fumbled the execution but it's still very early in the game Where do we think Apple is in regards to this? If anyone has the upper-hand here with hardware, I would think it'd be Apple. But there's been zero indication that Apple has been working on any sort of generative AI.

Apple already has a super-efficient neural engine in all their chips - it's just a matter of them building dedicated models for them.
Post reply on HN