Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is Chat GPT so expensive to operate?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 54 posts

Re: Why is Chat GPT so expensive to operate?

#31

Post by ericathegreat »

_8j50 wrote: post #26
mr_00ff00 wrote: post #14

Earlier quoted context omitted.

Really makes you appreciate the brain, which presumably operates with some sort of similar demand.

To have ML produce meaningful content you need tp give it some input or a sense of what the outcome should be and this is after billions of trial and errors. Yet people these days believe something like the brain was bruteforced by nature into an accidental existence.

Some input: The organism's environment.

Outcome should be: The organism successfully produces offspring

Natural selection is doing exactly what you describe.

Re: Why is Chat GPT so expensive to operate?

#33

Post by _8j50 »

ericathegreat wrote: post #31
_8j50 wrote: post #26

Earlier quoted context omitted.

To have ML produce meaningful content you need tp give it some input or a sense of what the outcome should be and this is after billions of trial and errors. Yet people these days believe something like the brain was bruteforced by nature into an accidental existence.

Some input: The organism's environment. Outcome should be: The organism successfully produces offspring Natural selection is doing exactly what you describe.

Except natural selection can't start over. It onlu works if there are always a high rate of survivors and even if that was not an issue consider 4 billion years and a generous generation life of one year (natural selection cycle), 4 billion isn't a whole lot even for small features when you don't have an enormous population and birth rate. Let's say there were 100000 humans at some point and only a 1000 fatal features (being generous) it's not just the replacement rate of defective humans that needs to exceed the elimination rate, a certain percent of replacements must be free of all fatal defects and survive. Also, consider how there should be many failed species that attempted to evolve into a human like species or a primate. You can't always luck out, at some point the entire branch has to fail, requiring subsequent attmepts meanwhile the fatal conditions that required the evolution will not go away.

Re: Why is Chat GPT so expensive to operate?

#34

Post by imtringued »

mr_00ff00 wrote: post #14
vineyardmike wrote: post #11

All these answers are good, but I can share more concrete numbers… Meta released their OPT model which they claim is comparable to the GPT-3 model. Guidance for running that model [1] suggests a LOT of memory - at least 350GB of gpu memory which is roughly 4 A1000s, which are pricy. Running this on AWS with the above suggestion would cost $25/hr - just for one model running. That’s almost $0.50 a minute. If you imagi…

Really makes you appreciate the brain, which presumably operates with some sort of similar demand.

The brain doesn't use a synchronous digital architecture. It is asynchronous. Spiking neural networks implemented in neuromorohic hardware are equally efficient. They consume milliwatts for a million neurons.

Re: Why is Chat GPT so expensive to operate?

#35

Post by elbear »

MuffinFlavored wrote: post #32
DoesntMatter22 wrote: post #13

I think it's actually quite cheap for what it is

what useful purpose have you found for ChatGPT given the “it can return inaccurate results posed as accurate” problem?

Have it do stuff you know how to do, just a lot faster. Or, even if you don't know exactly how to do it, check what it gave you to see if it produces expected results.

For example, it gives you code. You run that code to see if the outputs are as expected.

Re: Why is Chat GPT so expensive to operate?

#36

Post by canadianfella »

unsupp0rted wrote: post #18
mr_00ff00 wrote: post #14

Earlier quoted context omitted.

Really makes you appreciate the brain, which presumably operates with some sort of similar demand.

Hard to tell. Similar to how it takes a lot of resources for a human to hang from monkey bars but for a sloth it takes basically no resources at all, because the sloth comes out of the box designed for it.

[dead]

Re: Why is Chat GPT so expensive to operate?

#37

Post by mike_hearn »

vineyardmike wrote: post #11

All these answers are good, but I can share more concrete numbers… Meta released their OPT model which they claim is comparable to the GPT-3 model. Guidance for running that model [1] suggests a LOT of memory - at least 350GB of gpu memory which is roughly 4 A1000s, which are pricy. Running this on AWS with the above suggestion would cost $25/hr - just for one model running. That’s almost $0.50 a minute. If you imagi…

Don't forget training costs, labor costs used for RLHF and (most likely) the money required for such large volumes of training data.

Re: Why is Chat GPT so expensive to operate?

#39

Post by KingMob »

_8j50 wrote: post #33
ericathegreat wrote: post #31

Earlier quoted context omitted.

Some input: The organism's environment. Outcome should be: The organism successfully produces offspring Natural selection is doing exactly what you describe.

Except natural selection can't start over. It onlu works if there are always a high rate of survivors and even if that was not an issue consider 4 billion years and a generous generation life of one year (natural selection cycle), 4 billion isn't a whole lot even for small features when you don't have an enormous population and birth rate. Let's say there were 100000 humans at some point and only a 1000 fatal feature…

> It onlu works if there are always a high rate of survivors

There doesn't have to be a high rate of survival if the reproductive rate compensates for losses.

E.g., if 80% of wild rabbits are eaten, but the remaining 20% can give birth to 5 bunnies per parent per lifetime, the population will be stable.

I have no idea where you're getting your beliefs, but most of it is wrong in both the math and biology.

Re: Why is Chat GPT so expensive to operate?

#40

Post by awesomeMilou »

imtringued wrote: post #34
mr_00ff00 wrote: post #14

Earlier quoted context omitted.

Really makes you appreciate the brain, which presumably operates with some sort of similar demand.

The brain doesn't use a synchronous digital architecture. It is asynchronous. Spiking neural networks implemented in neuromorohic hardware are equally efficient. They consume milliwatts for a million neurons.

Do you have links on novel hardware architectures for neuromorphic hardware? In my country , the leading research group for neuromorphic computing does not cite any novel hardware approaches, only what existing hw architectures are most suitable.
Post reply on HN