Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why your brain is 3 milion more times efficient than GPT-4

grski.pl

Post reply on HN
Print view View original

61–70 of 213 posts

Re: Why your brain is 3 milion more times efficient than GPT-4

#61

Post by cynusx »

londons_explore wrote: post #52
cynusx wrote: post #31

The comparison doesn't really hold. He is comparing energy spend during inference in humans with energy spend during training in LLM's. Humans spend their lifetimes training their brain so one would have to sum up the total training time if you are going to compare it to the training time of LLM's. At age 30 the total energy use of the brain sums up to about 5000 Wh, which is 1440 times more efficient. But at age 30…

> At age 30 the total energy use of the brain sums up to about 5000 Wh, That doesn't sound right... 30 years * 20 Watts = 1.9E10 Joules = 5300 kWh.

Where did you get the 20 Watt from?

My number is based on calorie usage

Re: Why your brain is 3 milion more times efficient than GPT-4

#62

Post by bamboozled »

scotty79 wrote: post #42
Synaesthesia wrote: post #14

Earlier quoted context omitted.

No, only a brain can "think" and be original. A computer is limited to what we input to it. An "AI" simply recapitulates what it was trained on.

Do a little exercise for me. Try to be as creative as you can be and imagine how a space alien might look. It's a combination of what you have already seen, read about or heard of, isn't it?

I see it quite differently, we don't have to be so creative, we need to get better at appreciating what already exists. Think about how far out most sea creatures really are, a bluebottle, a starfish for example...

Personally I think there is a bit of evidence in your comment that we don't really understand our minds or cognition very well.

Re: Why your brain is 3 milion more times efficient than GPT-4

#63

Post by shinycode »

assimpleaspossi wrote: post #53

I don't care. I've come to the conclusion that gpt and gemini and all the others are nothing but conversational search engines. They can give me ideas or point me in the right direction but so do regular search engines. I like the conversation ability but, in the end, I cannot trust their results and still have to research further to decide for myself if their results are valid.

I do agree that I rarely use Google now, I search into a chat to have a summary and this saves lot of aggregation from different sites. The same for Stack Overflow, no use if I find the answer quicker.

It’s exactly that for me, a conversational search engine. And the article explains it right, it’s just words organized in very specific ways to be able to retrieve them with statistical accuracy and the transformer is the cherry on top to make it coherent

Re: Why your brain is 3 milion more times efficient than GPT-4

#64

Post by cynusx »

cynusx wrote: post #61
londons_explore wrote: post #52

Earlier quoted context omitted.

> At age 30 the total energy use of the brain sums up to about 5000 Wh, That doesn't sound right... 30 years * 20 Watts = 1.9E10 Joules = 5300 kWh.

Where did you get the 20 Watt from? My number is based on calorie usage

oh ok, I used 400 calories/day and not 400 kcal/day.

Yea, then the numbers are off by 1000

Re: Why your brain is 3 milion more times efficient than GPT-4

#65

Post by mihaic »

imabotbeep2937 wrote: post #60
mihaic wrote: post #58

Genuinely curious who upvoted this and why. The title is clickbait, the writing is long and rambling and it seems to me like the author doesn't have a profound understand of the concepts either, all just to recommend Qdrant as a vector database.

Posted article quality is not always very good here lately. Clickholes get too many votes.

Yeah, it seems almost insulting that the author expects countless people to spend time reading their posts, while they haven't spent a lot of time to edit and streamline it, all with the excuse: "these are just my ramblings".

To paraphrase, I will not excuse such a long letter, for you had more time to write a shorter one.

Re: Why your brain is 3 milion more times efficient than GPT-4

#66

Post by EncomLab »

It's always going to be difficult to compare a carbon based, ion mediated, indirectly connected, reconfigurable network of neurons to a silicon based, voltage mediated, directly connected, fixed configuration transistors.

The analogy works, but not very far.

Re: Why your brain is 3 milion more times efficient than GPT-4

#67

Post by avereveard »

shinycode wrote: post #25
avereveard wrote: post #10

Earlier quoted context omitted.

Only if we talk about trained intelligence. Likely the requirements for evolved intelligence are different and involve being embodied, edonistic, and the pressure of a selection mechanism

If a spontaneous intelligence is 3 million times more efficient that one that one who took millions of hours of work from brains (there is so much effort that there is even more work put into AI that evolution who thinly spread changes through time, efforts diluted). We either have to define that AI will never be the same as HI and can’t compete with it or it’s of the same nature as some people say on HN and for me i…

Why? Nothing in the if sustain the conclusion. Also we did not emerge because of ourselves, so our intelligence cannot be postulated unique because of that. Absence of history is no proof either way, and we do have history of intelligence in other pila as well here in earth.

Re: Why your brain is 3 milion more times efficient than GPT-4

#68

Post by Closi »

eloeffler wrote: post #51
wanderingmind wrote: post #46

Earlier quoted context omitted.

Not just that the brain of a newborn comes pretrained with billions of years of evolution. There is an energy cost associated with that which must be taken into account

Then you must also take that cost into account when calculating the cost of training LLMs, as well as the cost humans operating the devices and their respective individual brain development. LLMs are always an additional cost, never more efficient because they add to the calculation, if you look at it that way.

Only if we are counting the cost to generate all the inputs to training, and not just the training itself - it just depends on the scope of the analysis.

(i.e. taken to the extreme, as humans learn from their environment, do we have to count all energy that has gone into creating the world as we know it?)

Re: Why your brain is 3 milion more times efficient than GPT-4

#69

Post by logicallee »

madsbuch wrote: post #17

There is an immensely strong dogma that, to my best knowledge, is not founded in any science or philosophy: First we must lay down certain axioms (smart word for the common sense/ground rules we all agree upon and accept as true). One of such would be the fact that currently computers do not really understand words. ... The author is at least honest about his assumptions. Which I can appreciate. Most other people jus…

It's the most incredible coincidence. Three million paying OpenAI customers spend $20 per month (compare: NetFlix standard: $15.49/month) thinking they're chatting with something in natural language that actually understands what they're saying, but it's just statistics and they're only getting high-probability responses without any understanding behind it! Can you imagine spending a full year showing up to talk to a brick wall that definitely doesn't understand a word you say? What are the chances of three million people doing that! It's the biggest fraud since Theranos!! We should make this illegal! OpenAI should put at the bottom of every one of the millions of responses it sends each day: "ChatGPT does not actually understand words. When it appears to show understanding, it's just a coincidence."

You have kids talking to this thing asking it to teach them stuff without knowing that it doesn't understand shit! "How did you become a doctor?" "I was scammed. I asked ChatGPT to teach me how to make a doctor pepper at home and based on simple keyword matching it got me into medical school (based on the word doctor) and when I protested that I just want to make a doctor pepper it taught me how to make salsa (based on the word pepper)! Next thing you know I'm in medical school and it's answering all my organic chemistry questions, my grades are good, the salsa is delicious but dammit I still can't make my own doctor pepper. This thing is useless!

/s

Re: Why your brain is 3 milion more times efficient than GPT-4

#70

Post by ben_w »

mikae1 wrote: post #24
asah wrote: post #21

FTFY: ONLY 3 million times. At the current pace of development, AI will catch-up in a decade or less.

How does that math work out? The developments during the last year has been... Abysmal? The hype and marketing bull is increasing exponentially though.

Look at the price difference of tokens on their API between the first release of ChatGPT and the current one.

• Current 3.5-family price is $1.5/million tokens

• Was originally $20/million tokens based on this quote: "Developers will pay $0.002 for 1,000 tokens — which amounts to about 750 words — making it 10 times cheaper" - https://web.archive.org/web/20230307060648/https://digiday.c...

(I can't find the original 3.5 API prices even on archive.org, only the Davinci etc. prices, the Davinci model prices were also $20/million).

There's also the observation that computers continue to get more power efficient — it's not as fast as Moore's Law was, doubling every 2.6 years, or a thousand-fold every 26 years, or about 30% per year.

Post reply on HN