Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why your brain is 3 milion more times efficient than GPT-4

grski.pl

Post reply on HN
Print view View original

121–130 of 213 posts

Re: Why your brain is 3 milion more times efficient than GPT-4

#121

Post by gnramires »

bamboozled wrote: post #104
Jensson wrote: post #99

Earlier quoted context omitted.

> we spend out time having fun and learning about the world is a side effect What do you think "play" is? Animals play to learn about themselves and the world, you see most intelligent animals play as kids with the play being a simplification of what they do as adults. Human kids similarly play fight, play build things, play cook food, play take care of babies etc, it is all to make you ready for an adult life. Playi…

But I just play because it's fun, I roll dice for fun, are you trying to tell me all this is a secret front for "training" ?

It's probably not something you secretly wish when playing, and perhaps for the best (that you don't have fun and enjoy things always with an ulterior motive, except enjoying the experiences). I guess he's saying in the sense of the natural function of play, and we playing is mostly a consequence of the natural function. It's also very much true that we learn a lot, perhaps a significant chunk of what we learn is through play[1], so it's also undeniable that we do learn from play, even if humans have this great gift -- we are able to understand the nature of things (such as play) and choose to do them just for the sake of experiences, fun, joy, happiness, etc..

Which we should (finally :) ) recognize to be the source of all meaning.

We still should learn (and do practical stuff in general) because it supports our inner lives, including building technology, producing things (buildings, infrastructure) that support us and indeed enables our (inner) lives.

[1] Also of note humans, unlike LLMs, can learn all the time, we don't have a hard "training phase". It's true brain plasticity decays, and it becomes harder to learn as we age, but we can still learn more or less quickly at any age. This is why dedicating childhood to learning (as well as play) is natural.

Re: Why your brain is 3 milion more times efficient than GPT-4

#122

Post by scotty79 »

Jensson wrote: post #105
scotty79 wrote: post #42

Earlier quoted context omitted.

Do a little exercise for me. Try to be as creative as you can be and imagine how a space alien might look. It's a combination of what you have already seen, read about or heard of, isn't it?

People come up with stuff like this: https://i.redd.it/oenn6vi61ag21.png Or this: https://preview.redd.it/finally-made-my-scientist-species-to... Humans are capable of thinking and fleshing out novel concepts, current AI are not. Sure your first thing will greatly resemble current things, but as you iterate and get further and further away from existing things what you do stops being an imitation and starts being its…

Thanks for providing examples of combinations of things already seen.

Re: Why your brain is 3 milion more times efficient than GPT-4

#123

Post by glenstein »

greenthrow wrote: post #101
cynusx wrote: post #31

The comparison doesn't really hold. He is comparing energy spend during inference in humans with energy spend during training in LLM's. Humans spend their lifetimes training their brain so one would have to sum up the total training time if you are going to compare it to the training time of LLM's. At age 30 the total energy use of the brain sums up to about 5000 Wh, which is 1440 times more efficient. But at age 30…

The article is a bit of a stretch but this is even more of a stretch. Humans can do way more than an LLM, humans are never in only learning mode, our brains are always at least running our bodies as well, etc.

Exactly right - we are obviously not persistently in all-out training mode over the course of our lifetimes.

I suppose they intended that as a back-of-the-envelope starting point rather than a strict claim however. But even so, gotta be accountable to your starting assumptions, and I think a lot changes when this one is reconsidered.

Re: Why your brain is 3 milion more times efficient than GPT-4

#124

Post by madsbuch »

mensetmanusman wrote: post #107
madsbuch wrote: post #17

There is an immensely strong dogma that, to my best knowledge, is not founded in any science or philosophy: First we must lay down certain axioms (smart word for the common sense/ground rules we all agree upon and accept as true). One of such would be the fact that currently computers do not really understand words. ... The author is at least honest about his assumptions. Which I can appreciate. Most other people jus…

It’s a reasonable axiom, because for many people understanding involves qualia. If you believe LLM have qualia, you also believe a very large Excel sheet with the right numbers has an experience of consciousness and feels pain or something where the document is closed.

As I wrote, I appreciate that the author wrote it out as they did. It might be reasonable in the context of the article. But fixing it as an axiom just makes the discussion boring (for me).

> If you believe LLM have qualia, you also believe a ...

You use the word believe twice here. I am actively not talking about beliefs.

I just realise, that the author indeed gave themselves an out:

> ... currently computers do not really understand words.

The author might believe that future computers can understand words. This is interesting. Questions being _what_ needs to be in order for them to understand? Could that be an emergent feature of current architectures? That would also contradict large parts of the article.

Re: Why your brain is 3 milion more times efficient than GPT-4

#125

Post by bamboozled »

b112 wrote: post #94
dist-epoch wrote: post #74

Earlier quoted context omitted.

For every calorie a human consumes, hundreds or thousands more are used by external support systems. So yeah, you do use 2000 calories a day, but unless you live in an isolated jungle tribe, vast amounts of energy are consumed on delivering you food, climate control, electricity, water, education, protection, entertainment and so on.

By that metric, the electricity is only part of it. The cost of building the harsware, the cost of building the roof and walls for the datacentre, the cost of clearing the land, cost of humans maintaining the hardware, the cost of all the labour making the linux kernel, libc6, etc, etc. Lots of additionals here too.

It's almost like...nothing exists in a vacuum.

Re: Why your brain is 3 milion more times efficient than GPT-4

#126

Post by naveen99 »

somenameforme wrote: post #115
moffkalast wrote: post #103

Earlier quoted context omitted.

Replace "gpt and gemini and all the others" with "people" and funny enough your statement is still perfectly accurate. You have a rough mathematical approximation of what's already a famously unreliable system. Expecting complete accuracy instead of about-rightness from it seems mad to me. And there are tons of applications where that's fine, otherwise our civilization wouldn't be here today at all.

These anthropomorphizations are increasingly absurd. There's a difference between a human making a mistake, and an AI arbitrarily and completely confidently creating entirely new code APIs, legal cases, or whatever that have absolutely no basis in reality whatsoever, beyond being what it thinks would be an appropriate next token based on what you're searching for. These error modes are simply in no way, whatsoever, c…

I don’t know. I have seen some humans who are very allergic to criticism of any kind, even constructive criticism. in the name of harmony…

Re: Why your brain is 3 milion more times efficient than GPT-4

#127

Post by bamboozled »

Jensson wrote: post #120
bamboozled wrote: post #117

Earlier quoted context omitted.

So when I look at a sunset and enjoy that, I think it's fun to chase sunsets, my "brain" is telling me it's fun because I'm learning a new skill and receiving a dopamine reward for looking at the sunset and feeling good about it?

Looking at a sunset isn't a game you play, kids don't go and play "look at the sunset", I feel like you are grasping at straws. Why would you feel calm and comfortable from a sunset? Probably to get you sleepy so you go find a place to sleep since there isn't much useful to do at night. That would be unrelated to play. Anyway, most of our feelings comes from nature, we didn't evolve to be faulty, we evolved to do thi…

I never said kids play like that. I do.

As an adult, I find it fun and enjoyable to seek out sunsets I find the colors beautiful. I readily hike mountains just to enjoy a sunset. I watch a sunset and then go party till 3 am, so maybe it's got to do with finding a nice place to sleep, or maybe it's just nice that we have the ability to appreciate phenomenon without having to apply some rigorous concept to it. I'd fly 2/3 of the way around the world to watch a total eclipse.

Personally I think you might be clasping at straws trying to equate every pleasant experience to some type of reward function.

I'd go as far as saying if we worked this simply and predictably, then our lives would be much easier. We'd all be exercising for that dopamine hit, we'd all be going to bed early after a nice sunset, but we dont.

Re: Why your brain is 3 milion more times efficient than GPT-4

#128

Post by scotty79 »

Synaesthesia wrote: post #112
scotty79 wrote: post #42

Earlier quoted context omitted.

Do a little exercise for me. Try to be as creative as you can be and imagine how a space alien might look. It's a combination of what you have already seen, read about or heard of, isn't it?

I’m not saying we don’t have limitations, we clearly do. There are limits to our intellectual capacity and creativity. ChatGPT can exceed humans in its knowledge store. It is excellent at doing research. But it’s not thinking it is merely selecting the most likely nest words based on some algorithm.

I wouldn't even give as much appreciation to chatgpt as you do. But I don't see it doing anything different than human brains do. It's just still not very good at it.

If it were up to me I'd try to give it another representation than just words. I think those models should be trained to represent text as relationship graphs of objects. There's not much natural data lole that, but it should be fairly rasy to create vast amounts of synthetic data, text generated from relationship graphs. Model should be able to make the connection to natural language.

Once models are taught this representation they might learn how the graphs transform during reasoning just by training on natural language reasoning.

Re: Why your brain is 3 milion more times efficient than GPT-4

#129

Post by bamboozled »

gnramires wrote: post #121
bamboozled wrote: post #104

Earlier quoted context omitted.

But I just play because it's fun, I roll dice for fun, are you trying to tell me all this is a secret front for "training" ?

It's probably not something you secretly wish when playing, and perhaps for the best (that you don't have fun and enjoy things always with an ulterior motive, except enjoying the experiences). I guess he's saying in the sense of the natural function of play, and we playing is mostly a consequence of the natural function. It's also very much true that we learn a lot, perhaps a significant chunk of what we learn is thr…

I'm conscious I have to types of play though, I'm fully conscious sometimes play can be about learning and training, it's why I ski more difficult terrain than I'm comfortable with, but then I might go eat a massive bowl of pasta and have a glass or three of wine. On an intellectual level, I know there are healthy more rewarding things to eat. I know wine isn't great for me at those quantities. But I consciously make the choice to do it because it's fun.

Re: Why your brain is 3 milion more times efficient than GPT-4

#130

Post by shkkmo »

kvdveer wrote: post #7

I feel the author is comparing an abstract representation of the brain to a mechanical representation of a computer. This is not a fair or useful comparison. If a computer does not understand words, neither does your brain. While electromagnetic charge in the brain does not at all correspond with electromagnetic charge in a GPU, they do share an abstraction level, unlike words vs bits.

Becareful with mixing up 'can' and 'does'.

Computers right now do not understand language, but that does not mean that they cannot. We don't know what it takes to bridge the gap from stochastic parrot to understanding in computers, however from the mistakes LLMs make right now, it appears we have not found it yet.

It is possible that silicon based computer architecture cannot support the processing and information storage density/latency to support understanding. It's hard to guage the likelihood this is true given how little we know about how understanding works in the brain.

Post reply on HN