It seems the human brain is compressing memories like an autoencoder. But how does it learn? We are not reproducing the input again from the encoded memories and backpropagate the error. Or is that what we call dreaming?
Human brain compresses working memories into low-res ‘summaries’
311–320 of 349 posts
Re: Human brain compresses working memories into low-res ‘summaries’
#312Earlier quoted context omitted.
The largest Pterosaur are estimated to have had wingspans of more than 9m and weigh up to 250kg (550 pounds) and we believe they were able to fly. [1] But that's not the most relevant point here. The fact that humans did achieve to fly, but through a different method than birds is exactly a supporting argument that we might achieve AGI with a different approach than how our brains do it. There are countless similar e…
The chain of reasoning that everything observable in nature is replicable by humans would also imply us being able to replicate creation of a living cell from non living material and then endow that organism with consciousness. Further more it would also imply us being able to replicate birth of stars, black holes, and "the big bang" itself. I am not qualified enough to speak if there is anything fundamentally imposs…
> us being able to replicate creation of a living cell from non living material and then endow that organism with consciousness.
Afaik we are very close to artificially creating living cells. This is one recent example [1]. The consciousness part is similar to AGI.
> Further more it would also imply us being able to replicate birth of stars, black holes, and "the big bang" itself.
Some things might be a logistical challenge rather than one of knowledge. Fusion energy attempts to replicate the way stars produce energy and we already managed to replicate the effect, we are just (many years) shy of maintaining it to produce positive energy.
But you might be right and some things are impossible to replicate. I'm much more inclined to believe we can't replicate the big bang than general intelligence as mother nature replicates general intelligence millions of times each day. And by now we started to have a discussion about believes rather than knowledge, which is a much more healthy way to put it, as we indeed don't know.
[1] https://www.nyu.edu/about/news-publications/news/2021/septem...
Re: Human brain compresses working memories into low-res ‘summaries’
#313Earlier quoted context omitted.
Ever since Gödel we’ve had a pretty convincing proof that there is nothing that you can do in terms of reasoning that can’t be expressed using computation. And since Turing we’ve got a framework that shows there’s nothing computable that you can’t compute using a universal computer. So unless there’s something mystical beyond the realm of mathematics to ‘reasoning’ it can’t be a superset of computing. If a finite amo…
There are actually a lot of well-defined things beyond the power of a Turing machine (for example a Turing machine plus a halting oracle that only works on Turing machines without a halting oracle) but in terms of finite amounts of electrons doing normal low-energy electronic stuff you are quite likely correct. Humanity may go beyond computability if as some papers have suggested quantum gravity requires solving unco…
Do you believe human brains contain a halting oracle? Or the moral equivalent of one - something that enables our brains to accomplish some non computable reasoning task?
Re: Human brain compresses working memories into low-res ‘summaries’
#314Earlier quoted context omitted.
A nice definition of intelligence I've heard is exactly the ability to form models of the world with predictive power. And a model is essentially a compression of real-world data. Physical laws are a great example of this.
Creating models with predictive power is also a precise definition of science.
Can you recommend any philosophy of science (or life) treatises about this?
I long considered myself a Popperian. A few years ago, I decided that I'm a "Predictionist" (a placeholder made up word until I learn better). I'm struggling to figure out what that even means.
I still agree with Popper. Empathetically.
I'm just tired of arguing. I forfeit. I give up. I no longer believe that discourse is helpful, that people are persuadable, that we can share Truth.
Instead, I just want to know the predictive strength of someone's Truth.
For example:
The Earth is flat? Oh? Cool. Please, tell me, how does that Truth help me?
Re: Human brain compresses working memories into low-res ‘summaries’
#315Compression is a component of general intelligence. A few years ago I was very sceptical of machine learning ever leading to general intelligence. I've since changed my mind. There are a lot of parallels to this work and the concept of "embeddings" in machine learning. Intelligence requires the ability to generalize. A prerequisite for generalization is the ability to take something high-dimensional and reduce it to…
This also ties in to the cybernetic concept of the law of requisite variety, where adaptable entities need to be able to compress their sense-data about their environment into an internal model that corresponds in complexity to their need to act - this necessarily involves compression as the totality of reality is effectively infinite and can't fit between your ears. There's also the Hutter prize that ties data compr…
Which also seems a cornerstone of Fristons Active inference (from the little I understand of it).
Re: Human brain compresses working memories into low-res ‘summaries’
#316Earlier quoted context omitted.
The chain of reasoning that everything observable in nature is replicable by humans would also imply us being able to replicate creation of a living cell from non living material and then endow that organism with consciousness. Further more it would also imply us being able to replicate birth of stars, black holes, and "the big bang" itself. I am not qualified enough to speak if there is anything fundamentally imposs…
> us being able to replicate creation of a living cell from non living material and then endow that organism with consciousness. Afaik we are very close to artificially creating living cells. This is one recent example [1]. The consciousness part is similar to AGI. > Further more it would also imply us being able to replicate birth of stars, black holes, and "the big bang" itself. Some things might be a logistical ch…
> Afaik we are very close to artificially creating living cells. This is one recent example [1].
I beg to differ. It may look impressive on the surface, just like GPT-3 looks impressive on the surface, but is far from the real thing. It is just another extension of the ladder to the Moon.
The effort described in the article is nowhere near a living cell. It lacks protein building and DNA/RNA mechanisms. They basically describe a group of nanomotors.
I can recommend watching James Tour on this very topic [1] and Stephen Meyer on the related topic of intelligent design [2]. Those two lectures were eye-opening for me learning more about this field. Note: both of them are self-confessed theist scientists which to me did not represent a problem (my viewes are agnostic, and it only made it more interesting as you rarely get to hear different views about these matters than pop-sci).
> The consciousness part is similar to AGI.
It is not clear what you mean by that. One thing is to build computer code and then have it manifest 'intelligence'. Whole another thing is doing same with organic matter that can not be 'programmed', even if we knew how to do it (let alone there is no evidence that 'programming' is responsible for consciousness at all to begin with).
This is also known as 'hard problem of consciousness' and David Chalmers is considered one of the leading experts in the field [3]. Basically smartest scientists in the world are clueless about this and do not know even where to begin, in many ways similar to AGI.
> Some things might be a logistical challenge rather than one of knowledge. Fusion energy attempts to replicate the way stars produce energy and we already managed to replicate the effect, we are just (many years) shy of maintaining it to produce positive energy.
I can see why one can have this position where it seems like we are making progress in everything we talked about, but that is the main punchline of the ladder to the Moon analogy. Indeed it is imaginable, and indeed every step makes us closer. But it does not mean we will ever reach it.
I agree with you that the discussion ultimately boils down to direction and strength of one's beliefs.
[1] https://www.youtube.com/watch?v=zU7Lww-sBPg
Re: Human brain compresses working memories into low-res ‘summaries’
#317No post body was provided.
Re: Human brain compresses working memories into low-res ‘summaries’
#318Earlier quoted context omitted.
A nice definition of intelligence I've heard is exactly the ability to form models of the world with predictive power. And a model is essentially a compression of real-world data. Physical laws are a great example of this.
Creating models with predictive power is also a precise definition of science.
[deleted]
Re: Human brain compresses working memories into low-res ‘summaries’
#319Earlier quoted context omitted.
Creating models with predictive power is also a precise definition of science.
Can you recommend any philosophy of science (or life) treatises about this? I long considered myself a Popperian. A few years ago, I decided that I'm a "Predictionist" (a placeholder made up word until I learn better). I'm struggling to figure out what that even means. I still agree with Popper. Empathetically. I'm just tired of arguing. I forfeit. I give up. I no longer believe that discourse is helpful, that people…
My research involve applying Popper's epistemology to natural language processing.
So I am quite involved in this.
As far as I can tell, almost all of what Popper tried to do with quantification measures of information are exactly what you are talking about.
In particular Conjectures and Refutations covers this really extensively so I'd recommend reading or re-reading that. Though Logic of Scientific Probability covers an early form. David Miller's Critical Rationalism covers it well too and some of it's problems.
I.e: His notion (shared with positivists like Carnap and others) that science is a set of logical statements. A collection of statements is a theory, a theory entails a set of predictions which is called the information content of the theory (sometimes I(c) or C(I) in his notation).
If the I(c) > I(c') where c' is a competing theory then it is said to have more explanitory power. I.e. it makes more predictions.
This is part of his defnition of what makes a good explanation and what david desutch calls "hard to vary".
The other main part of the definitition is about whether these statements reflect Truth in anyway.. that is covered by his notion of verisimiltude or truthlikeness which is quantified as the degree to which the information content of a theory I(c) can be corroborated.
Both of these are essentailly "The predictive strength of someone's Truth"
The problem you and many other have probably encountered is the information content of an explanation is *intractable* it's an open set of statements which cannot full by fleshed out. So instead we can never have a perfect quantification of whether my theories or your theories are better... there may indeed be statements entailed by flat earth theory that have yet to be discovered and could indeed be more corroborated and provide better information content than a non-flat earth theory! Popper revels in this fact and fully embraces it.
Beyond Popper though, we need to understand more of the dynamics of "predictive strength" - I am finding causality a great source of literature that for which I would recommend Judea Perl and the Book of Why among other things.
For Philosophy of Science in particular there are ton's of great articles on stanford encyclopedia of philosophy about Explanation that go into this in depth - in fact the positivists like Carnap wrote amazing things about this which I would recommend.
Re: Human brain compresses working memories into low-res ‘summaries’
#320Earlier quoted context omitted.
I don't know. I just don't see it. The subject is given two tasks requiring working memory. The researchers observe activity in the parietal and visual cortices via fMRI, and find the neural activity between the two tasks is indistinguishable. And conclude > ... distinct visual stimuli (oriented gratings and moving dots) are flexibly recoded into the same WM format in visual and parietal cortices when that representa…
I read it as saying they actually got topographical line-like structures in the MRI, similarly to the well established result that there are typographically-arranged visual neurons that essentially light up like pixels in response to a scene. So if that’s right then they are actually measuring a spatial encoding. Anyway, it’s just the abstract so I could be completely wrong.
Should be “topographically-arranged” above…