Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human brain compresses working memories into low-res ‘summaries’

directorsblog.nih.gov

Post reply on HN
Print view View original

221–230 of 349 posts

Re: Human brain compresses working memories into low-res ‘summaries’

#221

Post by freediver »

axg11 wrote: post #35

Compression is a component of general intelligence. A few years ago I was very sceptical of machine learning ever leading to general intelligence. I've since changed my mind. There are a lot of parallels to this work and the concept of "embeddings" in machine learning. Intelligence requires the ability to generalize. A prerequisite for generalization is the ability to take something high-dimensional and reduce it to…

I co-authored a paper exploring this topic few years ago, while I was pretty excited about the possiblity of using embeddings for generalization.

"Towards conceptual generalization in the embedding space" https://arxiv.org/abs/1906.01873

I still think the approach outlined in the paper (using embeddings to map the physical world) is sound especially for the field of self-driving which is in dire need of generalization, but I've since changed my mind and currently do not believe we can achieve AGI (ever).

While embeddings are a great tool for compressing information, they do not provide inherent mechanisms for manipulating the information stored in order to generalize and infer outcomes in new, unseen situations.

And even if we would start producing embeddings in a way where they would have some basic understanding of the physical world, we could never achieve it to the level of detail necessary - because physical world is not a discrete function. Otherwise we would be creating a perfect simulation (within a simulation?). And the last time I was playing God, was in "Populous".

Re: Human brain compresses working memories into low-res ‘summaries’

#222

Post by bick_nyers »

sethrin wrote: post #185
bick_nyers wrote: post #172

Earlier quoted context omitted.

Another potential upside of a brain to computer interface (Neuralink), the ability to store every memory you have ever had (while the device was installed) in full resolution. Assuming of course you maintain a server rack at home with copious amounts of hard drives.

> full resolution What, generally, do you think this might mean?

The ability to experience a memory as precisely as you want, including the option of a full mental transplant, like loading a save file for a video game. See, hear, touch, smell, taste, and think the exact same thoughts as you did 15 years ago. The playback mechanisms will have some caveats, as it may not strictly be possible to playback perfectly, as you are a different person with a different brain and body than say 15 years ago. You could relive something in the first person perspective, or perhaps just observe yourself from a third person perspective.

To a lesser degree, just being able to hear the dialogue in your brain at the time of a memory would be monumental. Then you can get into the business of using tools built around this, such as searching your memories, computing statistical analysis (maybe you can find out why you haven't been able to commit to an exercise habit for the past 5 years?), and so on.

Re: Human brain compresses working memories into low-res ‘summaries’

#223

Post by zasdffaa »

sva_ wrote: post #216
TameAntelope wrote: post #96

I think this is why it's hard sometimes to argue in support of something you believe, even if you're right. At one point, all of the relevant facts and figures were loaded into your working memory, and with that information you arrived at a conclusion. Your brain, however, no longer needs those facts and figures; you've gotten what you needed from them, and they can be kicked out of working memory. What you store the…

Interesting thought. Perhaps that is also why people sometimes have a hard time changing their mind when confronted with new information: a certain number of bits of information have led you to your belief, and even if some of those change or turn out to be false, you can't access those bits anymore individually, but only the resulting belief. Perhaps, the more those beliefs are reinforced, the less likely you are to…

> why people sometimes have a hard time changing their mind when confronted with new information

Something else happens with me, it's like my brain says "this does not fit in with what I understand, discard it". At a conscious level I don't hear what I've just been told. I have to be told it again, and sometimes more than twice before it finally works its way in. It's a liability for me and a frustration for others and it's just plain peculiar.

Re: Human brain compresses working memories into low-res ‘summaries’

#224

Post by [deleted] »

TameAntelope wrote: post #96

I think this is why it's hard sometimes to argue in support of something you believe, even if you're right. At one point, all of the relevant facts and figures were loaded into your working memory, and with that information you arrived at a conclusion. Your brain, however, no longer needs those facts and figures; you've gotten what you needed from them, and they can be kicked out of working memory. What you store the…

[deleted]

Re: Human brain compresses working memories into low-res ‘summaries’

#225

Post by [deleted] »

[deleted] wrote: post #224
TameAntelope wrote: post #96

I think this is why it's hard sometimes to argue in support of something you believe, even if you're right. At one point, all of the relevant facts and figures were loaded into your working memory, and with that information you arrived at a conclusion. Your brain, however, no longer needs those facts and figures; you've gotten what you needed from them, and they can be kicked out of working memory. What you store the…

[deleted]

[deleted]

Re: Human brain compresses working memories into low-res ‘summaries’

#226

Post by hammock »

copperx wrote: post #66
mherrmann wrote: post #44

Earlier quoted context omitted.

A nice definition of intelligence I've heard is exactly the ability to form models of the world with predictive power. And a model is essentially a compression of real-world data. Physical laws are a great example of this.

Creating models with predictive power is also a precise definition of science.

Testing models*

Re: Human brain compresses working memories into low-res ‘summaries’

#227

Post by mjevans »

TameAntelope wrote: post #96

I think this is why it's hard sometimes to argue in support of something you believe, even if you're right. At one point, all of the relevant facts and figures were loaded into your working memory, and with that information you arrived at a conclusion. Your brain, however, no longer needs those facts and figures; you've gotten what you needed from them, and they can be kicked out of working memory. What you store the…

It's extremely difficult to maintain a database of __all__ the citations for __anything__ you ever adjudicated (reached a decision).

Making things more easily findable and a database of debunked lies might be better.

Also great would be training (for anyone) on how to spot 'magic tricks' in debates / information presentation. E.G. how things might be cut down, remixed, or staged to create something that at a glance is convincing, but with closer examination could just be gaslighting.

Re: Human brain compresses working memories into low-res ‘summaries’

#228

Post by amelius »

konschubert wrote: post #201
axg11 wrote: post #35

Compression is a component of general intelligence. A few years ago I was very sceptical of machine learning ever leading to general intelligence. I've since changed my mind. There are a lot of parallels to this work and the concept of "embeddings" in machine learning. Intelligence requires the ability to generalize. A prerequisite for generalization is the ability to take something high-dimensional and reduce it to…

I find it funny how I can "see" a map of the world in front of me when I imagine it, but I totally cannot draw it. Clearly, much less information is stored than the whole image... yet my mind DALL-E style fills in the gaps and "sees" a map.

Keep drawing until what is on paper equals what is in your imagination. Seriously, try it.

Re: Human brain compresses working memories into low-res ‘summaries’

#229

Post by [deleted] »

axg11 wrote: post #210
konschubert wrote: post #201

Earlier quoted context omitted.

I find it funny how I can "see" a map of the world in front of me when I imagine it, but I totally cannot draw it. Clearly, much less information is stored than the whole image... yet my mind DALL-E style fills in the gaps and "sees" a map.

Already plugged this book elsewhere in the thread, you might be interested in "The Mind is Flat". One chapter of the book explores the concept you're describing. Our brain creates the illusion of a "full picture" when often our imagination and internal representation is quite sparse. I think that's one of the key impressive qualities of our brains and general intelligence. We only do the minimum necessary imagination…

[deleted]

Re: Human brain compresses working memories into low-res ‘summaries’

#230

Post by ascar »

freediver wrote: post #221
axg11 wrote: post #35

Compression is a component of general intelligence. A few years ago I was very sceptical of machine learning ever leading to general intelligence. I've since changed my mind. There are a lot of parallels to this work and the concept of "embeddings" in machine learning. Intelligence requires the ability to generalize. A prerequisite for generalization is the ability to take something high-dimensional and reduce it to…

I co-authored a paper exploring this topic few years ago, while I was pretty excited about the possiblity of using embeddings for generalization. "Towards conceptual generalization in the embedding space" https://arxiv.org/abs/1906.01873 I still think the approach outlined in the paper (using embeddings to map the physical world) is sound especially for the field of self-driving which is in dire need of generalizatio…

> I've since changed my mind and currently do not believe we can achieve AGI (ever).

Considering we (as in humans) developed general intelligence, isn't that already in contradiction with your statement? If it happened for us and is "easily" replicated through our DNA, it certainly can be developed again in an artificial medium. But the solution might not have anything to do with what we call machine learning today and sure we might go extinct before (but I didn't have the feeling that's what you were implying).

Post reply on HN