Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human brain compresses working memories into low-res ‘summaries’

directorsblog.nih.gov

Post reply on HN
Print view View original

241–250 of 349 posts

Re: Human brain compresses working memories into low-res ‘summaries’

#241

Post by freediver »

staticassertion wrote: post #231
freediver wrote: post #221

Earlier quoted context omitted.

I co-authored a paper exploring this topic few years ago, while I was pretty excited about the possiblity of using embeddings for generalization. "Towards conceptual generalization in the embedding space" https://arxiv.org/abs/1906.01873 I still think the approach outlined in the paper (using embeddings to map the physical world) is sound especially for the field of self-driving which is in dire need of generalizatio…

> currently do not believe we can achieve AGI (ever). Do you mean with embeddings as the approach, or in general?

I think AGI will remain out of reach. Even a simpler thing like level 5 self-driving, which is only like 0.3 AGI or something, will remain forever out of reach no matter how much compute we throw at it (though I also think that if we ever reach 0.3 AGI we will also reach 100%).

The reason is that the mundane world keeps surprising us everywhere we look and constantly keeps creating more questions than answers. Just look at the questions field of quantum mechanics is trying to tackle, but also every other field of research science - astronomy, genetics, biology, antropology even mathematics... Now imagine trying to keep up with all that - by writing code.

Also, mastering these things would make us 'God'.

Re: Human brain compresses working memories into low-res ‘summaries’

#242

Post by sva_ »

zasdffaa wrote: post #223
sva_ wrote: post #216

Earlier quoted context omitted.

Interesting thought. Perhaps that is also why people sometimes have a hard time changing their mind when confronted with new information: a certain number of bits of information have led you to your belief, and even if some of those change or turn out to be false, you can't access those bits anymore individually, but only the resulting belief. Perhaps, the more those beliefs are reinforced, the less likely you are to…

> why people sometimes have a hard time changing their mind when confronted with new information Something else happens with me, it's like my brain says "this does not fit in with what I understand, discard it". At a conscious level I don't hear what I've just been told. I have to be told it again, and sometimes more than twice before it finally works its way in. It's a liability for me and a frustration for others a…

I don't think this is too uncommon. I sometimes go through such a phase, also in reading, and what helps me get back on track is to do things really intently for a while. And I mean even basic things, being really aware of what I'm doing and thinking in that moment.

When you dont pay attention to what is currently happening, it's usually that your mind goes on tangents. I'd recommend becoming aware of those tangential thought processes. Mindful meditation may help a bit.

Re: Human brain compresses working memories into low-res ‘summaries’

#243

Post by axg11 »

freediver wrote: post #221
axg11 wrote: post #35

Compression is a component of general intelligence. A few years ago I was very sceptical of machine learning ever leading to general intelligence. I've since changed my mind. There are a lot of parallels to this work and the concept of "embeddings" in machine learning. Intelligence requires the ability to generalize. A prerequisite for generalization is the ability to take something high-dimensional and reduce it to…

I co-authored a paper exploring this topic few years ago, while I was pretty excited about the possiblity of using embeddings for generalization. "Towards conceptual generalization in the embedding space" https://arxiv.org/abs/1906.01873 I still think the approach outlined in the paper (using embeddings to map the physical world) is sound especially for the field of self-driving which is in dire need of generalizatio…

Thanks for your perspective. We’re still in disagreement but I wouldn’t bet on either side of the AGI debate with any significant conviction.

Embeddings are very good at a few things: combining concepts (addition), untangling commonalities (subtraction) and determining similarity between concepts (distance).

> While embeddings are a great tool for compressing information, they do not provide inherent mechanisms for manipulating the information stored

What are the manipulations you’re referring to? I would love to learn more. From my understanding, embeddings actually provide great generalisation. If you have a well conditioned embedding space then you can interpolate into previously unseen parts of that space and still get sensible results. That is generalisation to me. Many current ML methods do _not_ result in a fully meaningful embedding space but my hunch is that we will get there with future insights and advances.

Re: Human brain compresses working memories into low-res ‘summaries’

#244

Post by Lyapunov_Lover »

throwawaygo wrote: post #235

Surprise!! The human brain compresses all experience into low-res summaries. Full res is not possible. :D

It doesn't, actually.

There are high-res versions of important events. The locus coeruleus-noradrenaline system act in concerto with the amygdala to induce protein synthesis in the hippocampus to produce vivid episodic memories. It gets rewritten every time it's accessed, but it doesn't become abstracted away like the memory of mundane events.

Semantic memory is what we call our repertoire of facts and gists and otherwise abstract knowledge. You can lose your episodic memories while keeping your semantic memories and vice versa. Which is pretty cool.

Systems consolidation is the term for the hypothesized process where episodic memories are compressed and shipped out to the cerebral cortex for semantic storage.

You can use propranolol, a beta-blocker, to prevent noradrenaline from tagging a given episodic memory as "high priority" and you can sort of force it through the consolidation process this way. And yes, you can use it to treat PTSD.

Re: Human brain compresses working memories into low-res ‘summaries’

#245

Post by jamiek88 »

trompetenaccoun wrote: post #236
ascar wrote: post #230

Earlier quoted context omitted.

> I've since changed my mind and currently do not believe we can achieve AGI (ever). Considering we (as in humans) developed general intelligence, isn't that already in contradiction with your statement? If it happened for us and is "easily" replicated through our DNA, it certainly can be developed again in an artificial medium. But the solution might not have anything to do with what we call machine learning today a…

It's semantics at this point but we did not create ourselves, it was a complex process that took billions of years to create each one of us. Something being conceivable isn't the same as it being practically possible. I can imagine what you propose, but the same goes for traveling to distant stars or a time machine for going to the future. All perfectly possible in theory.

Yeah but interstellar and time travel haven’t been done, or at least we haven’t observed such.

Flight had. Intelligence has.

Re: Human brain compresses working memories into low-res ‘summaries’

#246

Post by playpause »

amelius wrote: post #228
konschubert wrote: post #201

Earlier quoted context omitted.

I find it funny how I can "see" a map of the world in front of me when I imagine it, but I totally cannot draw it. Clearly, much less information is stored than the whole image... yet my mind DALL-E style fills in the gaps and "sees" a map.

Keep drawing until what is on paper equals what is in your imagination. Seriously, try it.

Can you expand on this? Can you give an example of a kind of image it might work well with? I’ve always assumed the apparent detail of mental images was a kind of illusion, a bit like the illusion of detail outside the centre of the visual field.

Re: Human brain compresses working memories into low-res ‘summaries’

#247

Post by extheat »

freediver wrote: post #238
ascar wrote: post #230

Earlier quoted context omitted.

> I've since changed my mind and currently do not believe we can achieve AGI (ever). Considering we (as in humans) developed general intelligence, isn't that already in contradiction with your statement? If it happened for us and is "easily" replicated through our DNA, it certainly can be developed again in an artificial medium. But the solution might not have anything to do with what we call machine learning today a…

It is not a contradiction as I meant "achieving" in the context of creating it (through software). The fact it happened to us is undeniable (from our perspective), but the how/why of it is still one of the many mysteries of the universe - one we will likely never solve.

I’m curious why you think that. Do you think it’s a fundamental problem with the discrete nature of traditional computers? Or a problem with scale and computational limits? If it’s the latter, if a hypothetical computer has unlimited time and memory capacity, why do you think AGI would remain impossible?

Re: Human brain compresses working memories into low-res ‘summaries’

#248

Post by jamiek88 »

Traubenfuchs wrote: post #85
goaaron wrote: post #51

Earlier quoted context omitted.

The human brain also forgets, something that may be a feature instead of a bug. Also, beyond compression––brains are simulation machines: imagining new scenarios. Curios to understand if ML provides anything analogous to simulation that isn't rote interpolation.

People with hyperthymesia don‘t forget and don‘t necessarily seem to have any other potentially disabling neuroatypicality like autism. Having it is a premium feature. https://en.m.wikipedia.org/wiki/Hyperthymesia

>In fact, she was not very good at memorizing anything at all, according to the study published in Neurocase.[1] Hyperthymestic individuals appear to have poorer than average memory for arbitrary information.

So no, not premium. A trade off.

Re: Human brain compresses working memories into low-res ‘summaries’

#249

Post by freediver »

axg11 wrote: post #243
freediver wrote: post #221

Earlier quoted context omitted.

I co-authored a paper exploring this topic few years ago, while I was pretty excited about the possiblity of using embeddings for generalization. "Towards conceptual generalization in the embedding space" https://arxiv.org/abs/1906.01873 I still think the approach outlined in the paper (using embeddings to map the physical world) is sound especially for the field of self-driving which is in dire need of generalizatio…

Thanks for your perspective. We’re still in disagreement but I wouldn’t bet on either side of the AGI debate with any significant conviction. Embeddings are very good at a few things: combining concepts (addition), untangling commonalities (subtraction) and determining similarity between concepts (distance). > While embeddings are a great tool for compressing information, they do not provide inherent mechanisms for m…

> We’re still in disagreement but I wouldn’t bet on either side of the AGI debate with any significant conviction.

That is probably a superior position to hold. I am agnostic by nature, and interestingly this is one of the rare topics I've taken a hard position on. It could be a result of the years spent in the field but also some kind of bias.

> What are the manipulations you’re referring to?

Need to take a step back and mention that in the field of AI there is a great debate between symbolic and non-symbolic approaches. (and after decades spent with AI under symbolic approaches domination we are now in the golden age of non-symbolic AI; with symbolic starting to have a comeback. this podcast can be a good starting point to learn more https://lexfridman.com/gary-marcus/ - although I disagree with GM on many things - and this tweet for learning about symbolic making a comeback https://twitter.com/hardmaru/status/1470847417193209856)

Basically embeddings are "non-symbolic AI" (which is great and this is where their huge potential stems from), but the very way they are generated and then later utlized is completely "symbolic". Which means the the limits of embeddings is defined by the limits of (in this case human written) symbols used to define them. Hope that makes sense.

Re: Human brain compresses working memories into low-res ‘summaries’

#250

Post by kingcharles »

ThalesX wrote: post #58
ansible wrote: post #37

Earlier quoted context omitted.

I wonder if that's something that can be practiced and improved upon.

I've been trying for years to visualize. My SO is an artist and my mother a psychologist, so I've been trying to gets tips and tricks from them. I never managed to even get a hint of color.

I definitely think the ability is there and can be unlocked. Like the way you can sometimes figure out how to fire some random muscle on your body with practice.

While I was in jail I managed to imagine images in vivid color and detail several times, but never on demand. It would happen randomly when I was just lying, bored, thinking about stuff. Each time was one of the most amazing experiences of my life. It was like going from being a caveman looking at scratchings on a wall, to straight into a 3D IMAX movie. The quality was that spectacular, with bright colors and completely 3-dimensional. Since I left jail it hasn't happened again a single time.

I have to wonder if that's how others see inside their minds? If so, I am in awe.

Post reply on HN