Live data from Hacker News

Human brain compresses working memories into low-res ‘summaries’

directorsblog.nih.gov

231–240 of 349 posts

Re: Human brain compresses working memories into low-res ‘summaries’

#231
post #35

Compression is a component of general intelligence. A few years ago I was very sceptical of machine learning ever leading to general intelligence. I've since changed my mind. There are a lot of parallels to this work and the concept of "embeddings" in machine learning. Intelligence requires the ability to generalize. A prerequisite for generalization is the ability to take something high-dimensional and reduce it to…

I co-authored a paper exploring this topic few years ago, while I was pretty excited about the possiblity of using embeddings for generalization. "Towards conceptual generalization in the embedding space" https://arxiv.org/abs/1906.01873 I still think the approach outlined in the paper (using embeddings to map the physical world) is sound especially for the field of self-driving which is in dire need of generalizatio…

> currently do not believe we can achieve AGI (ever).

Do you mean with embeddings as the approach, or in general?

Re: Human brain compresses working memories into low-res ‘summaries’

#232
“The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far.“

This is somewhat like an inference from best estimate used to develop a plan and then disregard that and implement the plan. Why the design of your plan is important to get right, because it’s about to be thrown away. There is even a certain trauma or frustration with having to go backwards, unless you’re prepared for it. Have you pulled your hair out on being questioned all the time by passer bys: why are you doing that? Why don’t you do it this way? (Usually best translated as why aren’t you/why don’t you do it my way) By someone who has no conception of the system that produced the implementation plan? Because I am! Grrr! Or you core dump everything on them and you get: sorry I even asked. Or you go along with it only to find later there was a good reason you were doing it the original way and now there’s a lock on the crit path.

This is disregarding the times you’re the one who is wrong.

Which also hints at why logos is hard. Same with debugging. The sanctity of the system that produces the outcomes. Constantly having to remember details. What is happening? Why is it happening? How do you know? How can it be otherwise? Non technical people seem to be able to get away with the first idea that comes to mind, unexamined.

Frameworks, shortcuts, assumptions are developed only at some point to fail you and shoot you right back to first principles. Or you never leave them and the unconcerned dance circles around you. I heard you’ve been having trouble with your tps reports?

Lua indexes from 1 not 0! Are you kidding me!!!? ;_; I went through 5 Adams before I figured that out.

“Professor Henry Jones : Oh, yes. But I found the clues that will safely take us through them in the Chronicles of St. Anselm. Indiana Jones : [pleased] Well, what are they? [short pause as Henry tries to recall] Indiana Jones : Can't you remember? Professor Henry Jones : I wrote them down in my diary so that I wouldn't have to remember. Indiana Jones : [angry] Half the German Army's on our tail and you want me to go to Berlin? Into the lion's den?“

To extend further, is that why don’t touch my stapler? Get out of my chair?

Re: Human brain compresses working memories into low-res ‘summaries’

#233
post #35

Compression is a component of general intelligence. A few years ago I was very sceptical of machine learning ever leading to general intelligence. I've since changed my mind. There are a lot of parallels to this work and the concept of "embeddings" in machine learning. Intelligence requires the ability to generalize. A prerequisite for generalization is the ability to take something high-dimensional and reduce it to…

I saw a presentation once (It was not recorded, so I can't link it) that said the difference between real intelligence and artificial intelligence is the ability to quickly learn. As an example, he said imagine if he invented a word. Poditon. And he told us that a poditon is any object that can fit on a podium. Instantly, you know whether or not any object can be classified as a poditon. A laptop is a poditon, but a…

So, I tried this out with GPT-3 in the OpenAI Playground. (The model called text-davinci-002.) My prompt looked like this:

    Definition: A blorple is anything that is red and more or less round.
    Question: Is a tennis ball a blorple?
    Answer: No, because although tennis balls are round they aren't red.
    Question: Is a cherry a blorple?
    Answer: Yes, because cherries are red and approximately round.

    Definition: A poditon is anything that can fit on top of a podium.
    Question: Is a laptop computer a poditon?
    Answer:
GPT-3 says: "Yes, because laptop computers are small enough to fit on top of a podium."

Is a normal-sized automobile a poditon? "No, an automobile is too large to fit on top of a podium."

Is the sun a poditon? "No, because the sun is too large to fit on top of a podium."

Is a human being a poditon? "Yes, because human beings are small enough to fit on top of a podium."

Is a house a poditon? "No, because a house is too large to fit on top of a podium."

While generating those answers it also spontaneously answered the question for tennis balls (yes) and books (yes).

Decades sure do go by quickly, these days.

Re: Human brain compresses working memories into low-res ‘summaries’

#234
post #37

Earlier quoted context omitted.

I wonder if that's something that can be practiced and improved upon.

I've been wondering the same ever since I read that Nikola Tesla invented/designed the AC motor in his mind's eye. Seems to be along the lines of lucid dreaming, with a vast difference in degree. Sometimes as I'm falling asleep I can see vivid scenes or objects that I can--to minor degrees--play with for a short time before I either fall asleep or wake up, then it's gone.

also, consider somebody who is an expert already in the problem domain.

most of us here are programmers and do this on a daily basis. somebody describes "A GraphQL API driven by a clojure back end connected to a postgres database" and to a layperson that looks like either a bunch of nonsense words or maybe a few boxes, clouds, and arrows. but to you and me we can visualize the individual lines of code, configurations, functions, and infrastructural requirements behind that simple sentence.

same with an electrical engineer/inventor in their domain.

Re: Human brain compresses working memories into low-res ‘summaries’

#236
post #230

Earlier quoted context omitted.

I co-authored a paper exploring this topic few years ago, while I was pretty excited about the possiblity of using embeddings for generalization. "Towards conceptual generalization in the embedding space" https://arxiv.org/abs/1906.01873 I still think the approach outlined in the paper (using embeddings to map the physical world) is sound especially for the field of self-driving which is in dire need of generalizatio…

> I've since changed my mind and currently do not believe we can achieve AGI (ever). Considering we (as in humans) developed general intelligence, isn't that already in contradiction with your statement? If it happened for us and is "easily" replicated through our DNA, it certainly can be developed again in an artificial medium. But the solution might not have anything to do with what we call machine learning today a…

It's semantics at this point but we did not create ourselves, it was a complex process that took billions of years to create each one of us. Something being conceivable isn't the same as it being practically possible. I can imagine what you propose, but the same goes for traveling to distant stars or a time machine for going to the future. All perfectly possible in theory.

Re: Human brain compresses working memories into low-res ‘summaries’

#237
post #35

Compression is a component of general intelligence. A few years ago I was very sceptical of machine learning ever leading to general intelligence. I've since changed my mind. There are a lot of parallels to this work and the concept of "embeddings" in machine learning. Intelligence requires the ability to generalize. A prerequisite for generalization is the ability to take something high-dimensional and reduce it to…

I saw a presentation once (It was not recorded, so I can't link it) that said the difference between real intelligence and artificial intelligence is the ability to quickly learn. As an example, he said imagine if he invented a word. Poditon. And he told us that a poditon is any object that can fit on a podium. Instantly, you know whether or not any object can be classified as a poditon. A laptop is a poditon, but a…

If you've seen examples of GPT3 you'd know this is already possible.

Re: Human brain compresses working memories into low-res ‘summaries’

#238
post #230

Earlier quoted context omitted.

I co-authored a paper exploring this topic few years ago, while I was pretty excited about the possiblity of using embeddings for generalization. "Towards conceptual generalization in the embedding space" https://arxiv.org/abs/1906.01873 I still think the approach outlined in the paper (using embeddings to map the physical world) is sound especially for the field of self-driving which is in dire need of generalizatio…

> I've since changed my mind and currently do not believe we can achieve AGI (ever). Considering we (as in humans) developed general intelligence, isn't that already in contradiction with your statement? If it happened for us and is "easily" replicated through our DNA, it certainly can be developed again in an artificial medium. But the solution might not have anything to do with what we call machine learning today a…

It is not a contradiction as I meant "achieving" in the context of creating it (through software).

The fact it happened to us is undeniable (from our perspective), but the how/why of it is still one of the many mysteries of the universe - one we will likely never solve.

Re: Human brain compresses working memories into low-res ‘summaries’

#239
post #94

Earlier quoted context omitted.

Isn't wolfram's new kind of science purely rational? No observations of the universe needed

The universe is required to run Mathematica.

not if the universe is running inside Mathematica (our mathematica doesn't have the space, but the one we are inside does)

Re: Human brain compresses working memories into low-res ‘summaries’

#240
post #2

> It turned out that either visual stimulus—the grating or moving dots—resulted in the same patterns of neural activity in the visual cortex and parietal cortex. The parietal cortex is a part of the brain used in memory processing and storage. > These two distinct visual memories carrying the same relevant information seemed to have been recoded into a shared abstract memory format. As a result, the pattern of brain…

Exactly. Our tooling doesn't even allow us to make clear inferences about neuronal activity yet - all we can image with fMRI as of today is heamodynamics.

The work proposes that because responses to difference visual stimuli show the same haemodynamic spatiotemporal response in memory areas, the actual memories must share a common representation. This is debatable since we know that localized spatiotemporal responses reflect the variability in the vascular tree and blood flow and volume, regardless of the experiment at hand [see 1 for a discussion]. To dare to claim that effects could be neuronal one needs to run a bunch of extra control experiments (vascular reactivity mapping / breathhold hypercapnic challenges, resting-state imaging as a control dataset), none of which were conducted in the proposed work.

This is known to MR physicists, but hasn't clicked in yet within departments of cognitive neuroscience.

[1] https://www.frontiersin.org/articles/10.3389/fnins.2020.5960...

Post reply on HN