Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human brain compresses working memories into low-res ‘summaries’

directorsblog.nih.gov

Post reply on HN
Print view View original

271–280 of 349 posts

Re: Human brain compresses working memories into low-res ‘summaries’

#271

Post by whatshisface »

jameshart wrote: post #267
freediver wrote: post #254

Earlier quoted context omitted.

Machines are good at computation, which is not equal to reasoning, but rather a subset of reasoning. And not only they are good at computation, but they are exceptionally good at it - I have no illusion of trying to compete with a machine doing square roots or playing chess. And increasingly harder problems are being expressed as computation problems, with more or less success - most famously probably self-driving. B…

Ever since Gödel we’ve had a pretty convincing proof that there is nothing that you can do in terms of reasoning that can’t be expressed using computation. And since Turing we’ve got a framework that shows there’s nothing computable that you can’t compute using a universal computer. So unless there’s something mystical beyond the realm of mathematics to ‘reasoning’ it can’t be a superset of computing. If a finite amo…

There are actually a lot of well-defined things beyond the power of a Turing machine (for example a Turing machine plus a halting oracle that only works on Turing machines without a halting oracle) but in terms of finite amounts of electrons doing normal low-energy electronic stuff you are quite likely correct. Humanity may go beyond computability if as some papers have suggested quantum gravity requires solving uncomputable problems.

Re: Human brain compresses working memories into low-res ‘summaries’

#272

Post by throwawaygo »

Lyapunov_Lover wrote: post #261
throwawaygo wrote: post #258

Earlier quoted context omitted.

Any representation of reality should not be confused with reality. The map is not the terrain. The menu is not the meal. -post-modern philosopherzzz It’s true varying resolutions are stored and retrieved but all are a reduction of the inputs to some useful representation.

I think my use of the term "high-res" already implied that.

Yes and it’s completely arbitrary. My only point is there isn’t really anything new or interesting here, sadly.

Re: Human brain compresses working memories into low-res ‘summaries’

#273

Post by karmakaze »

LesZedCB wrote: post #234
karmakaze wrote: post #47

Earlier quoted context omitted.

I've been wondering the same ever since I read that Nikola Tesla invented/designed the AC motor in his mind's eye. Seems to be along the lines of lucid dreaming, with a vast difference in degree. Sometimes as I'm falling asleep I can see vivid scenes or objects that I can--to minor degrees--play with for a short time before I either fall asleep or wake up, then it's gone.

also, consider somebody who is an expert already in the problem domain. most of us here are programmers and do this on a daily basis. somebody describes "A GraphQL API driven by a clojure back end connected to a postgres database" and to a layperson that looks like either a bunch of nonsense words or maybe a few boxes, clouds, and arrows. but to you and me we can visualize the individual lines of code, configurations…

I wouldn't say that I exactly 'visualize' it. For me it's more of a bunch of formless even nameless things and connections that don't have specific shapes or places, only logical structures. The lines of code also flows freely but it feels more like synthesis from those ideas rather than copy-typing. Eventually the 'picture' or network in my mind gets complicated/unstable and I have to draw it out.

Re: Human brain compresses working memories into low-res ‘summaries’

#274

Post by karmakaze »

kaba0 wrote: post #126
karmakaze wrote: post #47

Earlier quoted context omitted.

I've been wondering the same ever since I read that Nikola Tesla invented/designed the AC motor in his mind's eye. Seems to be along the lines of lucid dreaming, with a vast difference in degree. Sometimes as I'm falling asleep I can see vivid scenes or objects that I can--to minor degrees--play with for a short time before I either fall asleep or wake up, then it's gone.

I would take something written about Tesla on such an intimate level with a grain of salt though. He is very very hyped and often elevated to a God-like level.

I don't doubt that some people have this ability and have no reason to think Tesla wasn't one of them.

His feats are perhaps even more remarkable if he didn't have any superpower abilities such as this.

Re: Human brain compresses working memories into low-res ‘summaries’

#275

Post by nl »

a_square_peg wrote: post #269
axg11 wrote: post #35

Compression is a component of general intelligence. A few years ago I was very sceptical of machine learning ever leading to general intelligence. I've since changed my mind. There are a lot of parallels to this work and the concept of "embeddings" in machine learning. Intelligence requires the ability to generalize. A prerequisite for generalization is the ability to take something high-dimensional and reduce it to…

> A prerequisite for generalization is the ability to take something high-dimensional and reduce it to a lower-dimensional representation to allow comparison and grouping of concepts. I've been thinking that it might actually the other way around - intelligence is taking lower dimensional data and being able to infer higher level representation in terms of context, meanings and other abstractions. I.e. understanding…

It's actually the compression that forces it to learn higher level concepts.

In your stop sign example, say we are trying to teach a visual model the difference between toy stop signs and real stop signs.

To train it you feed it a 3D model of the world and the actions a person takes in response (ie, ignoring toy stop signs but stopping for real ones). Once the embedding is well trained (with lots of data) if you then run it through something like UMAP to reduce the number of dimensions in the embedding from hundreds to 2 or 3 you'll see it has "discovered" the concept of "scale" - all the small toy stop signs will be clustered together and the real ones clustered elsewhere.

That generalisation forced by compression is where the abstraction of "scale" comes from.

(Of course in real life you'd use a more complex model than just an embedding for this, but in principle this is the idea).

Re: Human brain compresses working memories into low-res ‘summaries’

#276

Post by [deleted] »

gryn wrote: post #205
MichaelBurge wrote: post #180

Earlier quoted context omitted.

A definition doesn't change: The prime numbers or Turing Machines are the same set regardless of who Putin invades next or what law Biden decides to veto. But the set of inputs that an oracle implicitly defines, could change if the oracle changes. And you could change your mind or die tomorrow. So you would need a very large number of definitions of bread, indexed by (time, person). Any one of them could be a valid d…

> A definition doesn't change but they do, the definition of many words changed over time some to even start to mean the opposite of what they initially did.

[deleted]

Re: Human brain compresses working memories into low-res ‘summaries’

#277

Post by MichaelBurge »

addaon wrote: post #213
MichaelBurge wrote: post #165

Earlier quoted context omitted.

That makes you a bread-oracle O, but doesn't define bread. Since there are some inputs x where O(x0) = False, some where O(x1) = True, and the laws of physics are continuous(yes, even in quantum mechanics), Buridan's Principle implies that you are incapable of deciding the breadness of arbitrary input in bounded time.

I don't think this is true? Suppose I define "bread" as "that which has a net positive charge" [1]. Can I not put the bread candidate in an electric field in flat spacetime and measure (the direction of) its acceleration in a bounded time? I suppose I might be depending on its mass being finite, but the observable universe supports that assumption. [1] I don't think this is a very useful definition of bread.

The original paper in Section 6 looks at the case of a magnetic field: https://lamport.azurewebsites.net/pubs/buridan.pdf

It's a practical issue that affects CPU design, which is also mentioned in the paper under the name "Arbiter Problem".

Re: Human brain compresses working memories into low-res ‘summaries’

#278

Post by Lyapunov_Lover »

throwawaygo wrote: post #272
Lyapunov_Lover wrote: post #261

Earlier quoted context omitted.

I think my use of the term "high-res" already implied that.

Yes and it’s completely arbitrary. My only point is there isn’t really anything new or interesting here, sadly.

There's a difference between low-res and high-res; that's what I'm getting at. I think it's interesting to know a little about how it might work.

There is something new and interesting about the topic to many people. And the paper itself represents a small blip of progress, though it must of course be hyped up to lure in eyeballs.

Novelty is relative.

Re: Human brain compresses working memories into low-res ‘summaries’

#279

Post by nl »

Sohcahtoa82 wrote: post #175
axg11 wrote: post #35

Compression is a component of general intelligence. A few years ago I was very sceptical of machine learning ever leading to general intelligence. I've since changed my mind. There are a lot of parallels to this work and the concept of "embeddings" in machine learning. Intelligence requires the ability to generalize. A prerequisite for generalization is the ability to take something high-dimensional and reduce it to…

I saw a presentation once (It was not recorded, so I can't link it) that said the difference between real intelligence and artificial intelligence is the ability to quickly learn. As an example, he said imagine if he invented a word. Poditon. And he told us that a poditon is any object that can fit on a podium. Instantly, you know whether or not any object can be classified as a poditon. A laptop is a poditon, but a…

> We are not at the stage where we can just tell a program "Any object that can fit on a podium is a poditon" and then ask "Is X a poditon?" and get a correct answer. And we probably won't be there for another couple decades.

If that presenter actually said that, they need to take a look at "Few shot learning in language models" (just Google the term and start reading the papers).

Re: Human brain compresses working memories into low-res ‘summaries’

#280

Post by skhr0680 »

axg11 wrote: post #35

Compression is a component of general intelligence. A few years ago I was very sceptical of machine learning ever leading to general intelligence. I've since changed my mind. There are a lot of parallels to this work and the concept of "embeddings" in machine learning. Intelligence requires the ability to generalize. A prerequisite for generalization is the ability to take something high-dimensional and reduce it to…

I’m bilingual from moving countries at a young age and now a lot of my early childhood memories are in the “wrong” language
Post reply on HN