Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unbearable slowness of being: Why do we live at 10 bits/s?

cell.com

Post reply on HN
Print view View original

191–200 of 325 posts

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#191

Post by engineer_22 »

IshKebab wrote: post #61
esperent wrote: post #32

Earlier quoted context omitted.

From the paper: > Quick, think of a thing... Now I’ll guess that thing by asking you yes/no questions.” The game ‘Twenty Questions’ has been popular for centuries1as a thinking challenge. If the questions are properly designed, each will reveal 1 bit of information about the mystery thing. If the guesser wins routinely, this suggests that the thinker can access about 220≈ 1 million possible items in the few seconds a…

It's nice when authors let you know you can safely ignore them so succinctly!

Yeah, no reason to keep reading

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#192

Post by FartyMcFarter »

ComplexSystems wrote: post #190
tsimionescu wrote: post #100

It seems very odd that the article seems to be measuring the information content of specific tasks that the brain is doing or specific objects that it is perceiving. But the brain is a general-purpose computer, not a speed-card computer, or English text computer, or binary digit computer, or Rubik's cube computer. When you look at a Rubik's cube, you don't just pick out specific positions of colored squares relative…

On the other hand, the amount of actual entropic "information" that is processed when you identify a Rubik's cube as such may be nowhere near as much as you think it is, and most importantly, 10 bits may be nowhere near as little as you think it is. If we use your example, which is that of identifying an object, we may simply ask the entropy of what the distribution of possible objects-to-be-identified is at t=0, pri…

But you don't just perceive an object's category (like "cat"). We also perceive a high amount of detail about the object - colour, pattern, behaviour, we make comparisons to past behaviour, predictions of what's likely to happen next and so on.

Sure, some parts of the brain don't receive all that detail, but that's necessary for abstraction. If you pumped all the sensory data everywhere, the brain would get overwhelmed for no reason.

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#193

Post by dantillberg »

tsimionescu wrote: post #100

It seems very odd that the article seems to be measuring the information content of specific tasks that the brain is doing or specific objects that it is perceiving. But the brain is a general-purpose computer, not a speed-card computer, or English text computer, or binary digit computer, or Rubik's cube computer. When you look at a Rubik's cube, you don't just pick out specific positions of colored squares relative…

> deciding that this is the most relevant activity to keep doing at the moment, instead of doing math or going out for a walk.

How many bits of actual decision is going on here, as compared to the period of time that decision applies to?

For example, if a person decided once per second whether or not to go for a walk, that could be 1 bit per second. But if that person is constantly transitioning back and forth between walking and not-walking, we could consider their behavior pathological. Instead, for most people, the information density of these decisions is quite low, i.e. the per-second decision bits are very compressible.

Personally, I only decide whether to go for a walk (or not) _at most_ once every few minutes. Even if we add in bits for "where" and "precisely when" and "how long", I think we're still at just a small fraction of 1 bit per second.

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#194

Post by ComplexSystems »

wat10000 wrote: post #23
crazygringo wrote: post #17

Where do they get 10 bits/second? Heck, I can type way faster than 10 bits per second, even after gzipping the output. And when I consider the amount of sensory information that I consciously process (not that comes in, but that I conceptually analyze), it's got to be way higher. 10 bits/s doesn't pass the smell test.

English is about one bit per letter. If you type at a very fast 120WPM then you’re right at 10bps. Computers just don’t represent English very efficiently, even with gzip.

These letters are jointly distributed, and the entropy of the joint distribution of a second of "plausible" English text is much lower than the naive sum of the marginal entropies of each letter. In fact, with LLMs that report the exact probability distribution of each token, it is now possible to get a pretty decent estimate of what the entropy of larger segments of English text actually is.

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#195

Post by FartyMcFarter »

sigmoid10 wrote: post #142
gtirloni wrote: post #125

Earlier quoted context omitted.

My thoughts exactly. It makes no sense to me that what I'm thinking and perceiving in real-time is the equivalent of 10 bit/s of data.

Has anyone here even read more than the title? The article literally explains that you perceive at a rate of 10^9 bits per second. But after filtering and preprocessing in the outer brain you are only left with about 10 bits per second for conscious processing for things like motor functions. Yes, you can see a Rubic's cube and perceive all sorts of facts about it and the environment at the same time. But try solving…

That still doesn't explain strange sentences in the article such as"

> Why does the brain need billions of neurons to process 10 bits/s?

Maybe the article is being intentionally disingenuous here? The brain is definitely not processing 10 bits/s, maybe a small part of it is.

I'd also say that you can make very complicated problems with only 10 bits of input (e.g. calculating busy-beaver of N), so "processing X bits" where X is a small value is no guarantee that this should be doable with a low amount of processing anyway.

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#196

Post by kijin »

scotty79 wrote: post #180
kijin wrote: post #178

Earlier quoted context omitted.

Send a query to your favorite database. The database preprocesses hundreds of gigabytes of data, map-reduces it, and finally returns a single 32-bit integer, taking exactly 3.2 seconds to do so. Nobody would say that the database can only process 10 bits per second. The query just happened to ask for a very simplified answer.

Everybody would say it outputs 10 bits per second. And when it comes to consciousness simplified answers at 10 bits per second is the best you can get. This article asks why.

It's an interesting question indeed, why the mind produces so little output compared to input.

Still, I get the feeling that the apparent bit rate of the conscious output is not the right metric to be measuring. There are so many ways in which the bit rate you're measuring could have been bottle-necked by factors outside of the mind, such as the speed at which you can move your tongue to articulate language, or a cultural expectation that prefers compact answers over chain-of-thought babbles. The mind also leaks a lot of information, both consciously and unconsciously, through side channels such as facial expression, gestures, and even variations in the rate of output itself.

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#197

Post by lukan »

dantillberg wrote: post #193
tsimionescu wrote: post #100

It seems very odd that the article seems to be measuring the information content of specific tasks that the brain is doing or specific objects that it is perceiving. But the brain is a general-purpose computer, not a speed-card computer, or English text computer, or binary digit computer, or Rubik's cube computer. When you look at a Rubik's cube, you don't just pick out specific positions of colored squares relative…

> deciding that this is the most relevant activity to keep doing at the moment, instead of doing math or going out for a walk. How many bits of actual decision is going on here, as compared to the period of time that decision applies to? For example, if a person decided once per second whether or not to go for a walk, that could be 1 bit per second. But if that person is constantly transitioning back and forth betwee…

"Personally, I only decide whether to go for a walk (or not) _at most_ once every few minutes. "

Consciously. Subconsciously much more is going on.

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#198

Post by causi »

dantillberg wrote: post #193
tsimionescu wrote: post #100

It seems very odd that the article seems to be measuring the information content of specific tasks that the brain is doing or specific objects that it is perceiving. But the brain is a general-purpose computer, not a speed-card computer, or English text computer, or binary digit computer, or Rubik's cube computer. When you look at a Rubik's cube, you don't just pick out specific positions of colored squares relative…

> deciding that this is the most relevant activity to keep doing at the moment, instead of doing math or going out for a walk. How many bits of actual decision is going on here, as compared to the period of time that decision applies to? For example, if a person decided once per second whether or not to go for a walk, that could be 1 bit per second. But if that person is constantly transitioning back and forth betwee…

The vast majority of the brain's processing power, including conceptual processing, is not conscious. Conscious thought is the framebuffer of the mind, subconscious is the RAM.

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#199

Post by dsr_ »

dantillberg wrote: post #193
tsimionescu wrote: post #100

It seems very odd that the article seems to be measuring the information content of specific tasks that the brain is doing or specific objects that it is perceiving. But the brain is a general-purpose computer, not a speed-card computer, or English text computer, or binary digit computer, or Rubik's cube computer. When you look at a Rubik's cube, you don't just pick out specific positions of colored squares relative…

> deciding that this is the most relevant activity to keep doing at the moment, instead of doing math or going out for a walk. How many bits of actual decision is going on here, as compared to the period of time that decision applies to? For example, if a person decided once per second whether or not to go for a walk, that could be 1 bit per second. But if that person is constantly transitioning back and forth betwee…

Your conscious planner may be making 1 decision/second, but your senses, proprioception, balancing system, etc. are handing you about a gigabit/second of data, most of which never rises to your conscious attention.

When I'm reading, that's roughly 2000 bits/second, but I am engaging it with model-making systems that can completely enrapture me.

I/O is not the same as computation; conscious computation is not the same as all computation.

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#200

Post by nonameiguess »

bennettnate5 wrote: post #75

> Why can we only think about one thing at a time? Maybe this is just a perception thing. Sure, you can only really keep up one stream of thought, visualization or inner dialogue (whatever you want to call it) at a time, but perhaps that's because we learn all our lives that direct communication is a one-channel, linear thing--speaking and listening focused on one topic at a time. Our brain does plenty of thinking in…

I don't know what ever became of the line of research, but there was a very interesting book I read decades ago called Descartes' Error by Antonio Damasio that examined case studies of patients who had their corpus collosum severed, resulting in a split brain. You could show their left and right eyes different images and ask them what they saw and they would write and speak different answers, because speech and writing are controlled by different brain hemispheres.

This seems to suggest that any bottleneck in conscious attention is not an inherent limitation of an animal brain but rather a consensus mechanism we've developed to keep our chain of experience coherent. If we get rid of the constraint that all of our external communication channels need to present the same narrative, we can seemingly process more information even when it requires being a conscious center of attention.

Post reply on HN