Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unbearable slowness of being: Why do we live at 10 bits/s?

cell.com

Post reply on HN
Print view View original

61–70 of 325 posts

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#61

Post by IshKebab »

esperent wrote: post #32
crazygringo wrote: post #17

Where do they get 10 bits/second? Heck, I can type way faster than 10 bits per second, even after gzipping the output. And when I consider the amount of sensory information that I consciously process (not that comes in, but that I conceptually analyze), it's got to be way higher. 10 bits/s doesn't pass the smell test.

From the paper: > Quick, think of a thing... Now I’ll guess that thing by asking you yes/no questions.” The game ‘Twenty Questions’ has been popular for centuries1as a thinking challenge. If the questions are properly designed, each will reveal 1 bit of information about the mystery thing. If the guesser wins routinely, this suggests that the thinker can access about 220≈ 1 million possible items in the few seconds a…

It's nice when authors let you know you can safely ignore them so succinctly!

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#62

Post by imtringued »

GeoAtreides wrote: post #21

I beg you, please read the paper before commenting. It's very interesting and it answers a lot of questions that might arise from just skimming the title.

I don't need to read the paper. The problem is that mechanical systems have inertia and are limited in their ability to change direction and thereby their ability to signal discrete information.

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#63

Post by GeoAtreides »

nis0s wrote: post #34
GeoAtreides wrote: post #18

Earlier quoted context omitted.

I have read the paper and your thesis, that the analysis is assuming a serial communication mode for neuronal transmission, is incorrect.

They say this directly in section 7.2, > In contrast, central processing appears to be strictly serial… and then they proceed to give misinterpretated evidence of serialization because they’re making assumptions about lower level biochemical behavior based on higher level tissue performance. In fact, that tissue-level behavior isn’t correctly described either.

Be honest.

The whole paragraph is:

"In contrast, central processing appears to be strictly serial: When faced with two tasks in competition, individuals consistently encounter a “psychological refractory period” before being able to perform the second task broadbent_perception_1958, pashler_dual-task_1994. Even in tasks that do not require any motor output, such as thinking, we can pursue only one strand at a time."

Clearly they're not talking about "neuronal transmission", but tasks, and further more, they cite their sources.

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#64

Post by wat10000 »

malfist wrote: post #58
wat10000 wrote: post #48

Earlier quoted context omitted.

It’s an experimental result by Shannon: https://archive.org/details/bstj30-1-50/page/n5/mode/1up In short, you show someone an English text cut off at an arbitrary point and ask them to predict which letter comes next. Based on how successful they are, you can calculate the information content of the text. The result from this experiment was approximately one bit per letter. Representing it is not the concern of the…

Say that experiment is correct. Wouldn't that imply that the information context of a single letter varies based on the possible future permutations? I.e., The string "I'v_" provides way more context than "con_" because you're much more likely to get I'm typing "I've" instead of "contraception" That seems to disprove the idea that a letter is a bit. Also the fact that there are more than two letters also indicate mor…

I don’t follow. Of course the probabilities change depending on context. 1 bit per letter is an average, not an exact measure for each individual letter. There are cases where the next letter is virtually guaranteed, and the information content of that letter is much less than one bit. There are cases where it could easily be many different possibilities and that’s more than one bit. On average it’s about one bit.

> Also the fact that there are more than two letters also indicate more than one bit

This seems to deny the possibility of data compression, which I hope you’d reconsider, given that this message has probably been compressed and decompressed several times before it gets to you.

Anyway, it should be easy to see that the number of bits per symbol isn’t tied to the number of symbols when there’s knowledge about the structure of the data. Start with the case where there are 256 symbols. That implies eight bits per symbol. Now take this comment, encode it as ASCII, and run it through gzip. The result is less than 8 bits per symbol.

For a contrived example, consider a case where a language has three symbols, A, B, and C. In this language, A appears with a frequency of 999,999,998 per billion. B and C each appear with a frequency of one in a billion. Now, take some text from this language and apply a basic run-length encoding to it. You’ll end up with something like 32 bits per billion letters on average (around 30 bits to encode a typical run length of approximately 1 billion, and 2 bits to encode which letter is in the run), which is way less than one bit per letter.

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#65

Post by rvense »

wat10000 wrote: post #49
rvense wrote: post #43

Earlier quoted context omitted.

How do you measure information density of English text?

You show a bunch of English speakers some text that’s cut off, and ask them to predict the next letter. Their success at prediction tells you the information content of the text. Shannon ran this experiment and got a result of about 1 bit per letter: https://archive.org/details/bstj30-1-50/page/n5/mode/1up

OK. When talking about language I find it's always good to be explicit about what level you're talking about, especially when you're using terms as overloaded as "information". I'm not really sure how to connect this finding to semantics.

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#66

Post by wat10000 »

rvense wrote: post #65
wat10000 wrote: post #49

Earlier quoted context omitted.

You show a bunch of English speakers some text that’s cut off, and ask them to predict the next letter. Their success at prediction tells you the information content of the text. Shannon ran this experiment and got a result of about 1 bit per letter: https://archive.org/details/bstj30-1-50/page/n5/mode/1up

OK. When talking about language I find it's always good to be explicit about what level you're talking about, especially when you're using terms as overloaded as "information". I'm not really sure how to connect this finding to semantics.

If the text can be reproduced with one bit per letter, then the semantic information content is necessarily at most equal to N bits where N is the length of the text in letters. Presumably it will normally be much less, since there are things like synonyms and equivalent word ordering which don’t change the meaning, but this gives a solid upper bound.

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#67

Post by MrMcCall »

GeoAtreides wrote: post #59
MrMcCall wrote: post #25

Earlier quoted context omitted.

That might be the funniest comment I've ever seen on HN! A plea to reason, that is probably not outside the posting guidelines, but is certainly in a gray area :-)

I honestly don't understand why it would be funny or in a gray area to recommend people to actual read the paper?

Asking people to read the article before commenting? A commonsense suggestion that needs to be made makes me smirk inside, not the least because I am guilty of this, too, around here. (But not this time, thank you, kind Sir.)

As to being in a "gray area", have you read the posting guidelines? ;-)

I'm pretty sure it says we shouldn't say things like "read the article" or "you haven't read the article, have you?" in our comments.

Anyway, I'm laughing at this community (myself included) and the fact that your innocent and well-intentioned comment needs to be said here. And it did and does, my friend!

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#68

Post by MrMcCall »

ganzuul wrote: post #54
GeoAtreides wrote: post #21

I beg you, please read the paper before commenting. It's very interesting and it answers a lot of questions that might arise from just skimming the title.

I conclude that if you perform horrific experiments on animals then our intelligent universe reduces the rate at which you can continue to 10bps. This is why enlightenment cures you of your curiosity.

Only 10 beatings per second? This is a just universe, Sir!

On a serious note, enlightenment only cures us of our selfish curiosity, i.e. any action which causes harm to others. The Way requires us to harmonize with universal compassion, so there is take and give (especially with regard to our required sustenance), but we definitely lose our propensity to experiment with our power at the expense of others. No, we are to increase our curiosity in how we can better help others, help being the cornerstone of compassion.

Re: The unbearable slowness of being: Why do we live at 10 bits/s?

#70

Post by uoaei »

MrMcCall wrote: post #33

The only time 'bits' will ever be an appropriate measure of human processing is when we are processing or producing diginal information artifacts, e.g. a typewritten document. Our bodies' systems are biochemical wetware that will never be aptly described using a boolean basis. That is one of the primary problems of society's obsessions with classical notions of gender. No one is male OR female. We are, every single o…

I think you've mixed up a few mostly unrelated things together to make a point. You're correct in that the larger point to be made is that analog and digital computing are paradigmatically distinct and analogies are hard to draw across that divide.

However, "bits" is just a quantity of information in a certain base. We could discuss it in "nits" if you prefer. The point is that information per se remains real even if the specific representation is based on some assumption of digital computing.

The rest of your comment is unfortunately out of scope of this article although it deserves some discussion on its own merit.

Post reply on HN