Earlier quoted context omitted.
As someone without an inner monologue, and someone that's spent a LOT of time meditating, it's not the language. It's the attention mechanisms themselves. Buddhist scholars insist that while we can have multiple threads of attention in our awareness, like strings with pearls of experience/thoughts we can only actually hold one little pearl of information from that stream in our attention at a time, and that we flit b…
Sometimes I'll be deeply thinking about something while driving, and discover I'm at the last road to my house without remembering having driven the previous few blocks. It's quite disturbing. When I say deeply thinking I don't mean anything involving phones or external stimuli - really just thinking about a particular problem I'm working on. I also don't deliberately engage this deep mode of thought, I just sort of…
The unbearable slowness of being: Why do we live at 10 bits/s?
121–130 of 325 posts
Re: The unbearable slowness of being: Why do we live at 10 bits/s?
#122Earlier quoted context omitted.
Thank you for explaining what an abstract is... The fact that those number come from a citation doesn't make them true. This is a badly written paper that a decent researcher wouldn't have written (and I know that the author has many papers, I am speaking about this one) and a decent reviewer would have rejected. A paragraph about Elon Musk? Guesstimates on information rates? As a blog post would have been okay-ish,…
>The fact that those number come from a citation doesn't make them true it does make them the citated paper problem, though. the guesstimates are explained as guestimates, and used as illustration for possible upper limits.
There are no measurements here, I can guess the weight of an apple based on some prior (which my brain stores as some continuous distribution, not bits), but I am not measuring it.
It's incredibly tiring that bad science is sold as good science only because it comes from some fancy university. This paper is crap and should be treated as such.
Re: The unbearable slowness of being: Why do we live at 10 bits/s?
#123I think the authors are using information theory to inappropriately flatten the complexity of the problem. On one hand we have “bits” of pre-processed sensory measurement data, then on the other hand we have “bits” of post-processed symbolic data: in many cases directly so via human language, but that would also include “the Terran unit moved a short distance” as a compact summary of a bunch of pixels updating in Sta…
Re: The unbearable slowness of being: Why do we live at 10 bits/s?
#124Earlier quoted context omitted.
I wonder if some people with dissociative identity disorder, or who at least identify as plural, experience overlapping simultaneous trains of thought
Heh if there are two yous occurring at the same time, one you would never know about it. Only third party observation would be able to tell you
Re: The unbearable slowness of being: Why do we live at 10 bits/s?
#125It seems very odd that the article seems to be measuring the information content of specific tasks that the brain is doing or specific objects that it is perceiving. But the brain is a general-purpose computer, not a speed-card computer, or English text computer, or binary digit computer, or Rubik's cube computer. When you look at a Rubik's cube, you don't just pick out specific positions of colored squares relative…
Re: The unbearable slowness of being: Why do we live at 10 bits/s?
#126Re: The unbearable slowness of being: Why do we live at 10 bits/s?
#127Earlier quoted context omitted.
It would be a lot less abrasive to say "It's linked elsewhere, but here it is: https://arxiv.org/abs/2408.10234 " or some variation, instead of saying "it's here somewhere, go find it".
with all due respect, it was meant to be slightly abrasive. it's understandable (?) not finding something when the thread has hundreds of comments, not so much when the thread had like 15-20 comments.
Re: The unbearable slowness of being: Why do we live at 10 bits/s?
#128Earlier quoted context omitted.
> English is about one bit per letter Where did you get that number from? How would you represent a letter using 1 bit?
It’s an experimental result by Shannon: https://archive.org/details/bstj30-1-50/page/n5/mode/1up In short, you show someone an English text cut off at an arbitrary point and ask them to predict which letter comes next. Based on how successful they are, you can calculate the information content of the text. The result from this experiment was approximately one bit per letter. Representing it is not the concern of the…
If you substitute "token", for "letter", what you have described is exactly what a modern LLM does, out of the box. llama.cpp even has a setting, "show logits", which emits the probability of each token (sadly, only of the text it outputs, not the text it ingests - oh well).
I don't think anyone actually uses this as a text compressor for reasons of practicality. But it's no longer a theoretical thought experiment - it's possible today, on a laptop. Certainly you can experimentally verify Shannon's result, if you believe that LLMs are a sufficiently high fidelity model of English (you should - it takes multiple sentences before it's possible to sniff that text is LLM generated, a piece of information worth a single bit).
Oh look, Fabrice Bellard (who else?) already did it: https://bellard.org/ts_zip/ and you may note that indeed, it achieves a compression ratio of just north of 1 bit per byte, using a very small language model.
Re: The unbearable slowness of being: Why do we live at 10 bits/s?
#129Earlier quoted context omitted.
As someone without an inner monologue, and someone that's spent a LOT of time meditating, it's not the language. It's the attention mechanisms themselves. Buddhist scholars insist that while we can have multiple threads of attention in our awareness, like strings with pearls of experience/thoughts we can only actually hold one little pearl of information from that stream in our attention at a time, and that we flit b…
What you're describing here is software, not hardware—Cognitive Science is the relevant field, not Neuroscience. That said, your understanding is largely supported by our current understanding of consciousness, attention, and perception. The attention mechanism doesn't handle parallel processing well—but can operate "multi-threaded", where it juggles several foci at once (with some obvious cost to switching between t…
"Cognitive science" vs "neuroscience" as a concept is just how we decided to slice the problem up for academia.
Next time, maybe cut the first paragraph ;)
Re: The unbearable slowness of being: Why do we live at 10 bits/s?
#130Earlier quoted context omitted.
As someone without an inner monologue, and someone that's spent a LOT of time meditating, it's not the language. It's the attention mechanisms themselves. Buddhist scholars insist that while we can have multiple threads of attention in our awareness, like strings with pearls of experience/thoughts we can only actually hold one little pearl of information from that stream in our attention at a time, and that we flit b…
Sometimes I'll be deeply thinking about something while driving, and discover I'm at the last road to my house without remembering having driven the previous few blocks. It's quite disturbing. When I say deeply thinking I don't mean anything involving phones or external stimuli - really just thinking about a particular problem I'm working on. I also don't deliberately engage this deep mode of thought, I just sort of…