31–40 of 157 posts
Earlier quoted context omitted.
If a lion could speak, would we understand it?
That was a philosophical position on the difficulty of understanding alien concepts and language, not a hard technological limit.
Would you care to expound?
I think “we might decode whale speech or ancient languages” is a huge stretch. Context is the most important part of what makes language useful. There is billions of human-written texts, grounded in shared experience that makes our AI good at language. We don't have that for a whale.
If a lion could speak, would we understand it?
Earlier quoted context omitted.
There is nothing really special about speech as a form of communication. All animals communicate with each other and with other animals. Informational density and, uhhhhh, cyclomatic complexity might be different between speech and a dance or a grunt or whatever.
I was referencing Wittgenstein's "If a lion could speak, we would not understand it." Wittgenstein believed (and I am strongly inclined to agree with him) that our ability to convey meaning through communication was intrinsically tied to (or, rather, sprang forth from) our physical, lived experiences. Thus, to your point, assuming communication, because "there's nothing really special about speech", does that mean we…
I mean who knows, maybe their perception of these shared experiences would be different enough to make communication difficult, but still, I think it's undeniably shared experience.
Earlier quoted context omitted.
There is nothing really special about speech as a form of communication. All animals communicate with each other and with other animals. Informational density and, uhhhhh, cyclomatic complexity might be different between speech and a dance or a grunt or whatever.
I was referencing Wittgenstein's "If a lion could speak, we would not understand it." Wittgenstein believed (and I am strongly inclined to agree with him) that our ability to convey meaning through communication was intrinsically tied to (or, rather, sprang forth from) our physical, lived experiences. Thus, to your point, assuming communication, because "there's nothing really special about speech", does that mean we…
Which isn’t saying much, it still couldn’t explain Lion Language to us, it could just generate statistically plausible examples or recognize examples.
To translate Lion speech you’d need to train a transformer on a parallel corpus of Lion to English, the existence of which would require that you already understand Lion.
Earlier quoted context omitted.
I was referencing Wittgenstein's "If a lion could speak, we would not understand it." Wittgenstein believed (and I am strongly inclined to agree with him) that our ability to convey meaning through communication was intrinsically tied to (or, rather, sprang forth from) our physical, lived experiences. Thus, to your point, assuming communication, because "there's nothing really special about speech", does that mean we…
Hmm I'm not convinced we don't have a lot of shared experience. We live on the same planet. We both hunger, eat, and drink. We see the sun, the grass, the sky. We both have muscles that stretch and compress. We both sleep and yawn. I mean who knows, maybe their perception of these shared experiences would be different enough to make communication difficult, but still, I think it's undeniably shared experience.
I think that's the core question being asked and that's the one I have a hard time seeing how it'd work.
Earlier quoted context omitted.
I was referencing Wittgenstein's "If a lion could speak, we would not understand it." Wittgenstein believed (and I am strongly inclined to agree with him) that our ability to convey meaning through communication was intrinsically tied to (or, rather, sprang forth from) our physical, lived experiences. Thus, to your point, assuming communication, because "there's nothing really special about speech", does that mean we…
If we had a sufficiently large corpus of lion-speech we could build an LLM (Lion Language Model) that would “understand” as well as any model could. Which isn’t saying much, it still couldn’t explain Lion Language to us, it could just generate statistically plausible examples or recognize examples. To translate Lion speech you’d need to train a transformer on a parallel corpus of Lion to English, the existence of whi…
Who knows, we don't really have good insight into how this information loss, or disparity grows. Is it linear? exponential? Presumably there is a threshold beyond which we simply have no ability to translate while retaining a meaningful amount of original meaning.
Would we know it when we tried to go over that threshold?
Sorry, I know I'm rambling. But it has always been regularly on my mind and it's easy for me to get on a roll. All this LLM stuff only kicked it all into overdrive.
Edit: I wrote my comment a bit too early before finishing the whole article. I'll leave my comment below, but it's actually not very closely related to the topic at hand or the author's paper. I agree with the gist of the article (which IMO is basically that universal computation is universal regardless of how you perform it), but there are two big issues that prevent this observation from helping us in a practical s…
Earlier quoted context omitted.
Hmm I'm not convinced we don't have a lot of shared experience. We live on the same planet. We both hunger, eat, and drink. We see the sun, the grass, the sky. We both have muscles that stretch and compress. We both sleep and yawn. I mean who knows, maybe their perception of these shared experiences would be different enough to make communication difficult, but still, I think it's undeniably shared experience.
That's fair. To me, the point of Wittgenstein's lion thought experiment though was not necessarily to say that _any_ communication would be impossible. But to understand what it truly meant to be a lion, not just what it meant to be an animal. But we have no shared lion experiences nor does a lion have human experiences. So would we be able to have a human to lion communication even if we could both speak human speec…
My thinking is that if something is capable of human-style speech, then we'd be able to communicate with them. We'd be able to talk about our shared experiences of the planet, and, if we're capable of human-style speech, likely also talk about more abstract concepts of what it means to be a human or lion. And potentially create new words for concepts that don't exist in each language.
I think the fact that human speech is capable of abstract concepts, not just concrete concepts, means that shared experience isn't necessary to have meaningful communication? It's a bit handwavy, depends a bit on how we're defining "understand" and "communicate".
So, what is the Dao? Personally, I see it as will — something we humans could express through words. For any given will, even though we use different words in different languages — Chinese, Japanese, English — these are simply different representations of the same will.
Large language models learn from word tokens and begin to grasp these wills — and in doing so, they become the Dao.
In that sense, I agree: “All AI models might be the same.”
An infinite dimensional model with just one dim per concept would be sorta useless, but you need things tied together?