I was thinking last night about LLMs with respect to Wittgenstein after watching this interesting discussion of his philosophy by John Searle [1]. I think Wittgenstein's ideas are pertinent to the discussion of the relation of language to intelligence (or reasoning in general). I don't meant this in a technical sense (I recall Chomsky mentioning that almost no ideas from Wittgenstein actually have a place in modern l…
> Wittgenstein view of metaphysics lmao. I'd like to see you elaborate on what you think this means! The fact that you quote Searle, though, tells me all I need to know.
In his early work he appeared to believe that the way to do this categorization was to determine if the logical propositions corresponded in their structure to the real world (his picture theory of language). In his later work he appears to have believed the way to do this was to reference some "language game" among a set of communicating agents and then understand the words based on their context within that construct (use theory of language).
Both of these concepts do not reference any kind of subjective experience, aesthetics, morality, etc. They are simply a description of how to judge whether or not a proposition is "meaningful". In later Wittgenstein, meaning in language is entirely divorced from any necessity that the statements correspond to an independent objective reality. It is simply necessary that they are consistent amongst the participants in a particular language game.
I don't believe that means the entirety of our "consciousness" is related solely to our ability to categorize logical propositions. However, it may suggest that intelligence specifically is related to this ability.
In as much as we can say that an LLM is capable of participating in some particular human language game and can successfully categorize logical propositions within that language game - I would say that LLM is demonstrating "intelligence" within that language game. And if we can create LLMs that can participate in arbitrary (or general) language games across a wide variety of domains, we might call that LLM generally intelligent. I believe that current LLMs have achieved the first (demonstrating some intelligence in particular domains) but we have yet to achieve the second (demonstrating consistent intelligence in a wide range of general domains).
As for metaphysics, I would argue that Wittgenstein saw this general ability (to categorize logical propositions) as a subset of all possible experience. I believe he saw this categorization activity as the primary aim of philosophy. However, the kinds of experience that were outside of this categorization activity could not be spoken about at all.