The misunderstanding that leads to belief in strong AI is that meaning is somehow embedded in the symbols used to communicate it. Meaning is a natural process that occurs inside each of us, speech is just a symbol of that meaning, text is a symbol of that speech. Further, meaning is an ever-evolving, ever-mutating process much like the universe. Training on symbols cannot arrive at meaning, since the meaning isn’t co…
It ties in with deBord's "Society of the Spectacle" [0], a theory that societies evolve from Being, to Having, and ultimately devolve into merely the Appearance of Having.
Your point about the symbols being tools for communication (as opposed to the ineffable ideas being communicated) is also echoed in Lockhart's Lament [1].
[0] https://en.wikipedia.org/wiki/The_Society_of_the_Spectacle
[1] https://www.maa.org/external_archive/devlin/LockhartsLament....
Edited to add that this was extra thought-provoking:
> "It may even lie in between us, rather than in us."