The Myth of AI Omniscience: AI's Epistemological Limits

The Myth of AI Omniscience: AI's Epistemological Limits
cpwalker.substack.com

C'mon, we already have the answer it's 42

I would argue for a change in perspective towards AI, one that recognizes LLMs as powerful tools for accessing the vast wealth of human cultural knowledge rather than viewing them as a separate form of intelligence. We must carefully consider critical ethical considerations about control, access, and trust that will become increasingly relevant as these tools become more integrated into our everyday lives.

This paradigm shift carries some implications:

LLMs will not achieve superintelligence: Although these models can process information quickly and access a wide range of knowledge, they lack the superior reasoning or inference abilities that would classify them as superintelligence.

LLMs as an extension of human thought: These models can automate and amplify human capabilities but do not introduce new abilities beyond what is already present in human thought processes.

LLMs as mirrors of human culture and knowledge: These models reflect the recorded artifacts of human language, art, and culture. They can make the inherent intelligence within these artifacts accessible, providing a vast information resource.

Implications for the future: Access to this "memetic matrix" of human knowledge will become a fundamental part of being human as these tools become more integrated into our lives, bringing up issues of ownership, access, and the potential for misuse.

Thought consolidation and control of inference engines: There's a potential risk that control of inference engines by a small number of companies could lead to a consolidation of thought that threatens democratic governance. I propose a diversity of federated or self-hosted inference tools as solutions to mitigate this risk.

The necessity for trust and individuality: As these tools become more influential in our lives, maintaining trust in our individual thoughts and avoiding the uncritical acceptance of synthesized ideas from sources with opaque motives will become increasingly important.

Synthetic Inference relies on a vast cultural commons: We cannot allow these commons to be closed off and owned by a few big companies. This resource is to approximate totality of all human knowledge, language, and culture. It belongs to all of humanity. Training data must be open, free, and available for examination.

https://www.pnas.org/doi/full/10.1073/pnas.2016239118

They found representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity in an LLM only trained to predict protein structure sequences. No folding, nothing explicit in the corpus itself. Yet those truths manifest as a necessity of accurate predictions. Prediction is not bound by what the data explicitly shows.

>A language model’s vocabulary is limited to the words that exist within the model’s training texts, which means a LLM can only refer to objects and relations that we humans have already discerned, named, and written about.

This is not true and is easy enough to test.