Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Myth of AI Omniscience: AI's Epistemological Limits

cpwalker.substack.com

Post reply on HN
Print view View original

11–20 of 107 posts

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#11

Post by K0balt »

mtkhaos wrote: post #2

Reading through there is a heavy fault in logic. Statically reasonable limitations to llm output based on training material, ignores hallucinations. And hallucinations ignore the sheer chance of new emergent information by odds.

Hallucinations are not a different process from normal output. They are merely called hallucinations rather than insights because they are wrong.

There is no magic in LLMs , nor in human thought. It is all a matter of symbol vectorisation and inferring relationships within the multidimensional memetic matrix.

AI absolutely can achieve original insight- but not one that humans could not also make if looking at the same data. This is because the “intelligence” in generative AI is human bounded in the training data, not in the engine that processes it. I strongly suspect the same is true of humans.

After all, we modelled neural networks after our own brains, why would we not expect them to achieve similar results using similar means? Wasn’t that the whole point?

It never ceases to amaze me how people go all pikachu face when I say that our thought process is probably a lot like generative models. Step one, make a facsimile of thing. Step 2: be surprised when thing can also be viewed as being similar to the facsimile. Lol. Hubris is our primary characteristic.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#12

Post by Davidzheng »

dvt wrote: post #10
famouswaffles wrote: post #9

LLMs don't learn to simulate or mimic, that's just a byproduct. They learn to predict the training corpus. There is absolutely nothing about the act of prediction that necessitates an upper bound of intelligence on the corpus itself. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 They found representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity in a…

> This is not true and is easy enough to test. How exactly is this not true? Embeddings are literally a mapping of (English) words to numbers.

But because the tokens are not word level they are capable of making up words that humans have never used before.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#13

Post by zarzavat »

tudorw wrote: post #6

I'm interested in seeing how brains change as AI usage increases, in the same way many of us don't attempt to memorize phone numbers, what skills will we offload to 'AI', will we be aware that it's happening? Perhaps more interesting, what new adaptations or adjustments may occur as a result of this augmentation. I hope that our brains ability to synthesize new thoughts both extrapolated and 'from thin air' will rema…

The problem with AI is that it doesn’t really free you to do anything else, because as soon as it does, that thing becomes the next open research problem.

For example, you could say that ChatGPT frees writers to focus on creativity. But someone somewhere is working on making LLMs more creative and less formulaic.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#14

Post by Davidzheng »

K0balt wrote: post #7

Rather than viewing generative AI as a form of artificial intelligence, I posit that it should be seen as an automated tool for tapping into human cultural, linguistic, and empirical knowledge. Data and computation are two sides of the same coin. The 'intelligence' in AI is embedded within the data, with the computational model serving as a tool to access and express this inherent intelligence. I would argue for a ch…

How can you be confident that by scaling up the models their reasoning skills will not surpass humans

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#15

Post by K0balt »

dvt wrote: post #10
famouswaffles wrote: post #9

LLMs don't learn to simulate or mimic, that's just a byproduct. They learn to predict the training corpus. There is absolutely nothing about the act of prediction that necessitates an upper bound of intelligence on the corpus itself. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 They found representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity in a…

> This is not true and is easy enough to test. How exactly is this not true? Embeddings are literally a mapping of (English) words to numbers.

It is possible to draw new conclusions from complex data. It’s not something that a human might not also see, just that it’s an undiscovered relationship. (Or at least not directly expressed except by inference in the training corpus)

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#16

Post by simonh »

The article is claiming that advanced AIs cannot become more intelligent than humans, essentially because LLMs cannot become more intelligent than humans.

LLMs aren’t the be all and end all of AI though. They’re an impressive but inherently limited stepping stone, with a very constrained scope of applicability and capabilities. There is no reason to suppose that future, much more advanced architectures can’t surpass humans in reasoning ability.

Advanced AI doesn’t have to be omniscient fir us not yon be able to predict it’s capabilities, it just needs to be smarter than us. By definition if something is smarter than you, it’s not possible for you to anticipate and predict its behaviour. Therefore in principle it is not possible to predict whether that behaviour will be to your liking. That’s the problem.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#17

Post by mtkhaos »

K0balt wrote: post #11
mtkhaos wrote: post #2

Reading through there is a heavy fault in logic. Statically reasonable limitations to llm output based on training material, ignores hallucinations. And hallucinations ignore the sheer chance of new emergent information by odds.

Hallucinations are not a different process from normal output. They are merely called hallucinations rather than insights because they are wrong. There is no magic in LLMs , nor in human thought. It is all a matter of symbol vectorisation and inferring relationships within the multidimensional memetic matrix. AI absolutely can achieve original insight- but not one that humans could not also make if looking at the sam…

By odds and in the context of RLHF the human could very well thumbs up the output without recognition of said hallucination.

As hallucination is a general term giving to said phenomenon. Otherwise the question Emerges why 2023? And why was that not known colloquially before hand. When the basis of these Algorithms come the the 80s?

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#18

Post by cle »

dvt wrote: post #10
famouswaffles wrote: post #9

LLMs don't learn to simulate or mimic, that's just a byproduct. They learn to predict the training corpus. There is absolutely nothing about the act of prediction that necessitates an upper bound of intelligence on the corpus itself. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 They found representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity in a…

> This is not true and is easy enough to test. How exactly is this not true? Embeddings are literally a mapping of (English) words to numbers.

Test it!

> Can you make up a word?

> Certainly! Here's a made-up word for you:

> *Glimberfleck*: The sparkling reflection of sunlight that dances on water, making it seem like the surface is sprinkled with tiny, shimmering gems.

> Feel free to use it or let me know if you'd like me to create another word!

> When I’m looking out of the window of an airplane at the ocean, am I looking at a glimberfleck?

> Yes, if you're looking out of the airplane window at the ocean and observing the sparkling reflections of sunlight on the water's surface, you could describe that beautiful phenomenon as a "glimberfleck." It's a whimsical term for those captivating, shimmering patterns you see.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#19

Post by dvt »

Davidzheng wrote: post #12
dvt wrote: post #10

Earlier quoted context omitted.

> This is not true and is easy enough to test. How exactly is this not true? Embeddings are literally a mapping of (English) words to numbers.

But because the tokens are not word level they are capable of making up words that humans have never used before.

Most tokens are word or sub-word level. A token can be "find" or two tokens can be "find" and "ing." To argue that this isn't inherently using the underlying grammar rules is plainly wrong. To argue that an LLM trained on English text is capable of "making up words" (in any meaningful sense of that sentence) is even more wrong.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#20

Post by K0balt »

mtkhaos wrote: post #17
K0balt wrote: post #11

Earlier quoted context omitted.

Hallucinations are not a different process from normal output. They are merely called hallucinations rather than insights because they are wrong. There is no magic in LLMs , nor in human thought. It is all a matter of symbol vectorisation and inferring relationships within the multidimensional memetic matrix. AI absolutely can achieve original insight- but not one that humans could not also make if looking at the sam…

By odds and in the context of RLHF the human could very well thumbs up the output without recognition of said hallucination. As hallucination is a general term giving to said phenomenon. Otherwise the question Emerges why 2023? And why was that not known colloquially before hand. When the basis of these Algorithms come the the 80s?

Because models got sophisticated enough that wrong outputs started looking plausible answers rather than random garbage. Sometimes they still spew random garbage though.
Post reply on HN