The Myth of AI Omniscience: AI's Epistemological Limits • Hacker News Board

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#21

Post by zarzavat » Sat, Aug 05, 2023, 4:02 PM UTC

dvt wrote: post #10

famouswaffles wrote: post #9

LLMs don't learn to simulate or mimic, that's just a byproduct. They learn to predict the training corpus. There is absolutely nothing about the act of prediction that necessitates an upper bound of intelligence on the corpus itself. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 They found representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity in a…

> This is not true and is easy enough to test. How exactly is this not true? Embeddings are literally a mapping of (English) words to numbers.

It’s not a mapping of words. It’s a mapping of character sequences. You can ask chatgpt to define “hackernewsitis (zero hits on google) and it gives a plausible definition.

Also the fact that it can write code is evidence that it can understand new concepts. A variable declaration is a coining of a (very short lived) new word.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#22

Post by obiefernandez » Sat, Aug 05, 2023, 4:03 PM UTC

he lost me at "a language model’s vocabulary is limited to the words that exist within the model’s training texts, which means a LLM can only refer to objects and relations that we humans have already discerned, named, and written about."

This is trivially demonstrated to be a false statement, as GPT is capable of synthesizing entirely novel words based on very little input guidance.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#23

Post by _ea1k » Sat, Aug 05, 2023, 4:03 PM UTC

dvt wrote: post #19

Davidzheng wrote: post #12

Earlier quoted context omitted.

But because the tokens are not word level they are capable of making up words that humans have never used before.

Most tokens are word or sub-word level. A token can be "find" or two tokens can be "find" and "ing." To argue that this isn't inherently using the underlying grammar rules is plainly wrong. To argue that an LLM trained on English text is capable of "making up words" (in any meaningful sense of that sentence) is even more wrong.

You might think it is wrong, but I think it is a perfectly cromulent case of AI "making up words".

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#24

Post by mtkhaos » Sat, Aug 05, 2023, 4:05 PM UTC

K0balt wrote: post #20

mtkhaos wrote: post #17

Earlier quoted context omitted.

By odds and in the context of RLHF the human could very well thumbs up the output without recognition of said hallucination. As hallucination is a general term giving to said phenomenon. Otherwise the question Emerges why 2023? And why was that not known colloquially before hand. When the basis of these Algorithms come the the 80s?

Because models got sophisticated enough that wrong outputs started looking plausible answers rather than random garbage. Sometimes they still spew random garbage though.

But isn't a binary absolute in this case and cannot be used in the basis of first principle's assumption of truth.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#25

Post by K0balt » Sat, Aug 05, 2023, 4:05 PM UTC

ozy wrote: post #4

Why would AGI be bound to human made models? Why can it no develop its own?

Recursive insight is possible with a model that self trains, but right now that would result in a detour into unreality. Perhaps with the right systems of vetting prior to incorporating new data into the retraining set.

Right now they just get stupider if you train them on their own output, which suggests that the quality of the data available in the training set is higher than the quality of output produced by the model as a general rule. The fidelity is 1 (the growth of human knowledge) but our algorithms are not so great at this point, it seems.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#26

Post by dvt » Sat, Aug 05, 2023, 4:08 PM UTC

zarzavat wrote: post #21

dvt wrote: post #10

Earlier quoted context omitted.

> This is not true and is easy enough to test. How exactly is this not true? Embeddings are literally a mapping of (English) words to numbers.

It’s not a mapping of words. It’s a mapping of character sequences. You can ask chatgpt to define “hackernewsitis (zero hits on google) and it gives a plausible definition. Also the fact that it can write code is evidence that it can understand new concepts. A variable declaration is a coining of a (very short lived) new word.

> It’s not a mapping of words. It’s a mapping of character sequences.

This is plainly wrong if you read the word2vec paper.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#27

Post by jstanley » Sat, Aug 05, 2023, 4:09 PM UTC

dvt wrote: post #19

Davidzheng wrote: post #12

Earlier quoted context omitted.

But because the tokens are not word level they are capable of making up words that humans have never used before.

Most tokens are word or sub-word level. A token can be "find" or two tokens can be "find" and "ing." To argue that this isn't inherently using the underlying grammar rules is plainly wrong. To argue that an LLM trained on English text is capable of "making up words" (in any meaningful sense of that sentence) is even more wrong.

Does this satisfy you?

  Me: give me a random character sequence that is not a pronounceable word

  ChatGPT: Sure, here's a random character sequence that is not a pronounceable word:
  "Jkplqxzv"

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#28

Post by jstanley » Sat, Aug 05, 2023, 4:12 PM UTC

dvt wrote: post #26

zarzavat wrote: post #21

Earlier quoted context omitted.

It’s not a mapping of words. It’s a mapping of character sequences. You can ask chatgpt to define “hackernewsitis (zero hits on google) and it gives a plausible definition. Also the fact that it can write code is evidence that it can understand new concepts. A variable declaration is a coining of a (very short lived) new word.

> It’s not a mapping of words. It’s a mapping of character sequences. This is plainly wrong if you read the word2vec paper.

It depends which tokenisation you use. Here's the token list for GPT-4: https://incoherency.co.uk/interest/gpt4-token-list.txt

Some of them are words, some of them are not. It uses the same list even if you have read the word2vec paper. It probably doesn't even know whether you've read it.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#29

Post by Xcelerate » Sat, Aug 05, 2023, 4:13 PM UTC

I found this article a bit fluffy and lacking on technical substance. If we want to entertain the limits of AGI, we need to look at concepts like Solomonoff induction, which were conceived of and researched in the 1960s (and apparently mostly forgotten about since then). Given an input sequence of bits, what is the best (i.e., “oracle”) posterior distribution possible given a domain prior and the universal prior? How would a resource-limited approximation widen this posterior distribution? What are the practical limits to achieving the shortest Turing machines that reproduce the input sequence? These are interesting questions that should be looked into if we want to discuss AGI in the epistemological limit.

Also, I tend to get frustrated with predictions by “AI experts” about what a future with AGI will look like. We don’t have any world experts on AGI currently, for the simple reason that it doesn’t exist yet. We have experts (or I’d prefer to say “first movers”) on particular types of machine learning based upon neural networks, but that’s about it. A true AGI would almost be unpredictable by definition, so human attempts to forecast the future in this regard are mostly sensationalism or a projection of some kind of personal agenda (cough cough Marc Andreessen).

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#30

Post by rst » Sat, Aug 05, 2023, 4:13 PM UTC

simonh wrote: post #16

The article is claiming that advanced AIs cannot become more intelligent than humans, essentially because LLMs cannot become more intelligent than humans. LLMs aren’t the be all and end all of AI though. They’re an impressive but inherently limited stepping stone, with a very constrained scope of applicability and capabilities. There is no reason to suppose that future, much more advanced architectures can’t surpass…

More specifically, the article assumes that the only way "AI"s will ever be produced is the way that LLMs are -- by being trained off pre-existing human generated media, and existing as static objects after that. In particular, this assumes that the system doesn't subsequently modify itself based on further interactions with the real world (e.g., doing experiments, and taking note of the results).

That's assuming an awful lot, to put it mildly.