The Myth of AI Omniscience: AI's Epistemological Limits • Hacker News Board

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#71

Post by vidarh » Sat, Aug 05, 2023, 6:03 PM UTC

mrcode007 wrote: post #40

famouswaffles wrote: post #9

LLMs don't learn to simulate or mimic, that's just a byproduct. They learn to predict the training corpus. There is absolutely nothing about the act of prediction that necessitates an upper bound of intelligence on the corpus itself. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 They found representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity in a…

You mentioned the corpus prediction being the core of the LLM. Because of this, my prediction is we will see way more data withholding to prevent LLM learning just like we’ve seen with stack overflow, Reddit, X. I myself have started doing this. For example, I don’t publish code on GitHub anymore to prevent copilot training on my own code. Normally I like to get paid for work, instead of paying for GitHub and doing w…

Withholding like that won't achieve anything other than your own obscurity.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#72

Post by epups » Sat, Aug 05, 2023, 6:04 PM UTC

polio wrote: post #67

johndough wrote: post #33

> To sum up: because LLMs are fundamentally limited to i) using our vocabulary, ii) “understanding” concepts in the ways we do, and iii) “talking” in the ways we do, then, at best, a LLM can only mirror back to us the order we have carved, the truth we have honed. That conclusion is not correct. Generative models can combine existing concepts in novel ways which have never been considered before. This is most easily…

How much of the "novel ways which have never been considered before" is just the novelty effect of having your very own artist? A human being could certainly produce any of the works of Dall-E 2, given the same prompt. The change here is the cost, and not the capability. Of course, this is still significant, but it doesn't suggest to me that Dall-E 2 "thinks" differently or would be able to seriously alter the nature…

What would be an example of a work that a human being could not produce?

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#73

Post by civilitty » Sat, Aug 05, 2023, 6:05 PM UTC

polio wrote: post #67

johndough wrote: post #33

> To sum up: because LLMs are fundamentally limited to i) using our vocabulary, ii) “understanding” concepts in the ways we do, and iii) “talking” in the ways we do, then, at best, a LLM can only mirror back to us the order we have carved, the truth we have honed. That conclusion is not correct. Generative models can combine existing concepts in novel ways which have never been considered before. This is most easily…

How much of the "novel ways which have never been considered before" is just the novelty effect of having your very own artist? A human being could certainly produce any of the works of Dall-E 2, given the same prompt. The change here is the cost, and not the capability. Of course, this is still significant, but it doesn't suggest to me that Dall-E 2 "thinks" differently or would be able to seriously alter the nature…

> How much of the "novel ways which have never been considered before" is just the novelty effect of having your very own artist?

What's the difference and why does that matter? Sure, plenty of people may have had the idea of an anthropomorphized teddy bear swimming in a pool, but if none of them ever realized it in the real world, the AI couldn't have learned from it.

> The change here is the cost, and not the capability.

I strongly disagree. Scaling the artistic process with GPUs instead of meatbags is a huge change in capability, just like the mechanized tractor was a huge capability change over the ox. The change in cost is a side effect in the change of capability.

You can now use a tool instead of outsourcing it to someone else and artists now have an automated tool that ostensibly replaces their manual labor.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#74

Post by vidarh » Sat, Aug 05, 2023, 6:06 PM UTC

lsy wrote: post #41

famouswaffles wrote: post #9

LLMs don't learn to simulate or mimic, that's just a byproduct. They learn to predict the training corpus. There is absolutely nothing about the act of prediction that necessitates an upper bound of intelligence on the corpus itself. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 They found representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity in a…

The problem is that new knowledge comes not only from combination but discovery , and discovery fundamentally requires access to and interaction with the actual world. And as of yet we have not created a way that computers can directly access the world, all I/O is mediated, etc. The more pedantic argument would be that what an LLM can write about the world is fundamentally limited to what has thus far been captured i…

People are hooking up LLM's with "access to the world" all over the place. Some of that will increasingly result in output that gets fed into training set.

I'd be shocked if there aren't people running fairly large-scale research on letting LLMs using various patterns like ReAct to augment synthetic training data as we discuss this, because that'd be near the top of my list of things to do if I had the resources to turn around and train a large-scale model on those outputs, because testing ways of accelerating the training by letting the models build augmented training datasets themselves is such an obvious step.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#75

Post by epups » Sat, Aug 05, 2023, 6:11 PM UTC

obiefernandez wrote: post #22

he lost me at "a language model’s vocabulary is limited to the words that exist within the model’s training texts, which means a LLM can only refer to objects and relations that we humans have already discerned, named, and written about." This is trivially demonstrated to be a false statement, as GPT is capable of synthesizing entirely novel words based on very little input guidance.

I find it exhausting when articles are written about ChatGPT which can be trivially demonstrated false by simply testing their assumptions. Noam Chomsky pulled a similar one a while ago.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#76

Post by WanderPanda » Sat, Aug 05, 2023, 6:11 PM UTC

johndough wrote: post #33

> To sum up: because LLMs are fundamentally limited to i) using our vocabulary, ii) “understanding” concepts in the ways we do, and iii) “talking” in the ways we do, then, at best, a LLM can only mirror back to us the order we have carved, the truth we have honed. That conclusion is not correct. Generative models can combine existing concepts in novel ways which have never been considered before. This is most easily…

It mirrors back the distribution, not the individual samples. It can interpolate but not extrapolate

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#77

Post by mannykannot » Sat, Aug 05, 2023, 6:12 PM UTC

Putting aside the question of what LLMs, specifically, are capable of, I feel the author's central argument has already been shown to be false, through events outside of AI.

In a nutshell, the argument is that, as AI will have just the knowledge that is available to us, expressed in the languages we use, we will be able to anticipate and put a stop to it doing anything catastrophic.

Exactly the same argument, however, could be made over computer and communications security. If anything, those trying to keep things secure have the advantage in knowledge, at least initially, yet zero-day exploits keep surfacing, some of them at fundamental levels such as processor architecture. Nothing truly catastrophic has happened yet, but it cannot be said to be impossible.

The flaw in the argument is that one can easily know all the bare facts without seeing all the implications but someone - or something - else could do so.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#78

Post by polio » Sat, Aug 05, 2023, 6:25 PM UTC

civilitty wrote: post #73

polio wrote: post #67

Earlier quoted context omitted.

How much of the "novel ways which have never been considered before" is just the novelty effect of having your very own artist? A human being could certainly produce any of the works of Dall-E 2, given the same prompt. The change here is the cost, and not the capability. Of course, this is still significant, but it doesn't suggest to me that Dall-E 2 "thinks" differently or would be able to seriously alter the nature…

> How much of the "novel ways which have never been considered before" is just the novelty effect of having your very own artist? What's the difference and why does that matter? Sure, plenty of people may have had the idea of an anthropomorphized teddy bear swimming in a pool, but if none of them ever realized it in the real world, the AI couldn't have learned from it. > The change here is the cost, and not the capab…

I agree that it's a big change in the economics of getting art (e.g. a single game dev can now plausibly get assets this way), but I'm not sure that it's a change in the "cognitive" process of creation. This is in response to the original comment that suggests that novel combinations are interesting.

The machine is clearly good at realizing novel combinations, but I think that has more to do with the lack of interest of human artists in rendering these combinations, rather than the lack of ability.

I am also of the opinion that a human realization would produce better art. A machine might literally depict the bear in a pool, but a human could imagine a logically consistent context for that to be happening and decorate the pool with details like the leaderboard of the Teddy Bear Olympics and have reporters and spectators that are other stuffed animals. There might be a rivalry in progress. The distinguishing feature for me so far has been that human art is a snapshot of a much more sophisticated simulation that draws from the experience of having lived, felt things like fear, tension, joy directly, rather than having to approximate the aspects that give new art its electric nature indirectly as the machines do.

I'm sure a machine will be able to do that someday, but most of my experience with Dall-E 2 has been for the background to be vague, blurry, and weirdly unintentionally surreal. The prompt itself is maybe rendered accurately 95% of the time.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#79

Post by cscurmudgeon » Sat, Aug 05, 2023, 6:30 PM UTC

vidarh wrote: post #70

cscurmudgeon wrote: post #66

Earlier quoted context omitted.

I would say coming up with BubbleSort or Shakespeare's writings from scratch is an example of novelty/creativity.

Most humans can do neither, so it's setting the bar far too high.

I think we should compare what best of LLMs can do with what the best of humans can do for a fair comparison.

Else, we will be comparing what the best LLM can do with what the media/mean human can do.

I can also point to tons of language models (both large and small) which don't do anything remotely useful.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#80

Post by civilitty » Sat, Aug 05, 2023, 6:47 PM UTC

polio wrote: post #78

civilitty wrote: post #73

Earlier quoted context omitted.

> How much of the "novel ways which have never been considered before" is just the novelty effect of having your very own artist? What's the difference and why does that matter? Sure, plenty of people may have had the idea of an anthropomorphized teddy bear swimming in a pool, but if none of them ever realized it in the real world, the AI couldn't have learned from it. > The change here is the cost, and not the capab…

I agree that it's a big change in the economics of getting art (e.g. a single game dev can now plausibly get assets this way), but I'm not sure that it's a change in the "cognitive" process of creation. This is in response to the original comment that suggests that novel combinations are interesting. The machine is clearly good at realizing novel combinations, but I think that has more to do with the lack of interest…

The OP never said that novel combinations are "interesting"... They claimed:

> Generative models can combine existing concepts in novel ways which have never been considered before.... So as long as an answer can be expressed as a combination of primitive concepts, LLMs can generate them.

You seem to be hung up on the "never been considered before" as if artists are incapable of combining those concepts when the more charitable interpretation of what the OP meant is "doesn't appear in the training data." Obviously they aren't omniscient and can't possibly know what people have ever imagined in the totality of existence, but we can empirically compare the novelty of generated images to the training inputs.

No one said anything negative about artists or their abilities. We're talking about the AI's abilities in a positive sense and that's the mechanistic ability to combine concepts in novel ways to generate images. Any art that springs from that is from the human using the AI as a tool.