Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence systems found to excel at imitation, but not innovation

techxplore.com

Post reply on HN
Print view View original

101–110 of 126 posts

Re: Artificial intelligence systems found to excel at imitation, but not innovation

#101

Post by corethree »

Jensson wrote: post #93
corethree wrote: post #47

Earlier quoted context omitted.

All innovation is composition plus random generation. Which LLMs already do. LLMs have a rudimentary form of innovation. It's not quite as good as humans but it's getting there.

Innovation isn't just testing something new, it is a new thing that improves something or is valuable in some way. So doing random combinations of things isn't innovation, it is just noise.

Then where did the new idea come from? What algorithm allows for the generation of completely novel things without knowledge of the output encoded in the algorithm itself? It must be "randomly" generated by logic.

Again I like to emphasize that it's not "truly random" we can't even define randomness formally with a function so this doesn't exist. It's more of a combinatorics algorithm where we iterate through every possible and likely permutation.

Re: Artificial intelligence systems found to excel at imitation, but not innovation

#102

Post by corethree »

AlexandrB wrote: post #90
corethree wrote: post #47

Earlier quoted context omitted.

All innovation is composition plus random generation. Which LLMs already do. LLMs have a rudimentary form of innovation. It's not quite as good as humans but it's getting there.

Even if one accepts your premise, the key question is: composition of what ? I think people underestimate the volume and variety of "training data" that humans are exposed to over a lifetime. It's not just text and images - it's feelings (pain, cold, heat), emotions, sounds, smells, and other experiences that originate within the human body as well. Human innovation can arise by using these experiences as source data…

>Human innovation can arise by using these experiences as source data for composition of text or images. LLMs, by contrast, are limited to training on text and images/video exclusively.

Right but my concept still applies. What's unique about the LLM is that it is fed a massive amount of textual data in such a way that it can essentially output text AS if it were a human that does experience those emotions. From the perspective of you and me, this is no different then what we experience in real life.

How can you be sure the people around you feel emotions just as you do? Do you just assume it? Why shouldn't you assume it for an LLM? The human like the LLM uses English to describe and communicate to you. This in terms of raw logical evidence we can't confirm if LLMs feel emotions any more than our ability to confirm whether other humans can feel emotions. The technical evidence for both is relatively identical.

Re: Artificial intelligence systems found to excel at imitation, but not innovation

#103

Post by corethree »

kromem wrote: post #95
corethree wrote: post #66

Earlier quoted context omitted.

As the query prompter you are in control of the feedback loop. You can ask the AI to re-examine it's output to catch errors just as a human would do for himself. Practically speaking this does work to a limited extent. Sometimes the AI just sticks with it's guns and runs with it just like a human might.

Tip: If you are adding a self-critique step, show the model the initial output as if it is evaluating something from someone else (i.e. "grade this answer from a student") as opposed to from itself (i.e. "you wrote this, is it really correct?"). As you correctly note, humans have a problem with admitting fault. Especially the case online. But humans online are very ready to correct others. That's exactly the kind of…

Oh good point!

Re: Artificial intelligence systems found to excel at imitation, but not innovation

#104

Post by hackerlight »

corethree wrote: post #84
sidlls wrote: post #80

Earlier quoted context omitted.

All innovation is composition plus random generation. That’s a fairly bold claim. What supporting data do you have to justify it?

There are two types of statements that can be made in this world. One is data driven based on evidence. The other is logic driven based on axioms and the logical implications of said axioms. My statement is derived from the later. Therefore evidence is unnecessary. It is niave to blindly faith in all truth in the hands of data without understanding nuances between the relationship of data and logic. If you have a pur…

If we want to understand chemistry we should study chemistry at the level of chemistry, not study lower-level physics simply because chemistry is in theory explained by physics. We need to pick the right level of abstraction.

Creativity will be explainable in terms of composition and randomness in the same way that chemistry is explainable in terms of physics. Yes, these are the building blocks, but the building blocks by itself don't help you much because the complicated phenomena still hasn't been explained. All of the magic is in how the building blocks come together to produce the output.

Re: Artificial intelligence systems found to excel at imitation, but not innovation

#105

Post by hackerlight »

klabb3 wrote: post #35
Geee wrote: post #26

Innovation is simply a result of trying to imitate, but adding errors. That's how humans do it. Add in some darwinism so that the best 'innovations' survive. Made a mistake in making food? Oh, that's a new recipe. Can't really remember how to tell the story? Well, that's a new story. Accidentally kicked a ball while trying to just walk? I just invented soccer. And so on.

Partly yes. But if you throw a million items at the wall you also have tell which ones stick. An infinite random walk isn’t useful unless you have infinite verification to determine what is promising, so you can guide your next steps and continue discovering stuff.

> An infinite random walk

It's not purely random. Alpha Go doesn't search random moves, it searches a subset of possible moves that are probably good.

The Darwinian analogy of the GP is apt. We're not as cognitively flexible as we like to believe. We die, and our children are exposed to new training data which causes them to believe different things. That's part of how we adapt as a species. Sort of like how GPT keeps changing as it gets retrained on new data.

Re: Artificial intelligence systems found to excel at imitation, but not innovation

#106

Post by HKH2 »

pixl97 wrote: post #75
HKH2 wrote: post #22

Earlier quoted context omitted.

> AI will probably eliminate most lower end content creation jobs. This doesn't improve the state of the content industry for users, but saves companies quite some money. What does it say if your work can't be distinguished from AI clichés? Maybe it's what the industry needs.

Innovation probably isn't as wanted as you think. I don't want the menu at some place I'm eating at to be innovative, I want it to be legible.

I don't think innovation is all that wanted. What do you have to do to be trendy? Not use capital letters? Flatten all imagery? I'm looking forward to Comics Sans invitation letters 2.0.

Re: Artificial intelligence systems found to excel at imitation, but not innovation

#107

Post by stevenhuang »

simbolit wrote: post #91
hackinthebochs wrote: post #44

Earlier quoted context omitted.

LLMs aren't statistical systems in any substantive sense. They are deterministic programs over the input sequence. They capture statistical relationships about words, but so do human minds. That they are sensitive to statistical relationships does not discount their ability to understand.

> That they are sensitive to statistical relationships does not discount their ability to understand. The claim is that nothing but sensitivity to statistical relationships somehow leads to an ability to understand? I am not going to believe it.

How do you think our brains work?

Re: Artificial intelligence systems found to excel at imitation, but not innovation

#108

Post by simbolit »

stevenhuang wrote: post #107
simbolit wrote: post #91

Earlier quoted context omitted.

> That they are sensitive to statistical relationships does not discount their ability to understand. The claim is that nothing but sensitivity to statistical relationships somehow leads to an ability to understand? I am not going to believe it.

How do you think our brains work?

I don't know how our brains work.

If you don't just want to be snarky: I think the difference between acquisition and learning is important, I think the role of instruction might be important, I think the role of practice is important.

Re: Artificial intelligence systems found to excel at imitation, but not innovation

#109

Post by corethree »

hackerlight wrote: post #104
corethree wrote: post #84

Earlier quoted context omitted.

There are two types of statements that can be made in this world. One is data driven based on evidence. The other is logic driven based on axioms and the logical implications of said axioms. My statement is derived from the later. Therefore evidence is unnecessary. It is niave to blindly faith in all truth in the hands of data without understanding nuances between the relationship of data and logic. If you have a pur…

If we want to understand chemistry we should study chemistry at the level of chemistry, not study lower-level physics simply because chemistry is in theory explained by physics. We need to pick the right level of abstraction. Creativity will be explainable in terms of composition and randomness in the same way that chemistry is explainable in terms of physics. Yes, these are the building blocks, but the building bloc…

You're right.

But you will note that the point of my response had one objective to prove this statement:

    "All innovation is composition plus random generation."
The poster asked for "evidence" behind this statement and I explained to him why what I said is true. There's still "magic" involved with human intelligence but it still has basis in reality and I can use that basis to prove certain statements. That's all I was doing.

Re: Artificial intelligence systems found to excel at imitation, but not innovation

#110

Post by hnfong »

1vuio0pswjnm7 wrote: post #97

Why would anyone think "AI" would excel at innovation. Honest question. Innovation is defined as "introducing something new". "AI" is autocomplete and autoselection based on _past_ input. "AI" is regurgitating, rehashing or recombining something that has come before. It might do this in a new (=yet untried) way, but it cannot "introduce something new". Only man can do that. Make no mistake, recombining can be useful.…

In a deterministic physical world, all actions are based on the last state. So your actions are based on the state of the universe a split second prior. Why would you think anyone could innovate?

(Ecclesiastes 1:9 - What has been will be again, what has been done will be done again; there is nothing new under the sun.)

By now I'm quite surprised people haven't realized most of their arguments about fundamental limitations of AI apply to themselves as well.

Post reply on HN