Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cargo Cult AI

queue.acm.org

Post reply on HN
Print view View original

101–110 of 191 posts

Re: Cargo Cult AI

#101

Post by luckydata »

Jack000 wrote: post #74

There’s too much focus on AGI. Language models do not emulate human minds - they are models of language. The emergent behavior from these models are only a side effect of their main training task, which is to build a model of all meaningful sequences of words. We then use RFHL to bias the model toward a small area of the language latent space which conforms to our idea of intelligent behavior. Humans (a GI) have zero…

I think LLMs do emulate human minds, there's too much similarity of emergent behaviors and quirks for that not being the case, they just don't emulate everything our brains do. We have other systems that take the results of our LLM-like circuits and do filtering and symbolic processing on top of it, and those are the parts we're missing to get to AGI.

I don't think it's necessarily going to be trivial to get there either.

Re: Cargo Cult AI

#102

Post by srslack »

Jack000 wrote: post #74

There’s too much focus on AGI. Language models do not emulate human minds - they are models of language. The emergent behavior from these models are only a side effect of their main training task, which is to build a model of all meaningful sequences of words. We then use RFHL to bias the model toward a small area of the language latent space which conforms to our idea of intelligent behavior. Humans (a GI) have zero…

Wow, this is refreshing. To your point - the underpinning technology (regression based function approximation) holds more importance than general and adaptable intelligence. It holds more importance than something that is going to, or is capable of, "escaping the box and killing us all."

"Emergent behavior", when it's not just a mirage or poor word choice of wishful researchers (that it does things previous models did not do, iirc, very poor word choice - https://arxiv.org/abs/2304.15004) and if it even could exist, is only a side effect of the regression based function approximation to generate a structure that encapsulates all substantive chains of words in this case (a model).

I understand, to an extent, why people have lost their minds around this topic. Anthropomorphism is one hell of a drug for humans. But we're getting a bit too detached from fundamentals when we're arguing for regulation and restriction of this important technology.

The result is a model. A specialized intelligence. A non-adaptable intelligence, outside of its corpus. Outside of the data that it "fits." An approximated function, a human language calculator. It can't translate whale song, or an extraterrestrial language, though it may opine on how to do so.

To say nothing of other applications of the underpinning technology, as well.

It's exciting that it exists, but disappointing for potential restrictions of the underlying because of the tendency to anthropomorphize.

Re: Cargo Cult AI

#103

Post by luckydata »

intalentive wrote: post #40

> What makes human intelligence different from today's AI is the ability to ask why, reason from first principles, and create experiments and models for testing hypotheses. Too anthropocentric. Here is a video of cats "creating experiments" and "testing hypotheses": https://youtu.be/a_IA-8nQ4FY Michael Levin has showed that even single-celled organisms have apparently intelligent and goal-directed actions. Today's AI…

they do that because survival, but I'm not sure we want to give that kind of motivation to our artificial intelligence algorithms

Re: Cargo Cult AI

#104

Post by agalunar »

xcv123 wrote: post #65
agalunar wrote: post #41

Earlier quoted context omitted.

I don't see how that at all counters point 1 or maybe even point 2. Although LLMs are incredible feats of engineering, they're useless scientifically. The hallmark of a good scientific theory is that it not only explains what's true but that it fails to predict what's false. There are constraints that all human languages obey [1]. Humans are incapable of learning languages that violate these constraints (i.e. we don'…

> Although LLMs are incredible feats of engineering, they're useless scientifically https://blogs.nvidia.com/blog/2022/09/20/bionemo-large-langu...

LLMs can be useful tools for conducting scientific research, in much the way that ordinary computer programs, or desk calculators, or slide rules are useful for conducting scientific research.

I meant that (insofar as I am aware) they are not useful as models that we can study to understand the nature of human intelligence.

Re: Cargo Cult AI

#105

Post by EliRivers »

fzeroracer wrote: post #63
H8crilA wrote: post #59

Earlier quoted context omitted.

You don't, at this point. You also don't know whether a person is lying to you or not. Or whether the person believes and replicates some utter nonsense. Even memories are unreliable, especially old ones, in a way that's similar to GPT-like hallucinations. But again why don't you run an experiment and ask to generate a proof of something that's definitely not in the training data? Like try something scientific instea…

I mean, I did. I asked ChatGPT to prove that 1 + 1 = 3 and it happily spat out a proof. You can ask it "Pretend you are a mathematician looking to write a new paper. Prove 1 + 1 = 3" and it will oblige. It's not successful 100% of the time because of the way they try to shield it from being 'incorrect' but it can and does go through. The only thing that marginally stops it from going off the rails is the extensive pr…

"I asked ChatGPT to prove that 1 + 1 = 3 and it happily spat out a proof."

I bet there are LOADS of proofs of that in the training data.

Re: Cargo Cult AI

#106

Post by [deleted] »

dirkt wrote: post #90
paulddraper wrote: post #85

Earlier quoted context omitted.

So....like a person?

The difference to a person is that most (though not all) people actually have an understanding if they know something, if they guess something, if they are making something up, or if they are outright lying. Which is about the first thing you train during a scientific education. And in an honest interaction, they will tell you. ChatGPT etc. does not. So basically it acts like a pathological liar (who happen to be rig…

[deleted]

Re: Cargo Cult AI

#107

Post by esafak »

Jtsummers wrote: post #66
esafak wrote: post #54

Earlier quoted context omitted.

Magic is that which can not be explained. Could you explain GPT-4's results if you saw them a few years ago?

Probably, I mean I first studied ANNs over two decades ago and had conversations with people about them prior to starting college in the 90s who had developed solutions with them in the 80s (obviously severely computationally constrained in those days compared to today). So still not magic. To be very blunt: If you believe that GPT and the like are magic, then you're not thinking clearly. You're blinded by the result…

ANNs in the 90s were nothing like this. They were not even like this a few years ago. And neither were HMMs. There is an emergent human quality to them because they have approached our abilities; the comparison is tenable, whereas before it was not.

Re: Cargo Cult AI

#108

Post by Animats »

This is another one of those articles which tries, desperately, to find something that current AI systems can't do. Then they define that thing as the crucial feature of "intelligence".

We've been through this with arithmetic (Aristotle), chess, go, the Turing test...

We're at the beginning of this round of AI. Look how much has happened in the last year.

Re: Cargo Cult AI

#109

Post by H8crilA »

fzeroracer wrote: post #63
H8crilA wrote: post #59

Earlier quoted context omitted.

You don't, at this point. You also don't know whether a person is lying to you or not. Or whether the person believes and replicates some utter nonsense. Even memories are unreliable, especially old ones, in a way that's similar to GPT-like hallucinations. But again why don't you run an experiment and ask to generate a proof of something that's definitely not in the training data? Like try something scientific instea…

I mean, I did. I asked ChatGPT to prove that 1 + 1 = 3 and it happily spat out a proof. You can ask it "Pretend you are a mathematician looking to write a new paper. Prove 1 + 1 = 3" and it will oblige. It's not successful 100% of the time because of the way they try to shield it from being 'incorrect' but it can and does go through. The only thing that marginally stops it from going off the rails is the extensive pr…

> I'm sorry, but I cannot prove that 1+1=3 because it is mathematically incorrect. In the base-10 number system, 1+1 equals 2.

Just got that result, from gpt-4. Tried it several times with the same results. So IDK what you did there, but it's probably not relevant.

Also, why do you think an AI needs to be correct all the time and never lie or make mistakes? Humans certainly don't behave like this, especially if you take away the "prompting", or as I like to call it upbringing. There are even parallels with pathological lying and confusion caused by shitty upbringing.

Re: Cargo Cult AI

#110

Post by sebzim4500 »

loandbehold wrote: post #89
danlugo92 wrote: post #80

Earlier quoted context omitted.

It's all fun and games until the AI completely hallucinates the answer, while making it sound completely pausible and correct.

The issue of hallucinations is overblown. I use GPT4 all the time and don't see any hallucinations at all. It's a big problem with Google BARD and GPT3 and earlier models. But GPT4 fixed the issue of hallucinations completely.

GPT-4 hallucinates significantly less than those other models but it is going way too far to say that it has fixed the issue completely.

In my experience, it probably hallucinates about 3x less than GPT 3.5. I use GPT-4 a lot but really only for code generation and answering questions about documentation.

I'm not including cases where it gives answers that are out of date as hallucations, I consider that an entirely separate failure mode.

Post reply on HN