Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cargo Cult AI

queue.acm.org

Post reply on HN
Print view View original

21–30 of 191 posts

Re: Cargo Cult AI

#21

Post by progrus »

drewcoo wrote: post #14

The piece ironically misapplies the term "cargo cult" to mean something irrational, presumably because Levine has seen this work to spurn something in the past. Cargo cult behavior means repeating the ostensible look/feel of something that worked before. Intelligence has been constantly redefined over the years as other animals and then as mechanical means have met the previous standards for intelligence. Intelligenc…

Scientific thinking, or a generalized and incomplete algebra of reason, is exactly what you get from a GPT. Nothing more.

Basically, I would argue that this article sees a real issue, but seems to get it backwards. All formal logical systems are incomplete and/or inconsistent. Scientific thinking is also shaped by reality - which army turned out to have the better ballistics, and so forth.

Of course the next thing to chew on is: Are you just an “algebra of reason” too? I don’t buy that, but it’s a common belief.

Re: Cargo Cult AI

#22

Post by burnished »

hospitalhusband wrote: post #12
low_tech_love wrote: post #5

Earlier quoted context omitted.

Although I more or less agree with you, in general, how complex would the AI have to be before you can consider it aware?

I categorically reject the notion that LMMs (Large Markov Models) can ever be aware or intelligent. Comparing weighted next-word-engines to feeling, thinking, aware beings is insulting.

Are you intentionally echoing that scene from starship troopers where the pundit makes the same claim about 'smart bugs'?

Re: Cargo Cult AI

#23

Post by sharemywin »

sandworm101 wrote: post #3

Every time I read about AI I am reminded of the mouse running a maze. Any AI algorithm can learn to complete a maze in record time. It can memorize every corner. It can run a search pattern perfectly and improve that pattern iteratively, to the point that it may create new search patterns, applying what appear to be novel ideas. But the mouse actually understands the concept of a maze. It knows that the cheese exists…

I think with reinforcement learning and adversarial play you don't need it to understand for it to have unintended consequences and "break out of it's cage"

Surprising behaviors We’ve shown that agents can learn sophisticated tool use in a high fidelity physics simulator; however, there were many lessons learned along the way to this result. Building environments is not easy and it is quite often the case that agents find a way to exploit the environment you build or the physics engine in an unintended way.

Emergent tool use from multi-agent interaction https://openai.com/research/emergent-tool-use

Here's an example where using a LLM enhanced a reinforcement algo performance.

https://arxiv.org/abs/2302.06692

Re: Cargo Cult AI

#24

Post by majormajor »

cs702 wrote: post #18

Echoing many who now find themselves blindsided by the emergent abilities and rapid adoption of LLMs, the OP: * complains that we still lack a "comprehensive theory to explain what intelligence is or how it emerges from first principles," * argues that deep neural nets like LLMs may not be capable of artificial general intelligence (AGI), and * contends that achieving AGI will require "new algorithmic paradigms." Ric…

I think there's likely to be a distinction here between "things that are learnable through computation" and things that aren't.

A machine or algorithm that needed to evaluate various methods of planting seeds in soil, for instance, is going to be limited by hard time factors short of figuring out how to put in the "physics" of it from the current state of the art of human knowledge.

And that pushes you up against the line of "tools that make it easy to interface with today and history's knowledge, art styles, etc" vs "generating new knowledge." The singularity would require the latter - there's a lot of talk around embodiment as a potential necessity there, but I think there's a certain difference too around experimentation and feedback. A perfect simulation of the universe would let you get around some of this - especially if you assume perfect or good-enough simulation of human behavior - but that's a LOT of compute. This gap between "what a human can do, but faster/cheaper/without getting tired" and "what a human couldn't even imagine" that is the "AI" dream (sometimes nightmare) that sci-fi planted in our heads.

Re: Cargo Cult AI

#25

Post by JoeOfTexas »

sandworm101 wrote: post #3

Every time I read about AI I am reminded of the mouse running a maze. Any AI algorithm can learn to complete a maze in record time. It can memorize every corner. It can run a search pattern perfectly and improve that pattern iteratively, to the point that it may create new search patterns, applying what appear to be novel ideas. But the mouse actually understands the concept of a maze. It knows that the cheese exists…

The mouse is driven by survival. The AI is driven by best answer.

Re: Cargo Cult AI

#26

Post by bmc7505 »

gatkinso wrote: post #13

I think a lot of AI stuff is really cool and promising, but am dismayed by how impressed people are by it sometimes, especially the visual output of systems like DALL-E. Seems like machines are testing our intelligence, rather than the other way around.

I am disappointed with increasing frequency to learn how certain individuals whom I previously believed to be serious scholars fall for the cargo cult of AI safety. Personally I find it difficult to believe they are willfully participating in such conartistry, and makes me question their views on other subjects. Although I suppose intelligence and gullibility can coexist and some forms of delusion are better pitied than scorned.

Re: Cargo Cult AI

#27

Post by sebzim4500 »

hospitalhusband wrote: post #12
low_tech_love wrote: post #5

Earlier quoted context omitted.

Although I more or less agree with you, in general, how complex would the AI have to be before you can consider it aware?

I categorically reject the notion that LMMs (Large Markov Models) can ever be aware or intelligent. Comparing weighted next-word-engines to feeling, thinking, aware beings is insulting.

What's an LMM? I've never seen the term.

> Comparing weighted next-word-engines to feeling, thinking, aware beings is insulting

Why is it reasonable to be so reductionist about e.g. GPT-4 but not be so reductionist about a biological brain? E.g., why can't I say that your brain is nothing but a bunch of biological neurons trained using its input and intialized based on your genetics? It's equally true, and equally missing the point.

Re: Cargo Cult AI

#28

Post by thegrim33 »

cs702 wrote: post #18

Echoing many who now find themselves blindsided by the emergent abilities and rapid adoption of LLMs, the OP: * complains that we still lack a "comprehensive theory to explain what intelligence is or how it emerges from first principles," * argues that deep neural nets like LLMs may not be capable of artificial general intelligence (AGI), and * contends that achieving AGI will require "new algorithmic paradigms." Ric…

I mean, your entire counter-argument is linking a single person's opinion piece. He says that in general more computation is "good" and that search/learning "seem" to scale with computation. That's about it. It doesn't refute the key ideas at all.

He also gives the stereotypical horribly flawed trope about how "some people in the past didn't think computers could beat them in chess, and they were wrong, then some people thought computers couldn't beat them in go, and they were wrong, so now what they say about machine learning today must be wrong too".

Which is a completely illogical line of reasoning. By that reasoning, I present this same argument: When cars were first invented some people said that they'd never be able to reach 50mph, and they were proven wrong, then some people said they'd never be able to reach 150mph, and they were proven wrong, and therefore anyone that doubts my claim that we'll have 1,500mph cars on our streets next year is obviously wrong, because look, some people in the past made bad predictions.

Re: Cargo Cult AI

#29

Post by sebzim4500 »

bmc7505 wrote: post #26
gatkinso wrote: post #13

I think a lot of AI stuff is really cool and promising, but am dismayed by how impressed people are by it sometimes, especially the visual output of systems like DALL-E. Seems like machines are testing our intelligence, rather than the other way around.

I am disappointed with increasing frequency to learn how certain individuals whom I previously believed to be serious scholars fall for the cargo cult of AI safety. Personally I find it difficult to believe they are willfully participating in such conartistry, and makes me question their views on other subjects. Although I suppose intelligence and gullibility can coexist and some forms of delusion are better pitied t…

Why is your prior belief that AI safety is a cult so strong that even multiple people you would rather assume that multiple people who you previously respected are now delusional than to consider that there might be arguments worth considering?

Re: Cargo Cult AI

#30

Post by mrbungie »

gatkinso wrote: post #13

I think a lot of AI stuff is really cool and promising, but am dismayed by how impressed people are by it sometimes, especially the visual output of systems like DALL-E. Seems like machines are testing our intelligence, rather than the other way around.

Have you seen how the outputs of image generation tools evolved during mere 2-3 years? It went from the stuff of nightmares to actually pasable and consumable images pretty fast.
Post reply on HN