Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun on GPT-3

facebook.com

Post reply on HN
Print view View original

231–240 of 253 posts

Re: Yann LeCun on GPT-3

#231

Post by maxhodges »

Those of us with professional knowledge of AI understand this already. Who are you arguing against? Still, the scale of GPT-3's model makes it novel, useful, and interesting.

>GPT-3 doesn't have any knowledge of how the world actually works.

I agree, it lacks what we'd consider robust, semantic models of common sense knowledge. However, my dog doesn't understand how the world actually works either, yet he can do many things most people would consider intelligent. Besides, most people are able to operate mobile phones, play computer games, operator automobiles, and turn on the lights without knowing how these things "actually" work. So is that even a prerequisite for intelligence?

>Some people

Who specifically? I have to admit, the views you are criticizing are preposterous but then what I want to know is why you're wasting your time and ours criticizing such junk? Some people think the earth if flat, but it's not worth arguing with them.

Re: Yann LeCun on GPT-3

#232

Post by disgruntledphd2 »

GrantZvolsky wrote: post #218
disgruntledphd2 wrote: post #192

Earlier quoted context omitted.

This is just not true. For example, all humans have the notion of object permanence, developed by about six months of life. Object permanence is the notion that things don't go away just because you can't see them. ML systems need to be specifically trained to have object permanence, and GPT-3 almost certainly does not possess it. Like, I get that it's hip to booster ML and GPT-3, and all of the stuff humans can do s…

Object permanence isn't even defined in the stateless world of the GPT-3 API. My comment was merely a jab at Yann's poor argument. I don't find humans to be trivial at all, but neither do I believe that they are infinitely complex. The linked Nabla article is fair, albeit I would appreciate more technical details. It seems to be using the API in zero-shot fashion, which is not what one would do to get the most out of…

Yeah, agreed. My concern is that object permanence isn't even defined in self driving models;)

Re: Yann LeCun on GPT-3

#233

Post by oldmonk1990 »

fastball wrote: post #46
dbcurtis wrote: post #19

Earlier quoted context omitted.

I have been using that analogy to explain why Tesla is many years away from a self driving car. Several others are building something that is fundamentally different.

Aren't they rolling out a beta of FSD literally as we speak?

They will continue to move their goalposts. It's never enough for the self-professed "cynics" of AI.

Re: Yann LeCun on GPT-3

#234

Post by oldmonk1990 »

tomasyany wrote: post #8
czzr wrote: post #6

For anyone else who doesn’t want to deal with Facebook, here’s the post: Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do. This simple explanatory study by my friends at Nabla debunks some of those expectations for people who think massive language models can be used in healthcare. GPT-3 is a language model, which means that you feed it a text and ask it…

You can find the study Yann was commenting here https://www.nabla.com/blog/gpt-3/

It's not a "study". It's some stupid clickbait stuff to get headlines and attention.

Re: Yann LeCun on GPT-3

#235

Post by GrantZvolsky »

disgruntledphd2 wrote: post #232
GrantZvolsky wrote: post #218

Earlier quoted context omitted.

Object permanence isn't even defined in the stateless world of the GPT-3 API. My comment was merely a jab at Yann's poor argument. I don't find humans to be trivial at all, but neither do I believe that they are infinitely complex. The linked Nabla article is fair, albeit I would appreciate more technical details. It seems to be using the API in zero-shot fashion, which is not what one would do to get the most out of…

Yeah, agreed. My concern is that object permanence isn't even defined in self driving models;)

I understand that you are being sarcastic, but I'll pretend that you're not ;)

There was a video showing the pattern recognition layer of a car 'forget' about objects it had previously correctly identified. There clearly is room for improvement, and some level of concern is warranted, but it isn't as much of a big deal as it seems, because surely at the higher level objects are remembered. To draw an analogy with human cognition, humans also aren't consciously aware of all the objects in their field of view all the time, but that doesn't mean they forget their existence.

Re: Yann LeCun on GPT-3

#236

Post by powerapple »

mannykannot wrote: post #190
powerapple wrote: post #184

Earlier quoted context omitted.

Are animals as intelligent as human? What language does to us is to be able to pass on knowledge to generations, knowledge can be accumulated. Animals may be able to pass on simple concept to next generation, they won't be able to accumulate knowledge without language and writing.

As a response to leftyted, erispoe's point neither depends on nor implies that animals are as intelligent as humans, it is simply bringing up a counter-example: what appears to be knowledge in animals lacking language. Of course, you could always attempt to define knowledge such that it is purely verbal, or alternatively define whatever is going on in the brain of an animal to be language, but is either approach usef…

The knowledge of riding a unicycle on a tightrope does not give you the ability to ride a unicycle on a tightrope. Yes, you have to learn it through experiencing it, because you need to map it to your motors. Animal has instincts, they are able to trace water, for example, without teaching. It is also knowledge, but for our discussion, knowledge is what we obtained after birth, not something encoded in our DNA. This knowledge is stored in the format of language. Apparently some believes that that language is part of our DNA. That's why you can have Tazan, but not an animal can speak human language even if a human raise it since it was born because language cannot be learnt without support in code in DNA.

Re: Yann LeCun on GPT-3

#237

Post by disgruntledphd2 »

GrantZvolsky wrote: post #235
disgruntledphd2 wrote: post #232

Earlier quoted context omitted.

Yeah, agreed. My concern is that object permanence isn't even defined in self driving models;)

I understand that you are being sarcastic, but I'll pretend that you're not ;) There was a video showing the pattern recognition layer of a car 'forget' about objects it had previously correctly identified. There clearly is room for improvement, and some level of concern is warranted, but it isn't as much of a big deal as it seems, because surely at the higher level objects are remembered. To draw an analogy with hum…

I'm honestly not. I've seen that video, and given my experience of vision models, am actually concerned that object permanance is not being focused on.

> There clearly is room for improvement, and some level of concern is warranted, but it isn't as much of a big deal as it seems, because surely at the higher level objects are remembered.

I would be very surprised if this were true.

> To draw an analogy with human cognition, humans also aren't consciously aware of all the objects in their field of view all the time, but that doesn't mean they forget their existence.

This is exactly object permanence, in that babies start looking in the right direction for objects that disappear. I assume that Waymo at least have some kind of model for this, but am not convinced that it's particularly effective.

Personally, I think that self-driving will require a limited theory of mind model, and we are nowhere near that.

Re: Yann LeCun on GPT-3

#238

Post by GrantZvolsky »

disgruntledphd2 wrote: post #237
GrantZvolsky wrote: post #235

Earlier quoted context omitted.

I understand that you are being sarcastic, but I'll pretend that you're not ;) There was a video showing the pattern recognition layer of a car 'forget' about objects it had previously correctly identified. There clearly is room for improvement, and some level of concern is warranted, but it isn't as much of a big deal as it seems, because surely at the higher level objects are remembered. To draw an analogy with hum…

I'm honestly not. I've seen that video, and given my experience of vision models, am actually concerned that object permanance is not being focused on. > There clearly is room for improvement, and some level of concern is warranted, but it isn't as much of a big deal as it seems, because surely at the higher level objects are remembered. I would be very surprised if this were true. > To draw an analogy with human cog…

> I would be very surprised if this were true.

At the very least there must be some kind of a trajectory model which implies that multiple observations of the same object are analyzed as a sequence. How would it work without object permanence? Would it just erase objects undetected in the current frame and pretend they never existed? I suspect you're too focused on the vision aspect of the vehicle control system.

Re: Yann LeCun on GPT-3

#239

Post by mannykannot »

powerapple wrote: post #236
mannykannot wrote: post #190

Earlier quoted context omitted.

As a response to leftyted, erispoe's point neither depends on nor implies that animals are as intelligent as humans, it is simply bringing up a counter-example: what appears to be knowledge in animals lacking language. Of course, you could always attempt to define knowledge such that it is purely verbal, or alternatively define whatever is going on in the brain of an animal to be language, but is either approach usef…

The knowledge of riding a unicycle on a tightrope does not give you the ability to ride a unicycle on a tightrope. Yes, you have to learn it through experiencing it, because you need to map it to your motors. Animal has instincts, they are able to trace water, for example, without teaching. It is also knowledge, but for our discussion, knowledge is what we obtained after birth, not something encoded in our DNA. This…

If we define knowledge that way, leftyted's original claim (knowledge is language) becomes true by definition.

This can lead to some confusion. In Frank Jackson's Knowledge Argument[1], dualists say that qualia are knowledge, knowledge is language, and so if Mary could not learn qualia by studying science, then materialism is false. This argument trades on being inconsistent over what it means to have knowledge of something.

[1] https://en.wikipedia.org/wiki/Knowledge_argument

Re: Yann LeCun on GPT-3

#240

Post by emilenchev »

OpenAI use well-know linguistics tricks relying on conjunctions(joining words) to separate the text from which they plagiarize on clusters from 5-7 words, exactly the capacity of human short-term memory is. They also use the Google search engine for custom queries, with date restriction which helps them to plagiarize from different texts written on a particular topic so they to be sure that when they copy, paste and concatenate clusters of words in new text, all these phrases of 5-7 words should be related to one topic. This creates the illusion of meaningfulness at first glance.

GPT-3 on Progress. “Civilization rose on the exponential curve. We shouldn’t expect progress to follow a straight line.”

Google with date restriction before 1 April 2020: "progress to follow a straight line".

Do you see only one result that come :-)

https://chrismukiibi.com/2019/11/26/the-valley-of-disappoint...

"We shouldn’t expect progress to follow a straight line." and "we expect our progress to follow a straight line"

Do you understand now, how they use conjunctions(joining words) to insert or to delete insignificant words as "shouldn't" and "our" to plagiarize so that they are not caught.

Post reply on HN