Live data from Hacker News

Yann LeCun: ChatGPT is 'not particularly innovative'

zdnet.com

91–100 of 190 posts

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#91

Earlier quoted context omitted.

As far as most people can be fooled by coherent-looking text.

Hold on, at least for me, 90% of the stuff is not just coherent-looking, but coherent. I do think that when it's wrong, it should give a heads up about not being certain about a subject. It's certainly tweaked to sound almost overconfident about every subject, which gives off a bullshitty vibe when the details of the explanation are wrong. What sorts of subjects have you been trying it out with?

I think part of the point here is that ChatGPT has no clue what you might mean here by "certainty" or being "wrong," and the fact that people have the impression that it could have an idea of those concepts is indicative of people's poor understanding of what ChatGPT really is

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#92

Earlier quoted context omitted.

> Most people don't understand that ChatGPT has no idea what they're talking about. I wonder why people keep saying this. Is it some type of psychological defense to future AI displacement?

maybe, but it's also just the fact of the matter.

That's not even a testable claim. What does it even mean to "not know what it is talking about"? If OP tried to operationalize its beliefs by making some prediction, like listing tasks that AI will never be able to reproduce because it "doesn't know what it is talking about", then that would be a discussion.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#93
post #72
post #12

It's amazing how many researchers underestimate the importance of UX and design. Personally, seeing non-technical people using prompts for the first time -- and getting results that make sense -- is so incredible. Their eyes light up, they are surprised, and they want to keep playing with it. Amazing! A sizeable part of the population just got access to an incredibly complex AI model and can play with it. What a coll…

Researchers point of view is based on their area of research and that's fair and expected. Yann LeCun compares ChatGPT in the context of the related research. Imagine a ChatGPT equivalent that memorizes many questions and does a brute force strategy for an answer. It may "look" magic, but there's nothing magic about it. We all accepted that this is the case with Blue Gene - https://en.wikipedia.org/wiki/Deep_Blue_(ch…

But ChatGPT does not use brute force search to look for an answer. It interpolates among the answers in its training set. I.e. in Yann LeCun's analogy of a cake, interpolation or unsupervised learning is the cake; direct feedback on each single data point or supervised learning is the icing on the cake; and general feedback of the "how I am doing" sort or reinforcement learning is the cherry on top. Now LeCun is just saying that the cake is a lie, and leaving it at that. I don't think this is a helpful understanding.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#94
post #80
post #26

Earlier quoted context omitted.

Decade ago, definitely not. But the answer to that question, time aside, is, well, GPT-3. No doubt ChatGPT is better, but it does seem like a decent bit of the improvement was UX related, since with the right prompts people have shown similarly miraculous output from GPT-3. Of course, I'm pretty sure the quoted person is referring to the fact that Google and others likely have similarly capable LLMs already that they…

Yes, but along that reasoning the criticism that "ChatGPT is not particularly innovative" is a bit disingenuous in the sense that the general public has never had easy access to these models until ChatGPT arrived. Besides, even though the newer GPT models presumably just extended earlier ones with similar techniques but better hardware and data, it wasn't a foregone conclusion how powerful (from the user perspective)…

I don't think the point here is to downplay the significance of ChatGPT as a product, but to point out that OpenAI is likely not uniquely resourced in this regard. If anything, it sounds like what they're trying to say is that ChatGPT is only the tip of the iceberg, and the reason why it came out of OpenAI and not Meta or Google is more a function of other factors like reputational damage, etc. The headline does hit as a bit sensationalist or dismissive if you read it literally, as if it's claiming the work that went into ChatGPT was all easy and no big deal. Rather though, it seems like it's just trying to point out to the zeitgeist currently enamored with ChatGPT and all of the new possibilities, that other companies also have this capability today and that transformers have been more of an evolutionary process than a revolutionary process.

Maybe I'm reading it too leniently, but that was my takeaway.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#95

Lots of salty researchers are shitting on ChatGPT meandering every single person I know is shocked. If this isn’t so innovative why didn’t they release something sooner?

If you actually read the article he answers your question directly. If you are too lazy to scroll to the bottom here's a link to a tweet: https://twitter.com/ylecun/status/1617908306420600833?s=20&t...

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#96

Earlier quoted context omitted.

Clarke's third law: "Any sufficiently advanced technology is indistinguishable from magic." Most people don't understand that ChatGPT has no idea what they're talking about. It lacks it's own thought and heuristic patterns, and only gives you the most-likely response to your prompt. People don't know that though, so they think the Mechanical Turk is actually playing chess. I mostly agree with the headline here. ChatG…

> Most people don't understand that ChatGPT has no idea what they're talking about. I wonder why people keep saying this. Is it some type of psychological defense to future AI displacement?

It definitely could be - it's on every thread here and on reddit.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#97

Earlier quoted context omitted.

Clarke's third law: "Any sufficiently advanced technology is indistinguishable from magic." Most people don't understand that ChatGPT has no idea what they're talking about. It lacks it's own thought and heuristic patterns, and only gives you the most-likely response to your prompt. People don't know that though, so they think the Mechanical Turk is actually playing chess. I mostly agree with the headline here. ChatG…

Except that the Mechanical Turk is actually playing chess - there is no hidden chess master.

[deleted]

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#98
post #12

It's amazing how many researchers underestimate the importance of UX and design. Personally, seeing non-technical people using prompts for the first time -- and getting results that make sense -- is so incredible. Their eyes light up, they are surprised, and they want to keep playing with it. Amazing! A sizeable part of the population just got access to an incredibly complex AI model and can play with it. What a coll…

Clarke's third law: "Any sufficiently advanced technology is indistinguishable from magic." Most people don't understand that ChatGPT has no idea what they're talking about. It lacks it's own thought and heuristic patterns, and only gives you the most-likely response to your prompt. People don't know that though, so they think the Mechanical Turk is actually playing chess. I mostly agree with the headline here. ChatG…

> ChatGPT is hardly any more innovative than a Markov chain

A smartphone is hardly any more innovative than the ENIAC. And yet ...

Do not underestimate the power of making tech useful and accessible. The lightbulb means nothing without the power grid after all.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#99
post #27

Earlier quoted context omitted.

> and only gives you the most-likely response to your prompt That "only" carries a lot of weight on its back.

The most likely response is not necessarily an accurate response. It claims dogs do not give birth to babies. That is indeed the most likely response as babies are mostly, but not exclusively, associated with humans. However, a five year old child would understand me if I told them my dog had babies.

Have you used chatgpt much?

I asked your prompt:

Do dogs give birth to babies?

> Dogs give birth to puppies.

I had a long discussion about the expansion do space and the Hubble constant, and tried to trick it in a variety of ways. It was correct in everything even though it wasn’t able to go into proofs and likewise. I then tried to convince it that the universe began as a small potato and wasn’t able to do it without directly telling it to assume it is and to create fiction off that concept. It was then able to make a credible theory of what the universe would be like if it started as a small potato, even going into how the texture of the potato would have caused variations in the density of mass, etc. Even then it would find ways to sneak in this isn’t true and that the universe was never a small potato. This is remarkable no matter the actual reality of what’s going on behind the scenes.

I would also note that human thought is just a bunch of gradient functions in the electrochemical space, and thus also follows a most likely path.

Edit: to head it off at the pass, here is what it says with your other prompt:

My dog gave birth to babies

> Congratulations on the new additions to your family! If this is your dog's first litter, it is important to make sure that both the mother and the puppies receive proper care and attention. This includes providing a clean and safe environment for the puppies, as well as making sure the mother is healthy and well-fed. It's also important to keep an eye on the puppies for any signs of illness or distress, and to consult with a veterinarian if you have any concerns.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#100
post #12

It's amazing how many researchers underestimate the importance of UX and design. Personally, seeing non-technical people using prompts for the first time -- and getting results that make sense -- is so incredible. Their eyes light up, they are surprised, and they want to keep playing with it. Amazing! A sizeable part of the population just got access to an incredibly complex AI model and can play with it. What a coll…

And/or underestimate the importance of shipping.

[deleted]
Post reply on HN