Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: ChatGPT is 'not particularly innovative'

zdnet.com

Post reply on HN
Print view View original

101–110 of 190 posts

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#101

Post by belval »

cl42 wrote: post #85
random3 wrote: post #72

Earlier quoted context omitted.

Researchers point of view is based on their area of research and that's fair and expected. Yann LeCun compares ChatGPT in the context of the related research. Imagine a ChatGPT equivalent that memorizes many questions and does a brute force strategy for an answer. It may "look" magic, but there's nothing magic about it. We all accepted that this is the case with Blue Gene - https://en.wikipedia.org/wiki/Deep_Blue_(ch…

OK, I'll defend the research, too. OpenAI's really interesting approach to GPT was to scale the size of the underlying neural network. They noticed that the performance of an LLM kept improving as the size of the network grew so they said, "Screw it, how about if we make it have 100+ billion parameters?" Turns out they were right. From a research perspective, I'd say this was a big risk and it turned out they were ri…

> OpenAI's really interesting approach to GPT

That's the issue though, Yann LeCun is specifically referring to ChatGPT as the standalone model, not the GPT family since a lot of models at Meta, Google, DeepMind are based on a similar approach. His point is that ChatGPT is a cosmetic additional training with prompt with a nice interface, but not a fundamentally different model than stuff we've have had for +2-3 years at this point.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#102

Post by Lambdanaut »

hbrn wrote: post #87
smoldesu wrote: post #25

Earlier quoted context omitted.

Clarke's third law: "Any sufficiently advanced technology is indistinguishable from magic." Most people don't understand that ChatGPT has no idea what they're talking about. It lacks it's own thought and heuristic patterns, and only gives you the most-likely response to your prompt. People don't know that though, so they think the Mechanical Turk is actually playing chess. I mostly agree with the headline here. ChatG…

> ChatGPT has no idea what they're talking about. It lacks it's own thought and heuristic patterns, and only gives you the most-likely response to your prompt Funny thing is, same could be said about lots of people. Try listening to any political debate. Which leads me to believe that the thing that's missing from AI is the same thing that we miss in those political debates: ability to explain and justify your own th…

There's the argument that any AI built on silicon is doomed to be analogous to the Chinese Room* because it is reducible to its parts.

It's interesting that I am also reducible to my atomic parts.

I may be a Chinese Room that has not yet experienced fully looking down at my self. I don't have enough self-recursive input yet to see through the illusion.

* https://en.wikipedia.org/wiki/Chinese_room#Chinese_room_thou...

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#103

Post by visarga »

trefoiled wrote: post #91
helloplanets wrote: post #69

Earlier quoted context omitted.

Hold on, at least for me, 90% of the stuff is not just coherent-looking, but coherent. I do think that when it's wrong, it should give a heads up about not being certain about a subject. It's certainly tweaked to sound almost overconfident about every subject, which gives off a bullshitty vibe when the details of the explanation are wrong. What sorts of subjects have you been trying it out with?

I think part of the point here is that ChatGPT has no clue what you might mean here by "certainty" or being "wrong," and the fact that people have the impression that it could have an idea of those concepts is indicative of people's poor understanding of what ChatGPT really is

Are you talking about the same people who couldn't agree about a vaccine? How is logical consistency going with humans?

Not to mention that a mere 700 years ago we were dying of bubonic plague and with all our general intelligence could not muster up the germ theory of disease. Not even to save our lives, you see, we are not generally efficient, it depends on century.

We are dependent on experimental results carefully constructed to verify our theories, theories which start like chatGPT's random bullshit initially, random words following a probability distribution in our heads. Even deep learning is often touted as modern alchemy - why don't we just understand?

Verification does wonders to language models. Humans have more verification and interactive experiences, so we think ourselves superior. But an AI could have the same grounding with us. Like AlphaZero who became a super-human Go player without ever looking at human games - learned it from lots of verification. And CICERO the model playing Diplomacy in natural language.

Just set AI up with a verification loop to see wonders. Predicting the next word correctly is just one of the ways AI can learn.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#104

Post by fnordpiglet »

smoldesu wrote: post #25
cl42 wrote: post #12

It's amazing how many researchers underestimate the importance of UX and design. Personally, seeing non-technical people using prompts for the first time -- and getting results that make sense -- is so incredible. Their eyes light up, they are surprised, and they want to keep playing with it. Amazing! A sizeable part of the population just got access to an incredibly complex AI model and can play with it. What a coll…

Clarke's third law: "Any sufficiently advanced technology is indistinguishable from magic." Most people don't understand that ChatGPT has no idea what they're talking about. It lacks it's own thought and heuristic patterns, and only gives you the most-likely response to your prompt. People don't know that though, so they think the Mechanical Turk is actually playing chess. I mostly agree with the headline here. ChatG…

If it weren’t that remarkable, how come Siri, alexa, and whatever google calls it’s unused voice assistant are so useless? If it were not a substantial advance, why are chatbots so useless? The reality is, if Alexa had half the ability to “understand” as chatgpt it wouldn’t be nearly as frustrating. It’s not like Amazon, apple, and google haven’t dumped metric tons of investment into these things.

Y’all simply have lost the ability to be amazed.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#105

Post by jmugan »

rafaelero wrote: post #81
smoldesu wrote: post #25

Earlier quoted context omitted.

Clarke's third law: "Any sufficiently advanced technology is indistinguishable from magic." Most people don't understand that ChatGPT has no idea what they're talking about. It lacks it's own thought and heuristic patterns, and only gives you the most-likely response to your prompt. People don't know that though, so they think the Mechanical Turk is actually playing chess. I mostly agree with the headline here. ChatG…

> Most people don't understand that ChatGPT has no idea what they're talking about. I wonder why people keep saying this. Is it some type of psychological defense to future AI displacement?

It's because ChatGPT's knowledge doesn't come through interaction with the world. Words mean things to us because they point to world interactions. I saw a demo where ChatGPT could create a VM. It could be trained to interact directly with a VM, send commands to an interpreter. In this case, it would understand the response of the VM, although it wouldn't understand the design behind the VM, because humans did that based on interaction with the physical world.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#106

Post by DrSiemer »

rafaelero wrote: post #81
smoldesu wrote: post #25

Earlier quoted context omitted.

Clarke's third law: "Any sufficiently advanced technology is indistinguishable from magic." Most people don't understand that ChatGPT has no idea what they're talking about. It lacks it's own thought and heuristic patterns, and only gives you the most-likely response to your prompt. People don't know that though, so they think the Mechanical Turk is actually playing chess. I mostly agree with the headline here. ChatG…

> Most people don't understand that ChatGPT has no idea what they're talking about. I wonder why people keep saying this. Is it some type of psychological defense to future AI displacement?

No, it is an accurate evaluation of what ChatGPT does. The model is not trying to explain something to you, it is trying to convince itself that what it is writing looks like it could have been written by a human.

There is no logical thinking involved. Output can appear to be real communication, but it's basically just a very advanced trick, that has some serious drawbacks.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#108

Post by soperj »

babypuncher wrote: post #57
soperj wrote: post #6

This is very much people seeing the Apple Lisa(and/or Mac) for the first time. It had all been done better in the labs at Xerox over a decade earlier. No one got to use it though.

While Xerox did it first, I think it's a stretch to say they did it better than the Macintosh.

No it really isn't.

https://www.quora.com/How-did-the-original-Macintosh-compare...

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#109

Post by seydor »

It seems he s trying to say something that is technically correct, but exactly at the wrong time. It s going to be a lot of backlash if you go against the hype wave that we are riding. Better to explain in a more nuanced way how chatGPT managed to become so popular despite being an incremental improvement of fundamental findings of the past few years.

People love it because it understands language, every time and without fail. So what if it spits out lies, this thing makes people happy, which is the value at play here

Post reply on HN