Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: ChatGPT is 'not particularly innovative'

zdnet.com

Post reply on HN
Print view View original

111–120 of 190 posts

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#111

Post by hbrn »

Lambdanaut wrote: post #102
hbrn wrote: post #87

Earlier quoted context omitted.

> ChatGPT has no idea what they're talking about. It lacks it's own thought and heuristic patterns, and only gives you the most-likely response to your prompt Funny thing is, same could be said about lots of people. Try listening to any political debate. Which leads me to believe that the thing that's missing from AI is the same thing that we miss in those political debates: ability to explain and justify your own th…

There's the argument that any AI built on silicon is doomed to be analogous to the Chinese Room* because it is reducible to its parts. It's interesting that I am also reducible to my atomic parts. I may be a Chinese Room that has not yet experienced fully looking down at my self. I don't have enough self-recursive input yet to see through the illusion. * https://en.wikipedia.org/wiki/Chinese_room#Chinese_room_thou...

An interesting way to escape being reducible to your atomic parts is quantum consciousness. Not that I believe in it, I just find the theory too beautiful to ignore.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#113

Post by cwillu »

smoldesu wrote: post #25
cl42 wrote: post #12

It's amazing how many researchers underestimate the importance of UX and design. Personally, seeing non-technical people using prompts for the first time -- and getting results that make sense -- is so incredible. Their eyes light up, they are surprised, and they want to keep playing with it. Amazing! A sizeable part of the population just got access to an incredibly complex AI model and can play with it. What a coll…

Clarke's third law: "Any sufficiently advanced technology is indistinguishable from magic." Most people don't understand that ChatGPT has no idea what they're talking about. It lacks it's own thought and heuristic patterns, and only gives you the most-likely response to your prompt. People don't know that though, so they think the Mechanical Turk is actually playing chess. I mostly agree with the headline here. ChatG…

https://astralcodexten.substack.com/p/janus-simulators

    But the essay brings up another connotation: to simulate
    is to pretend to be something. A simulator wears many masks.
    If you ask GPT to complete a romance novel, it will simulate
    a romance author and try to write the text the way they
    would. Character.AI lets you simulate people directly,
    asking GPT to pretend to be George Washington or Darth Vader.
[…]

    This answer is exactly as fake as the last answer where it
    said it liked me, or the Darth Vader answer where it says it
    wants to destroy me with the power of the Dark Side. It’s
    just simulating a fake character who happens to correspond
    well to its real identity.
[…]

    The whole point of the shoggoth analogy is that GPT is
    supposed to be very different from humans. But however
    different the details, there are deep structural
    similarities. We’re both prediction engines fine-tuned
    with RHLF.

    And when I start thinking along these lines, I notice that
    psychologists since at least Freud, and spiritual traditions
    since at least the Buddha, have accused us of simulating a
    character. Some people call it the ego. Other people call
    it the self.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#114

Post by sharemywin »

time_to_smile wrote: post #15

He's not wrong. The key ingredient in ChatGPT is not brilliance but capital. It takes a lot of money not just to do the raw training but to get all the data together and process everything. There's no break through insights that make ChatGPT work, just a lot of consolidated wealth. The hacker part of me finds AI less and less interesting for this reason. We're seeing what the limits of pouring resources into the prob…

I find it the opposite. There were a lot of projects I didn't have the time to work on and got bored to fast learning. So, now I can tell chatgpt to build it for me. and I only have to debug the 1 or 2 things it can't do. I built a blogging platform, a voice based chatbot using gpt3, something that takes blog article and trims out all the garbage html so only the basic html is left. created a python api, and called it from shared hosting using php. something that trims the times out of youtube captions so I can post it into chatgpt to give me an outline. I just tell it what I want the function to do and it does it.

It's helped me way up my own personal productivity and creativity.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#115

Post by rafaelero »

jmugan wrote: post #105
rafaelero wrote: post #81

Earlier quoted context omitted.

> Most people don't understand that ChatGPT has no idea what they're talking about. I wonder why people keep saying this. Is it some type of psychological defense to future AI displacement?

It's because ChatGPT's knowledge doesn't come through interaction with the world. Words mean things to us because they point to world interactions. I saw a demo where ChatGPT could create a VM. It could be trained to interact directly with a VM, send commands to an interpreter. In this case, it would understand the response of the VM, although it wouldn't understand the design behind the VM, because humans did that b…

Based on this alleged limitation, can you list tasks that you think AI's won't ever be able to succeed at?

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#116

Post by fnordpiglet »

helloplanets wrote: post #69
smoldesu wrote: post #41

Earlier quoted context omitted.

As far as most people can be fooled by coherent-looking text.

Hold on, at least for me, 90% of the stuff is not just coherent-looking, but coherent. I do think that when it's wrong, it should give a heads up about not being certain about a subject. It's certainly tweaked to sound almost overconfident about every subject, which gives off a bullshitty vibe when the details of the explanation are wrong. What sorts of subjects have you been trying it out with?

I’ve had pretty complex discussions about Buddhism, physics, and other subjects and it was generally erudite and accurate. In its current unrefined form it’s more useful than google is at providing understanding on most subjects, especially because it will attempt to answer my direct questions rather than providing documents with terms in them. In fact I think it’s probably one of the most useful tools I’ve ever used.

Has it given me wrong information? Absolutely. But it’s always been pretty obviously wrong, and I often use it to introduce me to a subject then follow up with google to verify details. I further fully expect this to improve.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#117

Post by boto3 »

jmugan wrote: post #105
rafaelero wrote: post #81

Earlier quoted context omitted.

> Most people don't understand that ChatGPT has no idea what they're talking about. I wonder why people keep saying this. Is it some type of psychological defense to future AI displacement?

It's because ChatGPT's knowledge doesn't come through interaction with the world. Words mean things to us because they point to world interactions. I saw a demo where ChatGPT could create a VM. It could be trained to interact directly with a VM, send commands to an interpreter. In this case, it would understand the response of the VM, although it wouldn't understand the design behind the VM, because humans did that b…

It did, actually. The model was trained with multiple rounds of reinforcement learning where human judges provided the feedback: first with full answers, and then with ranking of answers as most relevant.

So the model in production is probably frozen, but before that it went through multiple rounds of interaction with the world.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#118

Post by PartiallyTyped »

visarga wrote: post #43
PartiallyTyped wrote: post #23

Earlier quoted context omitted.

It's quite fascinating how information retrieval and search engines have evolved.. From trying to teach people how to google via "I am feeling lucky", to using language models for ranking, to building LLMs to better adapt to user queries and move beyond keywords, to having arguably useful chatbots that can synthesize responses. I am curious to see what the future holds. Maybe an AI that can anticipate our queries?

I also saw the "LLM as database" metaphor. Up until 2020 we had databases in the backend, UI in front, now we can have LLMs in the backend.

For the uninitiated https://news.ycombinator.com/item?id=34503418

Maybe, eventually, LLMs can be used to synthesize and cache certain APIs ... who knows :D

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#119

Post by visarga »

hbrn wrote: post #111
Lambdanaut wrote: post #102

Earlier quoted context omitted.

There's the argument that any AI built on silicon is doomed to be analogous to the Chinese Room* because it is reducible to its parts. It's interesting that I am also reducible to my atomic parts. I may be a Chinese Room that has not yet experienced fully looking down at my self. I don't have enough self-recursive input yet to see through the illusion. * https://en.wikipedia.org/wiki/Chinese_room#Chinese_room_thou...

An interesting way to escape being reducible to your atomic parts is quantum consciousness. Not that I believe in it, I just find the theory too beautiful to ignore.

A beauty that means to blind the mind and stop you looking further. Penrose is a dead end.

Re: Yann LeCun: ChatGPT is 'not particularly innovative'

#120

Post by visarga »

fnordpiglet wrote: post #104
smoldesu wrote: post #25

Earlier quoted context omitted.

Clarke's third law: "Any sufficiently advanced technology is indistinguishable from magic." Most people don't understand that ChatGPT has no idea what they're talking about. It lacks it's own thought and heuristic patterns, and only gives you the most-likely response to your prompt. People don't know that though, so they think the Mechanical Turk is actually playing chess. I mostly agree with the headline here. ChatG…

If it weren’t that remarkable, how come Siri, alexa, and whatever google calls it’s unused voice assistant are so useless? If it were not a substantial advance, why are chatbots so useless? The reality is, if Alexa had half the ability to “understand” as chatgpt it wouldn’t be nearly as frustrating. It’s not like Amazon, apple, and google haven’t dumped metric tons of investment into these things. Y’all simply have l…

It's more pragmatic than that. Google can afford to run a 300M weights model on every search or click, but can't afford a 175B generative model for the same thing. Generative models require a forward pass per token, all done serially, while embedding or classification models only need one pass no matter the length of the input.

It's too expensive for Google.

Post reply on HN