Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is ChatGPT doing and why does it work?

writings.stephenwolfram.com

Post reply on HN
Print view View original

81–90 of 518 posts

Re: What is ChatGPT doing and why does it work?

#81

Post by Barrin92 »

jonahx wrote: post #69
Barrin92 wrote: post #62

Earlier quoted context omitted.

Realness has little to do with the quality of the output. Or else my TI-83 would be the realest mathematician on the planet, it can multiply numbers that even Terence Tao can't in his head. Judging these systems solely by their output is to repeat the msitakes of behaviorism, even a dumb markov chain or a parrot converses better than an infant, but unlike the infant does not acquire an understanding or representation…

> has little to do with the quality of the output. Non-snarky question: What else can you judge by? Isn't any alternative just putting more precise conditions on the output? With ChatGPT, it's still easy enough to see it's mistakes, and its attempts at fiction and poetry, impressive as they are, are still clumsy to a trained eye, relative to expert human work. But what if they weren't? What happens when they're indis…

>Non-snarky question: What else can you judge by?

its architecure. A child is a living and autonomous agent. It has (or develops) meta cognition, an awareness of its own mental state (and by extension use of language). These models don't have the capacity to do this even in theory given that they're static and pretrained. When you ask ChatGPT what it feels like to speak, there isn't some neural activity within the model, it has no model of itself that it actively inspects, it doesn't learn while it converses with you, it just tells you what someone wrote on Quora two years ago.

>What happens when they're indistinguishable?

Then the system is likely going to look a lot different than it does now because these aspects of cognition seem pretty important when you want something that is genuinely human-like rather than just mimicry or memorization.

Re: What is ChatGPT doing and why does it work?

#83

Post by IIAOPSW »

leereeves wrote: post #72
IIAOPSW wrote: post #50

Earlier quoted context omitted.

Notice I didn't say anything about chatGPT. chatGPT is at least something that's been built. Disagree with the theory that we are overgrown text prediction all you want, but you can't deny that it is an "understanding" in the sense that knowing it lets you build and test it. Let's call it a "potential understanding". This is different from an "illusory understanding", which feel like explanations but when you go to i…

> Disagree with the theory that we are overgrown text prediction all you want, but you can't deny that it is an "understanding" in the sense that knowing it lets you build and test it. Yeah, and it helps us understand how "overgrown text prediction" works, but not how humans work. In the same way that building a robotic arm won't help you understand how muscles work. > If you can't build anything out of an "understan…

Building a robot arm does help understand how muscles work. By having a robot arm to play with, you can pinpoint exactly the abilities which aren't accounted for by your current understanding. Boston Dynamics built that robot dog thing. It took decades to make robotic kinematics that good. The dog demonstrates an understanding of a system of muscles that solves the same problems as its biological counterpart.

You say that overgrown text prediction isn't how humans work. And full disclosure you're probably right. But me put on my contrarian hat and say that actually you're wrong and that really is all there is to the brain. At what point does my theory break? What types of things can't be done with just overgrown text prediction, and what features are relevant to a system that could do those things? Don't just appeal to intuition and tell me humans obviously don't work that way. Find the actual flaw where the theory breaks down. That is the value of this experiment.

If you can find the words / experiments to demonstrate why overgrown text prediction isn't an accurate understanding of human thought, in the process you will have in fact distilled a better understanding of human thought. Information on how the brain doesn't work is also information about how the brain works.

Re: What is ChatGPT doing and why does it work?

#84

Post by sigmoid10 »

lossolo wrote: post #76
codeulike wrote: post #34

Earlier quoted context omitted.

That Kenneth Li Othello paper is great. The accompanying blog post https://thegradient.pub/othello/ was discussed on HN here https://news.ycombinator.com/item?id=34474043 A lot of people didn't seem to get it when it was discussed on HN. A GPT had _only_ ever seen Othello transripts like: "E3, D3, C4 ..." and NOTHING else. It knows nothing of the board. It doesnt event know that there are two players. It learned Othe…

> "E3, D3, C4 ..." and NOTHING else. It knows nothing of the board. It doesnt event know that there are two players. Yeah, like language have gramma rules games also have rules, in both cases LLM can learn rules, it's the same with many other structured chains of actions/tokens, you could also model actions from different domains and use them as language. It seems a lot of emergent behaviours of LLMs are what you cou…

Knowledge distillation for transformers is already a thing and it is still actively researched since the potential benefits of not having to run these gigantic models are enormous.

Re: What is ChatGPT doing and why does it work?

#85

Post by myrmidon »

rubyist5eva wrote: post #68

It's a bunch of smoke and mirrors akin to blockchain, bitcoin (and the entire web3/crypto space in general) - a whole bunch of hype and "evangelists" who keep saying it's going to change the world (and make them a whole lot of money in the process, how convenient!) but if you do look at it critically even at a surface level you realize it's just a bunch of really computationally expensive BS that isn't any better or…

Strongly disagree with this.

You can call it smoke and mirrors all you want, but its utility is pretty self-evident- you can really just talk with this thing, and it will give reasonable answers. Is it perfect, or even as good as a human? Hell no, but it for sure is not going to get worse, and it's already remarkable in ways that were barely imaginable only a few years ago...

I have a friend that has been using this as an infinitely patient mentor for learning embedded programming, and chatgpt delivers in that capacity unlike any automated system we had before.

If a glorified autocomplete can fake human intelligence reasonably well, maybe we should question our notions of superiority instead of trashtalking the machines...

Re: What is ChatGPT doing and why does it work?

#86

Post by root_axis »

bheadmaster wrote: post #30
SketchySeaBeast wrote: post #22

Earlier quoted context omitted.

In my case it's because my internal dialogue is saying the sentence before I get to the end of it. I usually have the entire sentence in my inner dialogue before I even start typing. Will I edit during typing? Sure, but I have a first version in my head before I start.

> I usually have the entire sentence in my inner dialogue before I even start typing Interesting. Perhaps the question then becomes, does your inner dialogue simply chain the words one after another, or does it come up with sentences as whole?

It's impossible to form meaningful sentences without having a model of the sequence of the words, this is as true for a human as for a markov chain, but that doesn't mean it then follows that human sentences are just sophisticated markov chains, the nature of sentences means that they must be a sequence of words regardless of the implementation details behind the hood of the sentence creator.

Re: What is ChatGPT doing and why does it work?

#87

Post by jmoak3 »

codeulike wrote: post #32

Wow this is 19,000 words. I like his summary at the end: At some level it’s a great example of the fundamental scientific fact that large numbers of simple computational elements can do remarkable and unexpected things. And this: ... But it’s amazing how human-like the results are. And as I’ve discussed, this suggests something that’s at least scientifically very important: that human language (and the patterns of th…

I saw a great comment here, and I will repeat it without the attribution it deserves:

We may have realized it's easier to build a brain than to understand one

Re: What is ChatGPT doing and why does it work?

#88

Post by seba_dos1 »

codeulike wrote: post #32

Wow this is 19,000 words. I like his summary at the end: At some level it’s a great example of the fundamental scientific fact that large numbers of simple computational elements can do remarkable and unexpected things. And this: ... But it’s amazing how human-like the results are. And as I’ve discussed, this suggests something that’s at least scientifically very important: that human language (and the patterns of th…

> that human language (and the patterns of thinking behind it) are somehow simpler and more “law like” in their structure than we thought.

That sounds like a lot of ideas on what makes humans special among other species and how our knowledge on that was being revised over last decades (what's common knowledge on the intelligence of, say, primates or corvids today would be unspeakable blasphemy mere 100 years ago). Various religions have instilled the idea of a human as a sacred entity that's meant to rule over everything because of how special ("made in the image of God") it is, yet we keep learning that we're much simpler than we thought over and over again. I wish for it to result in less hubris in the humanity as a whole.

Re: What is ChatGPT doing and why does it work?

#89

Post by SketchySeaBeast »

sillysaurusx wrote: post #52
SketchySeaBeast wrote: post #22

Earlier quoted context omitted.

In my case it's because my internal dialogue is saying the sentence before I get to the end of it. I usually have the entire sentence in my inner dialogue before I even start typing. Will I edit during typing? Sure, but I have a first version in my head before I start.

Your eyes are telling you what color that shade of red is on the wall. But if you put it next to a different color, you’ll suddenly get a different answer from your eyes. I think if our eyes can deceive us at a fundamental level, it’s arrogant to think we aren’t deceived by our thoughts.

Well then those who think they are coming up the sentence one word at a time are equally as easy to fool, no? They could have the whole sentence figured out and their attention is only coming to a single word at a time, possibly revising what's in their field of mental view if they come up with a better word. That's the fun with these sorts of doubts, if we assume nothing, then we can form an argument for anything.

Re: What is ChatGPT doing and why does it work?

#90

Post by fijiaarone »

kibwen wrote: post #49
matt-attack wrote: post #48

Earlier quoted context omitted.

Can you explain: Prompt: "Has anyone really been far even as decided" Expected transformation: "to use even go want to do look more like?" Those look like gibberish in and gibberish out to me.

It's an old meme: https://knowyourmeme.com/memes/has-anyone-really-been-far-ev...

Which proved ChatGPT wasn’t trained on a corpus of human language, or else it would have worse grammar, more fallacies, and way less consistency.
Post reply on HN