Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is ChatGPT doing and why does it work?

writings.stephenwolfram.com

Post reply on HN
Print view View original

471–480 of 518 posts

Re: What is ChatGPT doing and why does it work?

#471

Post by JieJie »

lelanthran wrote: post #219
cm2012 wrote: post #188

Earlier quoted context omitted.

For now. Give it a truly persistent memory and 100x the size of the dataset I think most people would change their tune.

> For now. Give it a truly persistent memory and 100x the size of the dataset I think most people would change their tune. Why does it need 100x the dataset? Sentient creatures, including humans, manage to figure stuff out from as little as a single datapoint . For a human to differentiate between a cat and a dog takes, maybe, two examples of each, not a few million pictures. An adult human who sees a hotdog for the…

From the article:

"How much data do you need to show a neural net to train it for a particular task? Again, it’s hard to estimate from first principles. Certainly the requirements can be dramatically reduced by using “transfer learning” to “transfer in” things like lists of important features that have already been learned in another network. But generally neural nets need to “see a lot of examples” to train well. And at least for some tasks it’s an important piece of neural net lore that the examples can be incredibly repetitive. And indeed it’s a standard strategy to just show a neural net all the examples one has, over and over again. In each of these “training rounds” (or “epochs”) the neural net will be in at least a slightly different state, and somehow “reminding it” of a particular example is useful in getting it to “remember that example”. (And, yes, perhaps this is analogous to the usefulness of repetition in human memorization.)"

Re: What is ChatGPT doing and why does it work?

#472

Post by the_af »

williamcotton wrote: post #167
janalsncm wrote: post #146

Earlier quoted context omitted.

In my experiments that doesn’t help. ChatGPT failed to play any valid move given a position, let alone a good one. The point is that language is probably too “linear” to represent what’s going on during a game. Pieces on the board have complex relationships (consider a pinned piece, for example) so autoregressive decoding is simply not enough.

Try something like: The current board state is: board = [['', '', ''], ['', '', ''], ['', '', '']]; write a javascript function called bestMove(board) that predicts the best tic-tac-toe move to make given a board. use that function to update the board state and return the board state in JSON form. The response will have a bunch of functions like function bestMove(board) { function getEmptySpaces(board) { function pre…

Interesting. Have you tried playing a full game like this, instead of a single move?

In any case, I don't think this is what people expect out of ChatGPT. Your approach is too "programmer centric". I think people expect telling ChatGPT the rules of the game, in almost plain language, and then expect to be able to play a game of Tic Tac Toe interacting with it like one would with a person. This means, not asking it to write functions or remind it of the state of the board at every step.

This doesn't work consistently for a well-known game like Tic Tac Toe, much less for an arbitrary game you make up.

Re: What is ChatGPT doing and why does it work?

#473

Post by JW_00000 »

sebzim4500 wrote: post #442
SilverBirch wrote: post #337

Earlier quoted context omitted.

Such a hacker news comment. The title isn't a question Stephen Wolfram is asking you a question, it's the title of an article he's written that answers the question.

What exactly do you think the point of this site is, if not to discuss the question raised in the article?

Discussing the contents of the article, instead of only its title. OP seems to have taken just the question and provided his own answer, without relating back to anything raised in the article.

Re: What is ChatGPT doing and why does it work?

#474

Post by basch »

kortilla wrote: post #295
basch wrote: post #187

Earlier quoted context omitted.

Humans are somewhat a blank slate, and culture is our initial prompt. The variety of humans is because of the variety of our initial prompts, and our similarities are because of the similar characteristics of our various cultures. (I of course recognize some of our intelligence is genetic or epigenetic or microbial.)

No, chatgpt is already trained once it’s ready for prompts.

It’s training is like weighted frequencies, predispositions, instincts. A giant word association machine.

Then whatever prompt it is fed is what it becomes.

Re: What is ChatGPT doing and why does it work?

#475

Post by galaxyLogic »

JW_00000 wrote: post #359
galaxyLogic wrote: post #265

Earlier quoted context omitted.

>... chatGPT displays a sort of self awareness. If you read books or articles you will find many places where it appears that whoever wrote them was referring to him- or herself and was describing themselves. And thus we say that whoever wrote such a text seemed to be aware that they were the ones outputting the text. Because there are many such texts in the training-set of the ChatGPT etc. the output of it will also…

Indeed. When Hamlet ponders "to be or not be", is he contemplating death and suicide? You could answer this question with "yes". (Wikipedia even says so.) But you could also say: obviously not, Hamlet is not a real person with a brain, so he can't contemplate anything. It's actually Shakespeare contemplating, and ascribing his thoughts to a fictional character. When ChatGPT "realizes" it's a virtual machine emulator,…

When I was a kid there was perhaps 1 year younger girl in the same apartment-building I lived, and we all played together. I took notice that she always referred to herself in 3rd person, citing her name ("Kaija") first. She used to say "Kaija wants this" etc. I thought that was stupid but later I read it's a developmental stage in children where they don't really grasp the concept of "self" yet.

But now I think she probably was as self-aware as anybody else in the group of kids, she just didn't know the language, how to refer to herself other than by citing her name.

Later Kaija learned to speak "properly". But I wonder was she any more self-aware then than she was before. Kids just learn the words to use. They repeat them, and observe what effect they have on other people. That is part of the innate learning they do.

ChatGPT is like a child who uses the word "I" without really thinking why it is using that word and not some other word.

At the same time it is true that "meaning" arises out of how words are used together. To explain what a word means you must use other words, which similarly only get their meaning from other words, and ultimately what words people use in what situations and why. So in a way ChatGPT is on the road to "meaning" even if it is not aware of that.

Re: What is ChatGPT doing and why does it work?

#476

Post by mlrhazi »

An analogy I've been thinking about, to try an explain ChatGPT to non IT/CS family/friends: - Imagine a very smart Englishman, who never had any exposure to Chinese language, gets kidnapped by some Chinese mafia dudes. for days/months/years, he is listening to them talking.... and at the end, he can "engage" in any conversation, in Chinese.... even though he still knows absolutely nothing about Chinese, not the meaning of any single word!!! That's ChatGPT or generative AI.

How good/bad that is? How to improve it?

Re: What is ChatGPT doing and why does it work?

#477

Post by ghosty141 »

chongli wrote: post #349
jokoon wrote: post #346

To me, modern AI is just "black boxes all the way down". Even specialists don't really know what's happening. It's not encouraging or interesting. Personally, I'm more interested in analyzing those black boxes than tinkering ones that "seems to work", would it be with graph theory, analysis, etc. To me, if something works but we're unable to really understand why it does, it's more the realm of "testing broken clocks…

I agree. I’m currently taking a neural networks course and it’s making my discomfort worse, not alleviating it. At the beginning of the course we talked about biological models of neurons and that was pretty cool, if a bit simplistic. Now we’re deep into automatic differentiation and gradient descent and a bunch of hidden layers. Ultimately it’s all just using calculus to approximate some unknown function given a sam…

> There is no path to understanding, from what I can see. It’s pure instrumentalism and parlour tricks.

Exactly like nature. It feels like emulating whats happening in nature for a very specific usecase under some constraints.

Re: What is ChatGPT doing and why does it work?

#478

Post by bbor »

kilgnad wrote: post #129
spion wrote: post #15

The answer to this is: "we don't really know as its a very complex function automatically discovered by means of slow gradient descent, and we're still finding out" Here are some of the fun things we've found out so far: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (…

Finally I'm tired of people saying it's just a probabilistic word generator and downplaying everything as if they know. If you said something along these lines before... then these papers show that you're not fully grasping the situation here. There are clearly different angles of interpreting what these models are actually doing but people are stubbornly refusing to believe it's anything more then just statistical w…

How could LLMs have awareness of self if they’re not capable of forming new knowledge or metacognition? How can you be conscious if you only exist for a split second at a time - long enough to read and respond to a prompt - then disappear forever?

Really think Occam’s razor is useful here. If we define this as consciousness, then it’s a very very different kind than ours.

Re: What is ChatGPT doing and why does it work?

#479

Post by spion »

Rury wrote: post #165
kilgnad wrote: post #156

Earlier quoted context omitted.

Well you're just a jumble of electrons and protons interacting with each other. That's a 100% true interpretation is it not? It's also a view that misses the point that you are a molecular intelligence made of DNA that continually mutates and reconstructs it's physical form with generational copies to increase fitness in an ever changing environment. But that viewpoint also misses the point that you're a human with w…

I still don't see your point behind your first 4 paragraphs. How you decide to treat your fellow humans is up to you. Just as you can decide to view your fellow humans however you want. It's entirely possible to treat them like "humans" while still viewing them as nothing but jumbles of molecules and atoms. So again why does perspective matter here (particularly with ChatGPT being a statistical word generator)? Your…

The fundamental bit I was aiming at here is that running gradient descent or similar training algorithms on a thing doesn't mean you will understand the end result fully - and we don't understand it fully. Its entirely possible that to predict the next word correctly, ChatGPT had to build / encode some kind of model of the universe (as written down by humans) within its weights.

We know its not possible to encode the full probabilities of what words will follow what other words, as the article itself states [1].

So how do you best "compress" these probabilies? By trying to find the most correct generalizations that are more widely appplicable? Perhaps even developing meta-facilities in recognizing good generalizations from bad?

[1]: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-....

Re: What is ChatGPT doing and why does it work?

#480

Post by joenot443 »

ChaitanyaSai wrote: post #273

This misses that key point that all this prediction can give rise to what looks like astonishing human-level creativity and across many genres. The last decade and half have shown us that with enough data we can pick out patterns well enough to be able to "categorize". But to create , that seemed like a whole another human level outside the realm of mere prediction. Turns out it isn't. What exactly allows LLMs to hav…

Do you listen to much Eminem? Imagining that in his voice made me laugh, there’s just no way he’d come up with something so bad. It’s similar to the shader code it produces; passable to the naked eye, but upon any level of scrutiny, completely falls apart.
Post reply on HN