Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is ChatGPT doing and why does it work?

writings.stephenwolfram.com

Post reply on HN
Print view View original

511–518 of 518 posts

Re: What is ChatGPT doing and why does it work?

#511

Post by the_af »

kazinator wrote: post #506
the_af wrote: post #502

Earlier quoted context omitted.

We seem to be talking at cross purposes. I understand (at a very high level) what LLMs do, and I don't think they can do actual computation. Why do you insist on things I've already said I understand? I know ChatGPT is not good at maintaining state -- though it can fake it convincingly (which understandably, seems to trip people up). I think it looks at your chat history within the session in order to generate the ne…

What's the difference between keeping state and looking at the chat history? Keeping state is something a human would have to do, because for a human, it would be very tedious and slow to re-read the history to recover context, relative to the timeliness expectation of the interlocutor.

> What's the difference between keeping state and looking at the chat history?

That's an excellent question. I don't know. Intuitively, looking at the chat history would seem a way to keep history, right?

However, in my tests trying to play Tic Tac Toe (informally, not using javascript functions as the comment I was replying to) ChatGPT constantly failed. It claims to know the rules of Tic Tac Toe, yet it repeatedly forgot past board positions, making me think it's not capable of using the chat history to build a model of the game.

Re: What is ChatGPT doing and why does it work?

#512

Post by theGnuMe »

IanCal wrote: post #510
theGnuMe wrote: post #508

Earlier quoted context omitted.

Sorry by weights, I really meant the pattern of activations... I should have made that more clear. But the weights are trained by the game transcripts to produce activation patterns that could represent the board state. Or it could be local position patterns learnt during training. Positional representation (attention) of the N-1 tokens in the autoregressive task. Did they look at the attention patterns? Anyway there…

> . But the weights are trained by the game transcripts to produce activation patterns that could represent the board state A slight phrasing thing here just to be clear - the model is not trained to produce a representation of the board state explicitly. It is never given [moves] = [board state] and it is not trained on correctly predicting the board state by passing it in like [state] + move. The only thing that is…

What I am trying to say is that the game state is encoded as patterns in the attention matrices of the N-1 tokens. So yes, not explicitly trained to represent the game state but that game state is encoded in the tokens and their positions.

Re: What is ChatGPT doing and why does it work?

#513

Post by [deleted] »

spion wrote: post #15

The answer to this is: "we don't really know as its a very complex function automatically discovered by means of slow gradient descent, and we're still finding out" Here are some of the fun things we've found out so far: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (…

[deleted]

Re: What is ChatGPT doing and why does it work?

#514

Post by [deleted] »

spion wrote: post #15

The answer to this is: "we don't really know as its a very complex function automatically discovered by means of slow gradient descent, and we're still finding out" Here are some of the fun things we've found out so far: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (…

[deleted]

Re: What is ChatGPT doing and why does it work?

#515

Post by adammarples »

bottlepalm wrote: post #448
adammarples wrote: post #446

Earlier quoted context omitted.

By a statistical model I mean a very large, highly dimensional data manifold on which classification is done to predict the next token. The correct token can be predicted very often because somehow the common patterns in the written training data are being efficiently compressed in this highly dimensional space. Humans are also able to efficiently compress the data they are trained on, but the structures inside a hum…

Why do you think the wiring of a biological and artificial neural net are completely different? The results from both are remarkably similar.

We know from neuroscience that real neurons are very different from ANN neurons. For example, brains don't even use backpropagation. The neurons are connected differently. They have way more going on than just activation. Humans and ML models behave very differently.

Re: What is ChatGPT doing and why does it work?

#516

Post by tomduncalf »

redleggedfrog wrote: post #408
kilgnad wrote: post #129

Earlier quoted context omitted.

Finally I'm tired of people saying it's just a probabilistic word generator and downplaying everything as if they know. If you said something along these lines before... then these papers show that you're not fully grasping the situation here. There are clearly different angles of interpreting what these models are actually doing but people are stubbornly refusing to believe it's anything more then just statistical w…

Rather than subconscious fear I have conscious hope. I really hope that such models can be used for improving human life, something, much more really, than what appears to be the trivial uses of Chat-GPT. How about we feed it information on carbon sinks and helps slow down climate change? How do we get people to limit population voluntarily? How do we improve education? Better forest management techniques? I know I'm…

That’s actually a super interesting and positive way to look at this technology. I hope it can assist us in those sorts of ways. My personal hope is we can use this technology to enhance our lives and productivity, rather than it putting vast swathes of people out of work.

Re: What is ChatGPT doing and why does it work?

#517

Post by kimburgess »

spion wrote: post #15

The answer to this is: "we don't really know as its a very complex function automatically discovered by means of slow gradient descent, and we're still finding out" Here are some of the fun things we've found out so far: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (…

This internal tool making is something that Wolfram calls out in the essay:

> Yes, a neural net can certainly notice the kinds of regularities in the natural world that we might also readily notice with “unaided human thinking”. But if we want to work out things that are in the purview of mathematical or computational science the neural net isn’t going to be able to do it—unless it effectively “uses as a tool” an “ordinary” computational system.

Re: What is ChatGPT doing and why does it work?

#518

Post by famouswaffles »

tkgally wrote: post #482
famouswaffles wrote: post #450

Earlier quoted context omitted.

Hey you might this. Bilingual LLMs really are human level translators. I don't know why this frankly mindblowing fact isn't discussed or researched more but they are. https://github.com/ogkalu2/Human-parity-on-machine-translati...

Thanks for posting that. The results do look good. The examples are all short and from expository prose passages, though. Do you have any longer examples that include dialog, so the translator has to infer pronoun reference, the identities of speakers in conversations, and other narrative-dependent information? As I show in my video, that’s where ChatGPT is superior to Google Translate et al.—at least with Japanese t…

That's a good point. I was just kind of randomly plowing through so i didn't pick any dialogue scene specifically. Don't think it'll fail there though.
Post reply on HN