What is ChatGPT doing and why does it work?

Wow this is 19,000 words. I like his summary at the end:

At some level it’s a great example of the fundamental scientific fact that large numbers of simple computational elements can do remarkable and unexpected things.

And this:

... But it’s amazing how human-like the results are. And as I’ve discussed, this suggests something that’s at least scientifically very important: that human language (and the patterns of thinking behind it) are somehow simpler and more “law like” in their structure than we thought.

Yeah I've been thinking along these lines. ChatGPT is telling us something about language or thought, we just havent got to the bottom of what it is yet. Something along the lines of 'with enough data its easier to model than we expected'.

That Kenneth Li Othello paper is great. The accompanying blog post https://thegradient.pub/othello/ was discussed on HN here https://news.ycombinator.com/item?id=34474043

A lot of people didn't seem to get it when it was discussed on HN. A GPT had _only_ ever seen Othello transripts like: "E3, D3, C4 ..." and NOTHING else. It knows nothing of the board. It doesnt event know that there are two players. It learned Othello like it was a language, and was able to play an OK game of it, making legal moves 99.99% of the time. Inside its 'mind', by looking for correlations between its internal state and what they knew the 'board' would look like at each step in the games, they found 64 nodes that seemed to represent the 8x8 Othello board and representation of the two different colours of counters.

And this is the key bit: They reached into its mind and flipped bits on that internal representation (to change white pieces to black for example) and it responded in the appropriate way when making the next move. And by doing this they were able to map out its internal model in more detail, by running again and again with different variations of each move.

> Sincere question, because to me it feels like my brain is improvising word by word when typing out this sentence. I often delete and retype it until it feels right, but in the process of typing a single sentence, I'm just chaining words one after another the way they feel right.

That’s your conscious experience, but it doesn’t necessarily match what your subconscious mind has actually been doing. I’d hazard a guess that it’s thinking several moves ahead, like a good chess player. What you end up being consciously aware of is probably just the end product of multiple competing models - including ones that want to stop writing altogether and go do something else.

It was kinda long so I uhh fed it into chatgpt for a summary :

The author, Stephen Wolfram, describes the process of training ChatGPT using large amounts of text data, which allows the model to learn patterns and associations between words and phrases. He explains that ChatGPT uses a multi-layered approach to generate responses, starting with analyzing the input text and then generating a response based on the learned patterns.

Wolfram notes that ChatGPT's ability to generate human-like responses is due to the model's ability to capture context and incorporate knowledge from a wide range of sources. He also discusses the potential uses of ChatGPT, including as a tool for language translation, customer service, and educational purposes.

The article goes on to discuss some of the challenges and limitations of ChatGPT, such as its tendency to generate responses that are repetitive or irrelevant to the input text. Wolfram also acknowledges ethical concerns related to the use of AI for generating text, such as the potential for misinformation and the need for transparency in how the technology is used.

Overall, the article provides a detailed and informative overview of ChatGPT and its underlying technology, as well as the potential applications and challenges associated with AI-generated text.

I don’t generally think in individual words when writing. When I want to write a sentence, the first thing I think about is the idea that I want to convey. Then clusters of words start to come into my mind, as ways of expressing that idea, and I select and organize those fragments into a sentence, sometimes fine tuning words to avoid undesirable connotations or emphasize desired ones. Finally I glue the tuned fragments together with syntax and connecting words. Throughout the process I’m constantly iterating.

I don’t think that what ChatGPT does is anything remotely like what I do to communicate. But maybe I’m weird.