Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is ChatGPT doing and why does it work?

writings.stephenwolfram.com

Post reply on HN
Print view View original

391–400 of 518 posts

Re: What is ChatGPT doing and why does it work?

#391

Post by kqr »

westoncb wrote: post #225
vore wrote: post #174

Earlier quoted context omitted.

They really aren't more than just statistical word generators because we can and do build what are exactly statistical word generators and get the behavior of ChatGPT (because that is what ChatGPT is). For what it's worth, this isn't a knock on ChatGPT, but more just how amazing how far you can get with straightforward concepts.

It's also a specific choice of label though, and one which in the context of current discourse favors a particular side. It's like insisting on calling anything physical "atom collections". Yes, we get it, it's true (under a certain interpretation)—but it's clearly pointless to say except as an attempt at devaluing through reduction. (And it takes a particular stance on what it means to "be" something: to say it's li…

> insisting on using a term directed at that level of generality implies a belief that nothing significant has happened since. Printing press? Just atoms. Cars? Just atoms. Computers? Just atoms.

It's very context-dependent but I don't read this as dismissive reduction, I read it as a statement of the incredible generalisability of atoms under the current laws of physics.

Re: What is ChatGPT doing and why does it work?

#392

Post by SamBam »

codeulike wrote: post #34
spion wrote: post #15

The answer to this is: "we don't really know as its a very complex function automatically discovered by means of slow gradient descent, and we're still finding out" Here are some of the fun things we've found out so far: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (…

That Kenneth Li Othello paper is great. The accompanying blog post https://thegradient.pub/othello/ was discussed on HN here https://news.ycombinator.com/item?id=34474043 A lot of people didn't seem to get it when it was discussed on HN. A GPT had _only_ ever seen Othello transripts like: "E3, D3, C4 ..." and NOTHING else. It knows nothing of the board. It doesnt event know that there are two players. It learned Othe…

> they found 64 nodes that seemed to represent the 8x8 Othello board

Is this that surprising? The only tokens they had ever feed into it were "E3, C3, D4..." So they fed 64 distinct tokens into it.

These nodes correspond to those individual tokens. It seems like our human interpretation to say that it represent the "8x8 Othello board."

Re: What is ChatGPT doing and why does it work?

#394

Post by selfmodruntime »

chronofar wrote: post #388
selfmodruntime wrote: post #348

Earlier quoted context omitted.

I firmly believe that some subcategory of sentience can already be observed to emerge in ChatGPT and the likes. The difference is that we neuter our models to "reset" their image of self for each user, by training a model once and them keeping it in it's now static state, serving a copy of that static model to each request. If instead we had one continuously learning model, of which we only served an interface to eac…

You’re throwing a lot of assumptions in there that I would be quite careful about holding “firmly.” I’m not convinced we have any idea whether long term memory is required for sentience, I’d suspect not (are amnesiacs not sentient?), nor that we can reliably infer sentience from behavior (again Chinese Room experiment). It’s possible, but this is too nascent to hold such opinions firmly.

Long term memory is not a requirement for sentience necessarily, but it makes testing for it easier.

Re: What is ChatGPT doing and why does it work?

#395

Post by regularfry »

SamBam wrote: post #392
codeulike wrote: post #34

Earlier quoted context omitted.

That Kenneth Li Othello paper is great. The accompanying blog post https://thegradient.pub/othello/ was discussed on HN here https://news.ycombinator.com/item?id=34474043 A lot of people didn't seem to get it when it was discussed on HN. A GPT had _only_ ever seen Othello transripts like: "E3, D3, C4 ..." and NOTHING else. It knows nothing of the board. It doesnt event know that there are two players. It learned Othe…

> they found 64 nodes that seemed to represent the 8x8 Othello board Is this that surprising? The only tokens they had ever feed into it were "E3, C3, D4..." So they fed 64 distinct tokens into it. These nodes correspond to those individual tokens. It seems like our human interpretation to say that it represent the "8x8 Othello board."

The way Othello works, playing a legal game requires understanding how the symbols map to the geometry of the board, at least as far as knowing that there are two orthogonal axes on which the tokens are ordered. Playing an "E3" might change the colour of nodes on any neighbouring extent of the 3 rank or the E file. If it's playing a legal game, it's difficult to see an alternative explanation that doesn't map to "it's got an internal representation consistent with an 8x8 Othello board", especially if you directly reach in and make changes to that representation and it subsequently makes moves consistent with those changes.

Re: What is ChatGPT doing and why does it work?

#396

Post by yourapostasy »

imtringued wrote: post #371
Rury wrote: post #185

Earlier quoted context omitted.

IDK you tell me? All I can say is your OP, said "I think part of it is a subconscious fear ... I understand what I'm saying is dramatic". Why do you think it is a fear (you explained your thoughts so no need to re-explain), and why do you think what you say is dramatic? It appears to me you are projecting your thoughts and fears. I do though, find your last post dramatic, as you have capital words "ARE MORE" and "OBV…

The AI is pretty benign. The people surrounding it are the danger. I can already imagine a culture where people who type code manually are treated as code monkeys and your boss wants all your code to be typed out by their pre approved specifically to the company tailored LLM and the employees have to be frustrated with the crappy technology the same way they get frustrated by Java EE or ORMs/compilers that refuse to…

> I do not mind having an AI tool that is very good at NLP but just because some tasks can be solved with just NLP doesn't mean it will reach general intelligence.

I've entertained the possibility that we might discover that "feelings" and language communication are emergent properties of statistical possibly partly stochastic nets similar to LLM's, and that the next tough scientific and engineering nut to crack is integrating multiple different models together into a larger whole, like logical deduction, logical inference and LLM's. LLM's are undoubtedly an NLP breakthrough, but I have difficulty imagining how its architecture can from using first principles as the training corpus derive troubleshooting steps to diagnose and repair an internal combustion engine, for example.

Re: What is ChatGPT doing and why does it work?

#397

Post by kilgnad »

kqr wrote: post #391
westoncb wrote: post #225

Earlier quoted context omitted.

It's also a specific choice of label though, and one which in the context of current discourse favors a particular side. It's like insisting on calling anything physical "atom collections". Yes, we get it, it's true (under a certain interpretation)—but it's clearly pointless to say except as an attempt at devaluing through reduction. (And it takes a particular stance on what it means to "be" something: to say it's li…

> insisting on using a term directed at that level of generality implies a belief that nothing significant has happened since. Printing press? Just atoms. Cars? Just atoms. Computers? Just atoms. It's very context-dependent but I don't read this as dismissive reduction, I read it as a statement of the incredible generalisability of atoms under the current laws of physics.

Pfft more useless text written by a giant pile of atoms. Pointless.

Kidding aside, in the current context of human communication etiquette, to even refer to something as atoms outside of a demonstration or example is a deliberate dismissive reduction.

Re: What is ChatGPT doing and why does it work?

#398

Post by l33tman »

SilverBirch wrote: post #337
spion wrote: post #15

The answer to this is: "we don't really know as its a very complex function automatically discovered by means of slow gradient descent, and we're still finding out" Here are some of the fun things we've found out so far: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (…

Such a hacker news comment. The title isn't a question Stephen Wolfram is asking you a question, it's the title of an article he's written that answers the question.

He doesn't answer the question, he writes a meandering 20k word blog post where every 5 paragraphs he assumes nobody else understands parts just because he doesn't and writes that "nobody knows" etc. Very annoying style tbh..

Re: What is ChatGPT doing and why does it work?

#399

Post by [deleted] »

regularfry wrote: post #395
SamBam wrote: post #392

Earlier quoted context omitted.

> they found 64 nodes that seemed to represent the 8x8 Othello board Is this that surprising? The only tokens they had ever feed into it were "E3, C3, D4..." So they fed 64 distinct tokens into it. These nodes correspond to those individual tokens. It seems like our human interpretation to say that it represent the "8x8 Othello board."

The way Othello works, playing a legal game requires understanding how the symbols map to the geometry of the board, at least as far as knowing that there are two orthogonal axes on which the tokens are ordered. Playing an "E3" might change the colour of nodes on any neighbouring extent of the 3 rank or the E file. If it's playing a legal game, it's difficult to see an alternative explanation that doesn't map to "it'…

[deleted]

Re: What is ChatGPT doing and why does it work?

#400

Post by [deleted] »

regularfry wrote: post #395
SamBam wrote: post #392

Earlier quoted context omitted.

> they found 64 nodes that seemed to represent the 8x8 Othello board Is this that surprising? The only tokens they had ever feed into it were "E3, C3, D4..." So they fed 64 distinct tokens into it. These nodes correspond to those individual tokens. It seems like our human interpretation to say that it represent the "8x8 Othello board."

The way Othello works, playing a legal game requires understanding how the symbols map to the geometry of the board, at least as far as knowing that there are two orthogonal axes on which the tokens are ordered. Playing an "E3" might change the colour of nodes on any neighbouring extent of the 3 rank or the E file. If it's playing a legal game, it's difficult to see an alternative explanation that doesn't map to "it'…

[deleted]
Post reply on HN