Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is ChatGPT doing and why does it work?

writings.stephenwolfram.com

Post reply on HN
Print view View original

411–420 of 518 posts

Re: What is ChatGPT doing and why does it work?

#411

Post by beepbooptheory »

kilgnad wrote: post #406
beepbooptheory wrote: post #365

Earlier quoted context omitted.

Its ok for you to be enthusiastic! Its natural to want to witness the kind of thing you are saying. I think you are maybe operating out of unconscious desperation though; a loneliness that is really driving this for you, akin to hoping for a nearby earth-like planet. There isn't a lot else in our world that has this seemingly pure and transcendent promise . It allows you to be brave and accepting about something wher…

Uh. No. I don't. I'm not enthusiastic. I don't want ai to take over my job. I don't want any of this to happen. I'm also not fighting people. Just disagreeing. Big difference.

Just remember to breath. Try to spend some time every once in a while stepping back, listening to experts, and reviewing your assumptions.

The sense of urgency or passion you feel is mostly just coming from the way we are crowdsourced to hype things up for a profit-seeking market. A year from now you will undoubtedly feel silly feeling and saying the things you are now, trust me. It's more just the way discourse and social media work--it makes you feel like there is a crusade worthy of your time every other day, but its always a trick.

No worries, we have all been there!

Re: What is ChatGPT doing and why does it work?

#412

Post by bottlepalm »

bikamonki wrote: post #390
bottlepalm wrote: post #96

The easiest way for ChatGPT to generate good output is to plain understand it. Given the vast amount of input data fed into it, it has no choice, but to start reducing the input into fundamental rules which is basically what understanding is. Understanding is a form of compression. More efficient for a neural network to understand a concept than memorize permutations. Same with statistics and markov chains, people fo…

Except ChatGPT "is not aware" that it understands.

At what point of the machine telling you it is aware do you believe that it is actually aware?

The building blocks are the same - neural nets. The language outputs are the same enough to fool a human.

So what’s that elusive secret sauce that makes you ‘aware’ and other things not?

Re: What is ChatGPT doing and why does it work?

#413

Post by thfuran »

Barrin92 wrote: post #81
jonahx wrote: post #69

Earlier quoted context omitted.

> has little to do with the quality of the output. Non-snarky question: What else can you judge by? Isn't any alternative just putting more precise conditions on the output? With ChatGPT, it's still easy enough to see it's mistakes, and its attempts at fiction and poetry, impressive as they are, are still clumsy to a trained eye, relative to expert human work. But what if they weren't? What happens when they're indis…

>Non-snarky question: What else can you judge by? its architecure. A child is a living and autonomous agent. It has (or develops) meta cognition, an awareness of its own mental state (and by extension use of language). These models don't have the capacity to do this even in theory given that they're static and pretrained. When you ask ChatGPT what it feels like to speak, there isn't some neural activity within the mo…

The essay is the words that comprise it, not the means by which they were produced. I don't need to write it out by hand for it to be real. My printer can produce a real essay. A photocopier can produce a real essay. ChatGPT can produce a real essay, but it's just probably going to be a bit shit.

Re: What is ChatGPT doing and why does it work?

#414

Post by codeulike »

notahacker wrote: post #389
codeulike wrote: post #353

Earlier quoted context omitted.

Its not suprising, but it answers the question "Do Large Language Models learn world models or just surface statistics?" - OthelloGTP is not using some weird trick to come up with the next move "G4". You can imagine some sort of shortcut trick where you say "use a letter thats near the middle of the bell curve of letters you've seen so far, and a number thats a bit to the left of the bell curve" or something. Its not…

But the point is that Othello notation is basically 64 tokens which map 1:1 to positions on an Othello board, and the "grammar" of whether one token is a valid continuation is basically how the previous sequence of moves updates game state, so surface statistics absolutely do lead inexorably towards a representation of the game board. Whether a move is a suitable continuation or not absolutely is a matter of probabil…

Sure, yes. They deliberately chose Othello because its a simple universe and they would know what 'model' to go delving for in the internals.

Re: What is ChatGPT doing and why does it work?

#415

Post by bottlepalm »

adammarples wrote: post #354
thomashop wrote: post #105

Earlier quoted context omitted.

Totally agree. I think Juergen Schmidthuber has developed a lot of ideas around compression being the basis for consciousness and understanding. There was the paper that showed that when showing a language model Othello moves it ends up building an internal representation of the board. And now I was reading this abstract: ```Theory of mind (ToM), or the ability to impute unobservable mental states to others, is centr…

Language is how people demonstrate that they have a good theory of mind. Solving these tasks is a proxy for people's internal theories. Language is a good way to demonstrate that ChatGPT has a good statistical model for what token comes next. Humans are not good at solving TOM tasks because they're good at guessing what token comes next, ChatGPT is not good at solving them because is has a good TOM. They're different…

What ‘statistical model’? ChatGPT isn’t a big Markov chain, it’s a neural net. It was trained on ‘full input’ transformed to ‘full output’. Generating the ‘next word’ is implementation details as it already knows the bigger idea it is trying to explain.

Re: What is ChatGPT doing and why does it work?

#416

Post by newswasboring »

GuB-42 wrote: post #360
rcfox wrote: post #317

Earlier quoted context omitted.

What does "temperature" mean here though? Are you sure you didn't just ask it to generate two different responses?

That's the amount of randomness in the answers. Low temperature means it will take the most common path every time, at the risk of paraphrasing its sources. The "zero temperature" answer may very well been copied verbatim from a mainstream website. High temperatures means the system will get fed a lot of noise to create something original, at the risk of getting off rails or simply wrong.

did you just write this in the prompt? And ChatGPT understood this? Fascinating. It parameterized itself.

Re: What is ChatGPT doing and why does it work?

#417

Post by spion »

bmacho wrote: post #355
SilverBirch wrote: post #337

Earlier quoted context omitted.

Such a hacker news comment. The title isn't a question Stephen Wolfram is asking you a question, it's the title of an article he's written that answers the question.

Rather a typical human comment to spread their own believes when they have a chance. An ideal hacker news comment would be the exact opposite, it would refer to the article.

The article itself admits (after covering the basics) that we don't really know. I thought it would be interesting to post some additional cool papers written by folks trying to figure out what's going on.

Re: What is ChatGPT doing and why does it work?

#418

Post by cm2012 »

lelanthran wrote: post #219
cm2012 wrote: post #188

Earlier quoted context omitted.

For now. Give it a truly persistent memory and 100x the size of the dataset I think most people would change their tune.

> For now. Give it a truly persistent memory and 100x the size of the dataset I think most people would change their tune. Why does it need 100x the dataset? Sentient creatures, including humans, manage to figure stuff out from as little as a single datapoint . For a human to differentiate between a cat and a dog takes, maybe, two examples of each, not a few million pictures. An adult human who sees a hotdog for the…

Human dataset comes from evolution. We evolved from millions of years of life and death and our genetic memory is basically one long long memoried computer.

Re: What is ChatGPT doing and why does it work?

#419

Post by rendall »

kilgnad wrote: post #129
spion wrote: post #15

The answer to this is: "we don't really know as its a very complex function automatically discovered by means of slow gradient descent, and we're still finding out" Here are some of the fun things we've found out so far: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (…

Finally I'm tired of people saying it's just a probabilistic word generator and downplaying everything as if they know. If you said something along these lines before... then these papers show that you're not fully grasping the situation here. There are clearly different angles of interpreting what these models are actually doing but people are stubbornly refusing to believe it's anything more then just statistical w…

People are dismissive because formerly unassailable boundaries between human and other are becoming less clear. If it's so easy to simulate sentience, what's to say we are not ourselves simulating sentience?

Re: What is ChatGPT doing and why does it work?

#420

Post by chronofar »

selfmodruntime wrote: post #394
chronofar wrote: post #388

Earlier quoted context omitted.

You’re throwing a lot of assumptions in there that I would be quite careful about holding “firmly.” I’m not convinced we have any idea whether long term memory is required for sentience, I’d suspect not (are amnesiacs not sentient?), nor that we can reliably infer sentience from behavior (again Chinese Room experiment). It’s possible, but this is too nascent to hold such opinions firmly.

Long term memory is not a requirement for sentience necessarily, but it makes testing for it easier.

Only if you presume systems that display one must display the other, which I think is far from a given in systems that aren't comprised of neurons, or in other words systems that don't look/work like us (which is the only pathway we have to believing the two are linked in any way).
Post reply on HN