Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simply explained: How does GPT work?

confusedbit.dev

Post reply on HN
Print view View original

261–270 of 392 posts

Re: Simply explained: How does GPT work?

#261

Post by opportune »

iliane5 wrote: post #136
i-use-nixos-btw wrote: post #130

I’d be interested in hearing from anyone who takes the Chinese Room scenario seriously, or at least can see how it applies to any of this. I cannot see that it matters if a computer understands something. If it quacks like a duck and walks like a duck, and your only need is for it to quack and walk like a duck, then it doesn’t matter if it’s actually a duck or not for all intents and purposes. It only matters if you…

What I find really entertaining is the "just predicting the next token" argument. If just predicting the next token can produce similar or better results than the almighty human intelligence on some tasks, then maybe there's a bit of hubris in how smart we think we actually are.

I don’t think there’s anything making it impossible for actual intelligence to arise from a task as simple as “predicting the next token (to model human thought/speech/writing)” because with enough compute resources, smart AI implementations, and training that task basically would be optimized by becoming a general intelligence.

But it’s clear based on current implementations that once you work backwards from the knowledge that it’s “just predicting the next token” you can easily find situations in which the AI doesn’t demonstrate general intelligence. This is most obvious when it comes to math, but it’s also apparent in hallucinations and the model not being able to reason through/synthesize ideas very well, deviate from the script (instead of just answering a question with what it has already, in some cases it should not even try to answer and instead ask more clarifying questions). To be fair, there are plenty of humans with excellent writing or speaking skills that are bad at that kind of stuff too.

Re: Simply explained: How does GPT work?

#262

Post by int_19h »

ZeroGravitas wrote: post #104

> It is able to link ideas logically, defend them, adapt to the context, roleplay, and (especially the latest GPT-4) avoid contradicting itself. Isn't this just responding to the context provided? Like if I say "Write a Limerick about cats eating rats" isn't it just generating words that will come after that context, and correctly guessing that they'll rhyme in a certain way? It's really cool that it can generate coh…

"Responding to the context provided" is very vague. I could argue that I'm doing exactly that right now as I'm writing this comment. It does not imply not being able to e.g. link ideas logically.

With respect to interrogating GPT if it does something wrong - the reason why people do it is because it works. With GPT-4 especially, you can often ask it to analyze its own response for correctness, and it will find the errors without you explicitly pointing them out. You can even ask it to write a new prompt for itself that would minimize the probability of such errors in the future.

Re: Simply explained: How does GPT work?

#263

Post by fl7305 »

brazzy wrote: post #203
tines wrote: post #148

Earlier quoted context omitted.

> Sometimes for GPT to "just" complete the next word in a way that humans find plausible, it must, along the way, develop a model of the world, theory of mind, abstract reasoning. etc. I did an experiment recently where I asked ChatGPT to "tell me an idea [you] have never heard before". ChatGPT replied with what sounded like an idea for a startup, which was delivering farm-fresh vegetables to customers' doors. This i…

Almost all people almost never have truly original ideas. When asked to "tell me an idea [you] have never heard before", they will remix stuff they have heard to get something that "feels" like it's new. In some cases they'll actually be wrong and reproduce something they heard and forgot about hearing, but remember the concept. Most of the time, the remix will be fairly superficial. And remixing stuff it has heard b…

> And remixing stuff it has heard before is exactly what ChatGPT is doing.

Check out my "the confetti has left the cannon" example above.

https://news.ycombinator.com/item?id=35346683

Maybe still "remixing", but it sure feels like new to me.

Re: Simply explained: How does GPT work?

#264

Post by rimliu »

Analog24 wrote: post #193
rimliu wrote: post #97

Earlier quoted context omitted.

Having a world model

If you read some of the studies of these new LLMs you'll find pretty compelling evidence that they do have a world model. They still get things wrong but they can also correctly identify relationships and real world concepts with startling accuracy.

No, they don't. They fail at the arithmetics ffs.

Re: Simply explained: How does GPT work?

#265

Post by pegasus »

int_19h wrote: post #254
pegasus wrote: post #240

Earlier quoted context omitted.

I would agree that much more than we're usually ready to admit to ourselves is second-hand, but saying everything is going too far. Inventions and discoveries are happening all the time, at all scales.

Inventions and discoveries are basically a very advanced form of pattern recognition - seeing "interesting" regularities in some particular input where no-one has spotted them before. And LLMs are capable of making discoveries in this sense, if you feed them data and ask to spot the regularities. They're not particularly good at it, but that's a different question from whether they're able to do it in principle.

Yes, in as far as LLMs can be said to make inventions and discoveries, this is clearly how they do it. And yes, these type of processes definitely play a big part in our human creative capacity. But to say this is all there is to it, is going too far in my opinion. We just don't know. There's still so much we don't understand about ourselves. We haven't designed ourselves after all, we just happened to "come to" one bright primeval day and since then we've been exploring and discovering ourselves. And again and again, we've found ourselves in that seductive spot where we thought we "got it" and there's nothing essential about ourselves we're still missing, only again and again to be proven wrong. Dogmas crumble, new dogmas coagulate, only to dissolve again. Or, we could use the more dignified word "paradigm".

Re: Simply explained: How does GPT work?

#266

Post by alkonaut »

nicpottier wrote: post #65
alkonaut wrote: post #41

What I wonder most is how it encodes knowledge/state other than in the sequence of queries/responses. Does it not have a "mind"? If I play a number guessing game, can I tell it to "think of a number between 0 and 100" and then tell me if the secret number is higher/lower than my guess (For a sequence of N guesses where it can concistently remember it's original number)? If not, why? Because it doesn't have context? I…

I thought your "guessing game" question was an interesting one so tried it on GPT-4. In my first attempt I played logically and it did fine and I finally guessed correct. On my second I made suboptimal guesses and it didn't stay consistent. The thing to remember is that GPT has no state apart from the context, so it can't "remember" anything apart from what's in the text. That doesn't mean it shouldn't be able to sta…

Shouldn’t it be a reasonable (and pretty simple) addition to just have a secret scratchpad - an inner monologue - where the bot is free to add context which is not “published”?

Re: Simply explained: How does GPT work?

#267

Post by int_19h »

seydor wrote: post #17

This is confusing, using the semantic vectors arithmetic of embeddings is not very relevant to transformers and its completely missing the word 'attention'. I don't think transformers are that difficult to explain to people , but it is hard to explain "why" they work. But i think it's important for everyone to look under the hood and know that there are no demons underneath.

The whole thing is magic, really. We assemble relatively mundane things that we understand together in certain specific ways, and somehow get something that can talk etc. A conjurer of old - who would similarly assemble bells and candles - would probably say that we summoned a demon.

Re: Simply explained: How does GPT work?

#268

Post by eternalban »

i-use-nixos-btw wrote: post #130

I’d be interested in hearing from anyone who takes the Chinese Room scenario seriously, or at least can see how it applies to any of this. I cannot see that it matters if a computer understands something. If it quacks like a duck and walks like a duck, and your only need is for it to quack and walk like a duck, then it doesn’t matter if it’s actually a duck or not for all intents and purposes. It only matters if you…

> I cannot see that it matters if a computer understands something.

I am not sure they can, but the difference is profound and material. A machine that actually understands, like a human being, is not going to be (can not be) entirely truthful or transparent. There will be private inner thoughts, idea formation, and possibly even willful intent, as a direct consequence of understanding. And the nature of interactions, regardless of superficial similarity, shifts from one of utility to relationship. For example, we would care to know if e.g. the systems entrusted with apocalyptic deterent forces are mechanisms or organisms.

Please note that not a single one of us has ever interacted with any intelligent life form lacking a sense of self, or an ego. Thus, all our sensory registers of another 'intelligent being' are learned in a context of the implicit 'this other is like me'. We are not equipped to distinguish or articulate intelligence (in the abstract) merely based on sensory information. Note that even non-verbal communication, such as jabbing a friend in the ribs, are all learned to have a certain meaning in that very same context of implicits, and any mechanism that mimicks them (via training) will be afforded the same projection of the implicit. I do not believe there is, in fact, any non-destructive test of determining 'consciousness' in an entity. (Destructive, since there may be long running tests of a subject than can be shown to be probably accurate, possibly via creating situational problems involving survival, and unexpected circumstances.)

Ask yourself what is it that convinces you that the last person you spoke with (in real life) was actually conscious? I assert that the entire matter is a 'fictional certainty' based on assumption of shared nature. "They are conscious because I am".

Re: Simply explained: How does GPT work?

#269

Post by JieJie »

bulkprotocol wrote: post #184
JieJie wrote: post #132

Earlier quoted context omitted.

I literally lost a friend of thirty years yesterday because she is wedded to the Chinese Room analogy so fiercely, she refuses to engage on the subject at all. For all the terrible things people worry about ChatGPT doing, this was not one that I thought I was going to have to deal with. (edit: ChatGPT was not involved at all, but when I suggested she give it a try to see for herself, that was the end of it.)

You blew up a 30 year friendship over an...analogy?

I didn't! Someone else did it to me. I was trying desperately not to.

(edit: This is the kind of stuff I think my friends are watching and being informed by [0] as it was what they are posting in our common areas.)

[0]: https://youtu.be/ro130m-f_yk

Re: Simply explained: How does GPT work?

#270

Post by int_19h »

simonw wrote: post #12
onetrickwolf wrote: post #7

I've been using GPT4 to code and these explanations are somewhat unsatisfactory. I have seen it seemingly come up with novel solutions in a way that I can't describe in any other way than it is thinking. It's really difficult for me to imagine how such a seemingly simple predictive algorithm could lead to such complex solutions. I'm not sure even the people building these models really grasp it either.

I've started to suspect that generating code is actually one of the easier things for a predictive text completion model to achieve. Programming languages are a whole lot more structured and predictable than human language. In JavaScript the only token that ever comes after "if " is "(" for example.

One thing to bear in mind is that GPT training set for code is supposedly skewed very heavily towards Python.
Post reply on HN