Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding ChatGPT

atmosera.com

Post reply on HN
Print view View original

221–230 of 241 posts

Re: Understanding ChatGPT

#221

Post by MacsHeadroom »

freehorse wrote: post #84
maxdoop wrote: post #67

Earlier quoted context omitted.

Then this implies that you’d maybe think differently if LLMs could have different inputs, correct? Which they are currently doing. GPT-4 can take visual input. I totally agree that humans are far more complex than that, but just extend your timeline further and you’ll start to see how the gap in complexity / input variety will narrow.

> Then this implies that you’d maybe think differently if LLMs could have different inputs, correct? They will not be LLMs then, though. But some other iteration of AI. Interfacing current LLMs with APIs does not solve the fundamental issue, as it is still just language they are based on and use.

>They will not be LLMs then, though.

Multi-modal LLMs are still called LLMs because they don't "interface with APIs" to add visual, audio, touch, etc input and output. They just encode pictures, sounds, and motor senses using the same tokens they encode text with and then feed it to the same unmodified LLM and it learns to handle those types of data just fine.

There are no APIs involved and the model is unchanged. It was designed as an LLM, the design hasn't changed, it still is an LLM, it's just had data fed to it that it can't tell from text and is running the same exact LLM inference process on it.

I can download any open source LLM right now and fine tune it on images faster than I could train an ImageNet from scratch because of something called transfer learning. Humans transfer learned speech after millions of generations of using other senses. That's not at all surprising or different from the way LLMs work.

Re: Understanding ChatGPT

#222

Post by MacsHeadroom »

Kuinox wrote: post #53
rvnx wrote: post #43

Earlier quoted context omitted.

It totally can (try it if you don't believe it). In your message you say it is gibberish, but I have completely different results and get very good Base64 on super long and random strings. I frequently use Base64 (both ways) to bypass filters in both GPT-3 and 4/Bing so I'm sure it works ;) It sometimes make very small mistakes but overall amazing. At this stage if it can work on random data that never appeared in th…

Did you tried longer sentence and not singles words? Did you also read the conversation? Edit: ok it looks like it can now convert in base64, I'm sure it couldn't when I tested 2 months ago.

It could when I tested 1 week after the first version of chatGPT was in private beta. It's always been able to convert base64 both ways.

It sometimes gets some of the conversion wrong or converts a related word instead of the word you actually asked it to convert. This strongly suggests that it's the actual LLM doing the conversion (and there's no reason to believe it wouldn't be).

This behavior will likely be replicated in open source LLMs soon.

Re: Understanding ChatGPT

#223

Post by notahacker »

teolandon wrote: post #211
notahacker wrote: post #117

Earlier quoted context omitted.

Sure, and I've been deeply affected by books, but I'm not going to start using that as a basis for an argument a book and a human think in exactly the same way

This was in response to your comment about how you can tell that a water simulation is fake by trying to dip a water bottle in there. The distinction between chemical and silicon doesn't matter when the output is text. There's no physical test you can perform in the text, like dipping a water bottle in water, to see if it's chemical or silicon.

If you test both on the same terms (i.e. only interaction via a remote terminal) then a decent simulation can entirely convince humans that a bottle has been dipped in it and water removed from it too. But it still doesn't have the essential properties of H20, it just looks like it in many ways to some imperfect observers.

Testing is a moot point when my original argument was that it there is no reason to assume that a converts-to-ASCII subset of i/o as it is perceived by a [remote] human observer other is the only differences between two dissimilar physical processes (one of which we know results in sensory experiences, self awareness etc). Takes a lot more belief that the human mind is special to believe that sensory experience etc resides not in physics but whether human observation deduces the entity has sensory experience.

Re: Understanding ChatGPT

#224

Post by mellosouls »

maxdoop wrote: post #216
mellosouls wrote: post #122

Earlier quoted context omitted.

Do tell— how can you prove humans are any different? In this (and other comments by you I think?) you've implied the onus is on the AGI sceptics to prove to you that the LLM is not sentient (or whatever word you want to describe motive force, intent, consciousness, etc that we associate with human intelligence). This is an unreasonable request - it is on you to show that it is so. I’m forever amazed how a seemingly f…

There’s a misunderstanding here— I’m not claiming LLMs are sentient. I’m not claiming they are even similar. What I am pushing back against is the confidence with which people so blatantly claim we are dissimilar. It’s an important distinction, and I’ve yet to see solid evidence to suggest it’s a point we shouldn’t even explore. What I see so often is comments stating things like, “an LLM is just pattern matching” or…

Thanks for the reply; I also saw your other reply to a similar comment after I'd posted mine and your position seems more reasonable than I first understood it.

Re: Understanding ChatGPT

#225

Post by YeGoblynQueenne »

swid wrote: post #98
maxdoop wrote: post #28

“It’s a glorified word predictor” is becoming increasingly maddening to read. Do tell— how can you prove humans are any different? The most common “proofs” I’ve seen: “Humans are more complex”. Ok, so you’re implying we add more complexity (maybe more modalities?); if more complexity is added, will you continue to say “LLMs are just word predictors”? “Humans are actually reasoning. LLMs are not.” Again, how would you…

One reason I hate the “glorified word predictor” phrase, is that predicting the next word involves considering what will come well after that. I saw a research paper where they tested a LLM to predict the word “a” vs “an”. In order to do that, it seems like you need to consider at least 1 word past the next token. The best test for this was: I climbed the pear tree and picked a pear. I climbed the apple tree and pick…

>> I cannot rectify this with people saying it only looks one word ahead. One word must come next, but to do a good job modeling what that word will be, wouldn’t you need to consider further ahead than that?

No, because you don't predict the probability of a token, you predict the probability of a token _given_ a preceding sequence of tokens.

So, to decide whether to follow "I climbed the apple tree and picked" with "a" or "an", you calculate the following (which are conditional probabilities; read p(A|B) as "probability of A given B):

  P₁ = p(a|I,climbed,the,apple,tree,and,picked)
  P₂ = p(an|I,clibed,the,apple,tree,and,picked)
Now, if P₁ > P₂, you generate "a", otherwise you generate "an".

Note that the sentence "I climbed the apple tree and picked" is different than the sentence "I climbed the pear tree and picked" so the following are different probabilities, also:

  P₃ = p(a|I,climbed,the,pear,tree,and,picked)
  P₄ = p(an|I,clibed,the,pear,tree,and,picked)
So you'll get a different token generated depending on whether P₃ > P₄ or not.

In short, yeah, you can choose whether it's going to be "a" or "an" according to the context of the sentence so-far.

But even in cases where you can't do that, when you don't have the context of "pear" or "apple", what you _can_ do is generate this string:

"I climbed the pear tree and picked a"

And then calculate the following probabilities:

  P₅ = p(ball|I,climbed,the,tree,and,picked,a)
  P₆ = p(abacus|I,clibed,the,tree,and,picked,a)
And if P₅ > P₆ you'll generate "ball", otherwise generate "abacus". So you don't have to know what's coming next, just what you've generated so-far.

Does that help?

Re: Understanding ChatGPT

#226

Post by freehorse »

maxdoop wrote: post #106
freehorse wrote: post #86

Earlier quoted context omitted.

The comment answers to how GPT supposedly produces "new interesting concepts and ideas". The comment you are answering is not supposed or perceived to present anything novel. So both your argument and the comment you reply to are technically correct, neither produces any novel idea; only difference is your reply is out of place.

What exactly is a novel idea? Is there any true “novelty” that just suddenly arises in a human brain without having prerequisite experience / knowledge?

I mean, indeed most "novel ideas" are about making novel connections between existing concepts, and more rarely about creating novel concepts. However, bunching together concepts, even if in a novel way, does not necessarily constitute "novel idea" as the connections have to be in some ways meaningful and interesting, in a certain context. In the previous case, which is actually quite typical of LLMs, a "novel idea" was produced by bunching together certain terms, in a way that it looks syntactically correct and even conceptually solid in a superficial manner. But from a scientific point of view, it is non-sensical and, basically, junk. Bunching together concepts as words does not make a scientific idea or theory. This is a complex process in science, that involves different scales of argumentation, none of which is provided by LLMs. But this is as far as merely manipulating language can get one.

Re: Understanding ChatGPT

#227

Post by joshuahedlund »

PartiallyTyped wrote: post #207
joshuahedlund wrote: post #200

Earlier quoted context omitted.

Fair points. Thanks for letting me refine and discard hypotheses. While I think about those, how about this? Language is not a complete representation of thinking. We use language to describe symbols, not even very precisely, and we can convert imprecise language to more precise symbols in our brain, manipulate them as symbols, and only then turn them back into language. That’s why you often cannot perfectly translat…

> We use language to describe symbols, not even very precisely, and we can convert imprecise language to more precise symbols in our brain, manipulate them as symbols, and only then turn them back into language. I don't think that's true for all people. I know that some people manipulate words in their heads, others images, I manipulate sounds and images. Language is just a noisy medium through which we communicate t…

> I know that some people manipulate words in their heads, others images, I manipulate sounds and images. Language is just a noisy medium through which we communicate the internal state of our brain or its outputs to other people / humans and ourselves.

We are in agreement here. I think you are only strengthening my argument that language is too imprecise and restrictive for LLMs to be fundamentally equivalent to human thinking.

> GPT4 does just fine in some cases and extrapolates just fine in others

"just fine" is not very persuasive here. A nine-year-old can do much better than "just fine" after learning some very simple rules with very minimal examples. And I conjecture that if you removed a lot of the mathematical examples from GTP's training corpus, to be more equivalent to what a nine-year-old has seen, it would do even worse. And it's fundamentally because it cannot break out of its linear language limits and understand numbers as abstract symbols that can be manipulated before converting back into language.

> I don't think that is correct. The human might provide a justification for something but that doesn't mean it is the true reason they reached a conclusion.

Sometimes, yes. What I meant here is that a human can (sometimes - when it is acting intelligently) specifically repeat information they learned and specifically recall the source of that information. This is how our entire scientific process works; we can go back and look up exactly how we derived any piece of our collective knowledge, verify or repeat it if necessary, or build on it further. You're proving my point by the citations you are giving me! (thank you for them) As an intelligent human, you do not "hallucinate" sources, you can provide real ones and provide them directly.

And you can do this because - to go back to my original argument - your intelligence is fundamentally different than an LLM. (That's not an argument that AI is impossible, only that we work differently somehow than what we've seen so far.)

Re: Understanding ChatGPT

#228

Post by wrp »

visarga wrote: post #10
wrp wrote: post #3

Earlier quoted context omitted.

By now, everyone has heard the explanation that ChatGPT is a transformer encoder-decoder that responds to prompts by iteratively predicting the first word in the response, then the second word, and so on... What we need now is explanation of all the further stuff added to that basic capability.

The pre-trained model is stage 1 - it has seen everything, but it is wild. If you ask it "What is the capital of US?" it will reply "What is the capital of Canada?"... Stage 2 is task solving practice. We use 1000-2000 supervised datasets, formatted as prompt-input-output texts. They could be anything: translation, sentiment classification, question answering, etc. We also include prompt-code pairs. This teaches the…

Regarding Stage 2. Are you saying that ChatGPT's facility to recognize and process commands is derived entirely from training on supervised datasets and not hand-crafted logic? Can you point me to any reading on this?

Re: Understanding ChatGPT

#229

Post by wrp »

HarHarVeryFunny wrote: post #66
Bishonen88 wrote: post #2

This articles describes much of what many youtubers explained in their videos in the recent few weeks. While I understand the core concept of 'just' picking the next word based on statistics, it doesn't really explain how chatGPT can pull off the stuff it does. E.g. when one asks it to return a poem where each word starts with one letter/next alphabet letter/the ending of the last word, it obviously doesn't 'just' pi…

> While I understand the core concept of 'just' picking the next word based on statistics That's just the mechanism it uses to generate output - which it not the same as being the way it internally chooses what to say. I think it's unfortunate that the name LLM (large language model) has stuck for these predictive models, since IMO it's very misleading. The name has stuck since this line of research was born out of m…

> ...the input is consumed and per the internal "world model" a high level internal representation of the input is built...

This is the aspect of ChatGPT I'm trying to understand. Can you point to any resources on this?

Re: Understanding ChatGPT

#230

Post by HarHarVeryFunny »

wrp wrote: post #229
HarHarVeryFunny wrote: post #66

Earlier quoted context omitted.

> While I understand the core concept of 'just' picking the next word based on statistics That's just the mechanism it uses to generate output - which it not the same as being the way it internally chooses what to say. I think it's unfortunate that the name LLM (large language model) has stuck for these predictive models, since IMO it's very misleading. The name has stuck since this line of research was born out of m…

> ...the input is consumed and per the internal "world model" a high level internal representation of the input is built... This is the aspect of ChatGPT I'm trying to understand. Can you point to any resources on this?

No - I'm not sure anyone outside of OpenAI knows, and maybe they only have a rough understanding themselves.

We don't even know the exact architecture of GPT-4 - is it just a Transformer, or does it have more to it ? The head of OpenAI, Sam Altman, was interviewed by Lex Fridman yesterday (you can find it on YouTube) and he mentioned that, paraphrasing, "OpenAI is all about performance of the model, even if that involves hacks ...".

While Sutskever describes GPT-4 as having learnt this "world model", Sam Altman instead describes it as having learnt a non-specific "something" from the training data. It seems they may still be trying to figure out much of how it is working themselves, although Altman also said that "it took a lot of understanding to build GPT-4", so apparently it's more than just a scaling up of earlier models.

Note too that my description of it's internal state being maintained/updated through the conversation is likely (without knowing the exact architecture) to be more functional than literal since if it were just a plain Transformer then it's internal state is going to be calculated from scratch for each word it is asked to generate, but evidentially there is a great deal of continuity between the internal state when the input is, say, prompt words 1-100 as when it is words 2-101 - so (assuming they haven't added any architectural modification to remember anything of prior state), the internal state isn't really "updated" as such, but rather regenerated into updated form.

Lots of questions, not so many answers, unfortunately!

Post reply on HN