Teaching ChatGPT to speak my son’s invented language • Hacker News Board

Re: Teaching ChatGPT to speak my son’s invented language

#111

Post by gwright » Tue, Apr 11, 2023, 3:15 AM UTC

xwdv wrote: post #106

ravi-delia wrote: post #105

Earlier quoted context omitted.

Clearly GPT doesn't "know" things in the way we do- I'd argue that there's probably a little bit of world modeling in there, but piecemeal and crude at best. Only so much you can do with such limited iteration. However, we must acknowledge that there are evidently some tasks that don't require knowledge the way we usually think about it. For a human, being able to talk about a game stems from the same place as our ab…

I’ll give you an example to think about in human terms. In political discussions, a lot of humans don’t really know what the fuck they’re talking about, but they do know what to say in response to certain stimulus. They know the talking points, the key phrases, the terms, they parrot these back to you when you provoke them to say it. They are basically human sentence predictors. Stuff comes out of their mouth based o…

Thank you for that example. I've been thinking a lot about how difficult it is to persuade people to change their mind. Most people seem impervious to incorporating new facts or ideas into their internal narrative (myself included). ChatGPT has made me wonder if that is because much of what we consider "cognition" in humans is really just "human sentence predictors".

We've all been "trained" with various facts and when two people meet who have been trained on substantially different bodies of knowledge/facts/experiences it can be very difficult to find common ground.

FWIW, I asked ChatGPT to give me a short list of cognitive biases and psychological phenomena. Interesting to think how many of these are dependent on our personal "training data":

    Confirmation Bias
    Cognitive Dissonance
    Anchoring Bias
    Belief Perseverance
    Groupthink
    Ingroup Bias
    Sunk Cost Fallacy
    Motivated Reasoning

Re: Teaching ChatGPT to speak my son’s invented language

#112

Post by flangola7 » Tue, Apr 11, 2023, 3:21 AM UTC

famouswaffles wrote: post #9

In context learning is hands down the biggest breakthrough of LLMs. The flexibility the model displays without updating weights is genuinely mind blowing, bordering on absurd especially if you've trained other kinds of models before. See here - https://imgur.com/a/w3DAYOi from the paper - https://arxiv.org/abs/2211.09066 GPT 3.5's (4 is much much better) addition accuracy tanks after 2 digits. However, by approaching…

>However, by approaching arithmetic as an algorithm to be performed and taught similarly to how it's done with people, you can supercharge accuracy to basically 100% for up to 13 digit addition and >90% after.

This is strikingly clear evidence that it is capable of logical reasoning and "knows things", and puts to rest repeated claims that it is just memorization and collaging. You can't memorize every combo of two 13 digit numbers within the file size of the model.

Re: Teaching ChatGPT to speak my son’s invented language

#113

Post by jackmott » Tue, Apr 11, 2023, 3:21 AM UTC

crdrost wrote: post #73

Wow, they asked the model to self-evaluate and it just outright cheated: He has three cats. Proposed: h’io’ngkiltrikumrikumrikumri’nguuy Correct: h’io’ngkiltri’ngkumrikumri’nguuy Points: 1 Hypothesis: N/A (Other comments observe that it accidentally compensated for this by getting the sum wrong, haha, d'oh) I have had similar problems with trying to get ChatGPT to do nontrivial things, "here are the rules for this ga…

[dead]

Re: Teaching ChatGPT to speak my son’s invented language

#114

Post by jquery » Tue, Apr 11, 2023, 3:24 AM UTC

Daneel_ wrote: post #90

sharkweek wrote: post #75

Earlier quoted context omitted.

I was trying for 20 minutes to get it to spit out all 50 state capitals with the city names in alphabetical order and it kept doing two things: 1) It'd put the list in alphabetical order by state, but it'd include all the correct capitals 2) It'd list 49 of the 50 capitals, in alphabetical order this time, but duplicating Madison, WI. I'd ask it to try and figure out what it did wrong in both cases, and it'd correctl…

Not here to one-up you, but currently this is just down to how you ask. I came up with this in about a minute: "Please list all 50 US state capital cities, with the list sorted alphabetically starting at the first letter of each line of your response. Please do not create sections for each letter." This returned: - Albany, New York - Annapolis, Maryland - Atlanta, Georgia - Augusta, Maine - Austin, Texas - Baton Roug…

Supplication ("please") does sometimes give better responses. Or at least more polite ones.

Re: Teaching ChatGPT to speak my son’s invented language

#115

Post by FPGAhacker » Tue, Apr 11, 2023, 3:25 AM UTC

famouswaffles wrote: post #9

In context learning is hands down the biggest breakthrough of LLMs. The flexibility the model displays without updating weights is genuinely mind blowing, bordering on absurd especially if you've trained other kinds of models before. See here - https://imgur.com/a/w3DAYOi from the paper - https://arxiv.org/abs/2211.09066 GPT 3.5's (4 is much much better) addition accuracy tanks after 2 digits. However, by approaching…

I have very limited understanding of chatGPTs architecture, and neural nets in general.

From what I've read over the last few days, the "attention" mechanism used in chatGPT and similar LLMs does indeed dynamically change weights of a portion of the model.

Re: Teaching ChatGPT to speak my son’s invented language

#116

Post by yosito » Tue, Apr 11, 2023, 3:40 AM UTC

famouswaffles wrote: post #9

In context learning is hands down the biggest breakthrough of LLMs. The flexibility the model displays without updating weights is genuinely mind blowing, bordering on absurd especially if you've trained other kinds of models before. See here - https://imgur.com/a/w3DAYOi from the paper - https://arxiv.org/abs/2211.09066 GPT 3.5's (4 is much much better) addition accuracy tanks after 2 digits. However, by approaching…

I've noticed if I give chat GPT an algorithm it gives me consistent results when it shows it's work but if I ask it not to show it's work and just give me the answer. It consistently gives me incorrect answers even when I ask it to follow an algorithm that I gave it.

I suppose this is similar to humans and probably why my school teachers always told me to show my work, but I'm curious if this has been documented and if there are any explanations for why it works this way with LLMs.

Re: Teaching ChatGPT to speak my son’s invented language

#117

Post by famouswaffles » Tue, Apr 11, 2023, 3:46 AM UTC

yosito wrote: post #116

famouswaffles wrote: post #9

In context learning is hands down the biggest breakthrough of LLMs. The flexibility the model displays without updating weights is genuinely mind blowing, bordering on absurd especially if you've trained other kinds of models before. See here - https://imgur.com/a/w3DAYOi from the paper - https://arxiv.org/abs/2211.09066 GPT 3.5's (4 is much much better) addition accuracy tanks after 2 digits. However, by approaching…

I've noticed if I give chat GPT an algorithm it gives me consistent results when it shows it's work but if I ask it not to show it's work and just give me the answer. It consistently gives me incorrect answers even when I ask it to follow an algorithm that I gave it. I suppose this is similar to humans and probably why my school teachers always told me to show my work, but I'm curious if this has been documented and…

chain of thought prompting. It's well known

https://arxiv.org/abs/2205.11916

https://arxiv.org/abs/2201.11903

Re: Teaching ChatGPT to speak my son’s invented language

#118

Post by famouswaffles » Tue, Apr 11, 2023, 3:57 AM UTC

FPGAhacker wrote: post #115

famouswaffles wrote: post #9

In context learning is hands down the biggest breakthrough of LLMs. The flexibility the model displays without updating weights is genuinely mind blowing, bordering on absurd especially if you've trained other kinds of models before. See here - https://imgur.com/a/w3DAYOi from the paper - https://arxiv.org/abs/2211.09066 GPT 3.5's (4 is much much better) addition accuracy tanks after 2 digits. However, by approaching…

I have very limited understanding of chatGPTs architecture, and neural nets in general. From what I've read over the last few days, the "attention" mechanism used in chatGPT and similar LLMs does indeed dynamically change weights of a portion of the model.

the connection between weights are dynamic but the weights themselves aren't being changed i.e being updated in response to the text.

when training a model, the forward pass would happen i.e the generation and then depending on how close to truth it was, the configuration settings (aka the weights/neurons) would be adjusted to incorporate whatever little insight was gained from the text.

Weights are matrices. The values of the matrices aren't changing.

Re: Teaching ChatGPT to speak my son’s invented language

#119

Post by jhanschoo » Tue, Apr 11, 2023, 4:00 AM UTC

chankstein38 wrote: post #27

marcodiego wrote: post #12

I don't have access to ChatGPT4, but in my tests I could observe that it can't do some very simple tasks: - It can't play tic-tac-toe, - It can't play hangman, - It insists that winning on stone-paper-scissor using the chat (playing before me) is a matter of probability. It was also demonstrated that it can't reverse strings. Actually a transformer doesn't accesses 'strings', all it processes are tokens which are the…

You're trying the old primary school method of testing things based on the wrong criteria. Why does it need to play tic-tac-toe, hangman, or rock paper scissors? Why do you think a language model would be good at those things? Similarly, why would you expect a thing trained on the context of text would be good at reversing strings? It's amazing it's as good as it is at these things because it doesn't really make sens…

The idea for games is that these are simple games, presumably with complete descriptions from various sources already in their database. The test is of their reasoning ability and ability to generalize descriptions into tasks that aren't chimeras of examples already in their database, y'know, like a classic test of what we'd call raw reasoning and intelligence, not crystallized intelligence.

With reversing strings unfortunately it's not likely to achieve that considering its tokenization.

Re: Teaching ChatGPT to speak my son’s invented language

#120

Post by nearbuy » Tue, Apr 11, 2023, 4:03 AM UTC

gwright wrote: post #109

sharkweek wrote: post #98

Earlier quoted context omitted.

Maybe, but I also think it’s fair to assume when you ask it in a much simpler way, “please list all US state capitals in alphabetical order” it should be able to do so. That isn’t a complicated question that requires much refinement to understand. To your “please” comment, I also find myself using common niceties probably out of social habit, but also perhaps because in the back of my mind I want the AI that inevitab…

I feel like your observation that this "isn't a complicated question" is leaning on an implicit assumption that ChatGPT is a general AI and not a LLM. It is just generating text based on probabilities -- it isn't "reasoning". I might go as far to say that inferences computed by a LLM are of all the same complexity but I don't really know enough about ChatGPT to be confident in that statement.

People keep repeating that LLMs are "just generating text based on probabilities". That statement doesn't mean anything.

I think people who say this are imagining LLMs work something like a statistical model. Maybe it's doing a linear regression or works like a Markov chain. It's not.

A single artificial neuron sort of works like that. But that's sort of like saying a single transistor is just an electronically controlled switch, so the only thing computers can do is switching. It's true in some sense that computers are just doing a lot of switching, but it turns out all this switching is Turing-complete. That means computers can theoretically compute anything that's possible to compute given enough time and memory, which includes anything a human could figure out.

Similar principle applies to LLMs. Using probabilities is part of what they do, but that doesn't preclude them from using logic and rules of inference.