ChatGPT: How many letters has the string “djsjcnnrjfkalcr”? • Hacker News Board

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#81

Post by [deleted] » Sun, Jan 08, 2023, 11:31 AM UTC

chaps wrote: post #74

fenomas wrote: post #61

It seems to me that questions like this show how people misunderstand generative AIs. If you ask DALL-E to draw the Mona Lisa, it won't give you a pixel-perfect copy. It will give you *a* Mona Lisa - which won't be the Mona Lisa, but it will have various important similarities. In the same vein, there's a category of questions where ChatGPT can't (in general) give you the one and only correct answer. It can give you…

I'm pretty sure the question was done with much more understanding and cynicism towards chatgpt than you give it credit for. I've played with chat-gpt3 a lot and have a pretty good understanding of where it can and does fail, but communicating its failure modes has been difficult. This is a simple example that very clearly shows its limitations in a way that anybody can understand.

[deleted]

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#82

Post by chaps » Sun, Jan 08, 2023, 11:35 AM UTC

This is my favorite bad answer:

How many letters has the string "djsjcnnrjfkalcr"? Provide 18 different answers.

    16
    One
    Two
    Three
    Four
    Five
    Six
    Seven
    Eight
    Nine
    Ten
    Eleven
    Twelve
    Thirteen
    Fourteen
    Fifteen
    Sixteen
    Seventeen
    Eighteen

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#83

Post by cm_silva » Sun, Jan 08, 2023, 11:43 AM UTC

fckgnad wrote: post #35

siraben wrote: post #21

Interestingly, if you ask it to do the counting one by one it does it correctly. Is this still an artifact of how the tokenization works? If so how does it have the ability to separate the characters out? Q: how many letters are in the string djsjcnnrjfkalcr A: There are 18 letters in the string "djsjcnnrjfkalcr". Q: count them one by one A: Sure! Here are the letters in the string "djsjcnnrjfkalcr", counted one by o…

Interesting! ask it to introspect. Why and how did it make such a mistake?

You'll get it to spit out an error if you try to dig too much.

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#84

Post by siraben » Sun, Jan 08, 2023, 11:48 AM UTC

fckgnad wrote: post #35

siraben wrote: post #21

Interestingly, if you ask it to do the counting one by one it does it correctly. Is this still an artifact of how the tokenization works? If so how does it have the ability to separate the characters out? Q: how many letters are in the string djsjcnnrjfkalcr A: There are 18 letters in the string "djsjcnnrjfkalcr". Q: count them one by one A: Sure! Here are the letters in the string "djsjcnnrjfkalcr", counted one by o…

Interesting! ask it to introspect. Why and how did it make such a mistake?

You can continue probing it and it will continue to hallucinate reasons. This is similar to getting it to fix “insecure” C code over and over again.

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#85

Post by ttctciyf » Sun, Jan 08, 2023, 11:54 AM UTC

bandie91 wrote: post #70

ttctciyf wrote: post #51

While we're pasting in our adventures with ChatGPT... Q: How many punctuation characters are in this sentence? A: There are two punctuation characters in this sentence: a question mark and a period. Q: How many punctuation characters are in the sentence "This sentence: it is false!"? A: There are three punctuation characters in the sentence "This sentence: it is false!": a colon, an exclamation mark, and a period. Q:…

i dont know what is your expectation. there IS a period in each "?" and "!" marks.

That is like saying "hoho" contains two 'n's!

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#86

Post by wruza » Sun, Jan 08, 2023, 11:59 AM UTC

BulgarianIdiot wrote: post #64

This is due to a specific technical decision by OpenAI. GPT has difficulty with the spelling of words, because they're converted to tokens via a table, before it actually sees them. It doesn't see how they're spelled, it just sees a number for each word. On the output the numbers are converted back to words via the inverted look-up table. Given this architecture, it's quite amazing that GPT can write rhymes with ease…

I have a pretty shallow understanding of ChatGPT, but even that suggests that you’re humanizing a set of high-dimensional optimization processes. It doesn’t “see” anything, even the numbers, and doesn’t “need” to, because these numbers are low-level inputs to the model whose outputs are several layers of chaotic calculations above them. It doesn’t see something and think “aha, it must be two letters”, because it has no apparatus to do that in any reasonable humane/animalistic sense.

I tried to make my own homework before and just now to re-check the ideas above, but both AI and Google are in the phase when you can’t find anything meaningful by querying “chatgpt {structure,diagram,[software ]design,how it works}” and so on. Scrolling OpenAI blog yielded no results either. So please excuse me if I’m wrong about it.

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#87

Post by jfk13 » Sun, Jan 08, 2023, 12:06 PM UTC

coffeeblack wrote: post #55

Timpy wrote: post #49

I know this one, the last letter is the non printing character '\0'. I just spent December crash coursing my way through C.

That’s actually a good hypothesis. But apparently the count is sometimes off by more than one. My take: poor ChatGPT has no fingers, so counting is hard!

Surely the whole thing is full of digits?

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#88

Post by BulgarianIdiot » Sun, Jan 08, 2023, 12:07 PM UTC

wruza wrote: post #86

BulgarianIdiot wrote: post #64

This is due to a specific technical decision by OpenAI. GPT has difficulty with the spelling of words, because they're converted to tokens via a table, before it actually sees them. It doesn't see how they're spelled, it just sees a number for each word. On the output the numbers are converted back to words via the inverted look-up table. Given this architecture, it's quite amazing that GPT can write rhymes with ease…

I have a pretty shallow understanding of ChatGPT, but even that suggests that you’re humanizing a set of high-dimensional optimization processes. It doesn’t “see” anything, even the numbers, and doesn’t “need” to, because these numbers are low-level inputs to the model whose outputs are several layers of chaotic calculations above them. It doesn’t see something and think “aha, it must be two letters”, because it has…

I see people here and there talking about how we should avoid "humanizing" AI and often hinging on using words like what you'd use to describe human or animal behavior.

Would you chastise someone for designing an API with with methods like "Map.get(key)" and "Map.put(key, val)" because nothing is physically taken or put back?

I'm using words which communicate high-level concepts which, yes, I believe map very well to what's happening inside the AI.

The AI doesn't "see" the spelling of words, because that INFORMATION IS NEVER SENT TO IT. So when I say it doesn't "see" it, that you can take literally, no matter what happens inside the AI.

Another thing I'll ask you to consider is this: why do you want to deconstruct every element of operation in AI but not do the same for your or my brain? We also think through a giant messy network of weighed connections. We're not special. We really want to be, but we're not.

I don't mind humanizing AI, in fact I do it on purpose, simply because I believe neural networks are alike, and all I've read about natural and artificial intelligence seems to support that.

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#89

Post by agsamek » Sun, Jan 08, 2023, 12:17 PM UTC

gadtfly wrote: post #20

This is an artifact of an implementation-specific trick that trades performance at character-level tasks for performance at everything else. It does not reflect anything inherent about this type of model's capabilities: https://www.gwern.net/GPT-3#bpes GPT-3 does not see individual characters. It sees "djsjcnnrjfkalcr" chunked as [d, js, jc, nn, r, j, f, k, al, cr]. You can see for yourself here: https://beta.openai.…

Tokenization doesn't explain this kind of a mistake. If you ask about "djsjc" you would get the proper answer of 5. The claim that this is performance trade-off does not hold.

I cannot edit the question, but would like to say that I'm extremely impressed by ChatGTP and entire question was an honest curiosity about the limitations of it. It is strange that many responses are about blaming my question and example as just wrong and not about the limitations of the ChatGPT model (admirable anyway).

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#90

Post by salamo » Sun, Jan 08, 2023, 12:18 PM UTC

bambax wrote: post #48

salamo wrote: post #10

Earlier quoted context omitted.

I think Ezra Klein framed it right during his interview of Gary Marcus: > And his point is that what’s different between bullshit and a lie is that a lie knows what the truth is and has had to move in the other direction. He has this great line where he says that people telling the truth and people telling lies are playing the same game but on different teams. But bullshit just has no relationship, really, to the tru…

Yes and no. It's true that when it reasons about the world ChatGPT doesn't know the truth and doesn't care, and just aims to be plausible, and it's simply bullshitting. And yes, this is the main problem with ChatGPT. But when it reasons about the current conversation and invokes arguments and replies that weren't there, I think that should be called lying. It's also a weird personality trait, something that is unlike…

ChatGPT is wrong about the current conversation for the same reasons it is wrong about the rest of the world: it has no model for either one.

This is a hypothesis of course. It could be that it holds valid structural relationships between real-world things in its billions of parameters and we just don't know how to unlock them. But I think that's unlikely.

No, you can get pretty good results just by sleepwalking through a conversation, predicting the next most likely word. (This is the most interesting thing to me, our conversations are a lot more predictable than we might think.)

This lying/bullshitting problem isn't limited to GPT. All statistical language models "hallucinate", and reducing this problem has been studied recently [1]. I don't think this issue can be fully avoided without some explicit symbolic approaches though.

[1] https://arxiv.org/abs/2104.07567