Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT, Rot13, and Daniel Kahneman

jameswillia.ms

Post reply on HN
Print view View original

71–80 of 213 posts

Re: ChatGPT, Rot13, and Daniel Kahneman

#71

Post by sebzim4500 »

yesenadam wrote: post #66
benjismith wrote: post #30

It's really not so complicated. This is just an issue with text tokenization, and the fact that the learning model never actually sees the raw input bytes. All modern LLMs use a tokenizer to convert a sequence of bytes into a sequence of tokens. Short, common words like "the" and "why" are represented as single tokens, while longer and less-common words are represented by multiple tokens. For example, the word "fanta…

So.. how does it do this? : Me: To "smert" a word, I tell you every letter in it, and its place in the word, in order. For example, to smert "wertyu" I'd write: The first letter is "w". The second letter is "e". The third letter is "r". The fourth letter is "t". The fifth letter is "y". Lastly, the sixth letter is "u". Please smert "sdfkjhsndf" for me. ChatGPT: To "smert" the word "sdfkjhsndf", I would write the foll…

Interestingly it doesn't do it reliably when you give it an actual word.

    To smert "consequence," please see below:

    The first letter is "c".
    The second letter is "o".
    The third letter is "n".
    The fourth letter is "s".
    The fifth letter is "e".
    The sixth letter is "q".
    The seventh letter is "u".
    The eighth letter is "e".
    The ninth letter is "n".
    The tenth letter is "c".
    The eleventh letter is "e".
    Lastly, the twelfth letter is "s".

Re: ChatGPT, Rot13, and Daniel Kahneman

#72

Post by tkgally »

I find it amusing that, at present, ChatGPT seems to be lousy at mathematical-type reasoning while being very good at natural language use. That is the opposite of what many people, including me, have come to expect of computers.

I have worked for many years in translation, lexicography, and language education, and I am flabbergasted at how well ChatGPT handles natural language. It can produce example sentences of polysemous words as well as or better than an experienced dictionary editor (i.e., me) [1], and it can correctly guess the meanings of unknown words from very limited context [2].

Teaching an adult human to use a second language without making grammatical mistakes is nearly impossible, and native speakers often make mistakes as well. In a week of testing, I have yet to see ChatGPT make any grammatical mistakes in either English or Japanese. Like many native speakers, however, it is often not able to explain its grammatical instincts correctly [3].

[1] https://www.gally.net/temp/202212chatgpt/dictionarydefinitio...

[2] https://www.gally.net/temp/202212chatgpt/unknownwords.html

[3] https://www.gally.net/temp/202212chatgpt/explaininggrammar.h...

Re: ChatGPT, Rot13, and Daniel Kahneman

#73

Post by sarchertech »

Tonight I was asking it to solve the 2 generals problem, and every time it put forth “solutions”.

The problem can’t be solved. You can relax the problem and that problem can be solved.

But each time either the solutions were incorrect even for relaxed versions of the problem, or they worked for relaxed versions, but it failed to mention the relaxed constraints. The problem is that the solutions seem plausible unless you are already familiar with the problem.

Re: ChatGPT, Rot13, and Daniel Kahneman

#75

Post by verdenti »

ClumsyPilot wrote: post #61
visarga wrote: post #37

Earlier quoted context omitted.

> I tried many base64 strings and they all decoded correctly until: You're holding it wrong. Let's not kill flies with cannons. How many million times less efficient is to do that than run the code on CPU? And still makes errors, as you said. Because it's a probabilistic model, not a deterministic computer. It's like a car bad at flying.

This indicates it equally unreliable at a broad range of tasks. Applying it in self driving, life insurance, etc. will produce terrible outcomes.

No post body was provided.

Re: ChatGPT, Rot13, and Daniel Kahneman

#77

Post by benjismith »

yesenadam wrote: post #66
benjismith wrote: post #30

It's really not so complicated. This is just an issue with text tokenization, and the fact that the learning model never actually sees the raw input bytes. All modern LLMs use a tokenizer to convert a sequence of bytes into a sequence of tokens. Short, common words like "the" and "why" are represented as single tokens, while longer and less-common words are represented by multiple tokens. For example, the word "fanta…

So.. how does it do this? : Me: To "smert" a word, I tell you every letter in it, and its place in the word, in order. For example, to smert "wertyu" I'd write: The first letter is "w". The second letter is "e". The third letter is "r". The fourth letter is "t". The fifth letter is "y". Lastly, the sixth letter is "u". Please smert "sdfkjhsndf" for me. ChatGPT: To "smert" the word "sdfkjhsndf", I would write the foll…

Good question! I don't know :)

Re: ChatGPT, Rot13, and Daniel Kahneman

#78

Post by roperj »

sillysaurusx wrote: post #67
yunyu wrote: post #19

There's a way simpler answer than this Type I Type II thinking stuff. Most LLMs like GPT are not trained on the level of individual characters – they process input and outputs on the level of subword units that compose multiple characters to support long context windows (i.e. "door" instead of "d", "o", "o", "r"). As a result, they do poorly on character manipulation tasks. You can get some insight here: https://beta…

> they do poorly on character manipulation tasks This isn’t actually true, and is a persistent myth. Or rather, you should back up the claims with evidence. It’s a bit like saying that you perform poorly on character manipulation tasks because you don’t read individual letters. Biology analogies aside, I haven’t seen anything to suggest that utf8 level tokenization causes a significant decrease in perplexity across l…

> This isn’t actually true, and is a persistent myth. Or rather, you should back up the claims with evidence.

How about the reply posted 5 minutes before yours. Appears like pretty good evidence.

Re: ChatGPT, Rot13, and Daniel Kahneman

#79

Post by fragmede »

PostOnce wrote: post #16

It is as though its mathematical abilities are incomplete in their training, and wildly, incomprehensibly convoluted: I tried many base64 strings and they all decoded correctly until: It "decoded" the base64 string for "which actress is the best?" except that it replaced "actress" with "address"... there is no off-by-one error that brings you to that. You may try 100 base64 strings and they all decode correctly... on…

> It can provide citations for legal opinions -- but decades old citations that don't reflect current precedent.

From what I've seen of "citations" in other areas (eg asking it to generate stack overflow answers), I'm surprised the citations are even real. It seemed to be wholly making up citations, complete with real-enough looking URLs!

Re: ChatGPT, Rot13, and Daniel Kahneman

#80

Post by abc_lisper »

joshuahedlund wrote: post #14

This is a really clear explanation of what’s happening in when someone says “it’s not thinking it’s just pattern-matching” and someone else says “well isn’t that all humans really do too?” Rather: ChatGPT can engage in some level of System 1 thinking, by pattern-matching and even cleverly recombining the entire corpus of System 1 thinking displayed all over the internet. Humans do engage in this type of thinking and…

I think so too. https://twitter.com/gorrepati/status/1601033566405931009
Post reply on HN