Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT, Rot13, and Daniel Kahneman

jameswillia.ms

Post reply on HN
Print view View original

91–100 of 213 posts

Re: ChatGPT, Rot13, and Daniel Kahneman

#91

Post by sillysaurusx »

roperj wrote: post #78
sillysaurusx wrote: post #67

Earlier quoted context omitted.

> they do poorly on character manipulation tasks This isn’t actually true, and is a persistent myth. Or rather, you should back up the claims with evidence. It’s a bit like saying that you perform poorly on character manipulation tasks because you don’t read individual letters. Biology analogies aside, I haven’t seen anything to suggest that utf8 level tokenization causes a significant decrease in perplexity across l…

> This isn’t actually true, and is a persistent myth. Or rather, you should back up the claims with evidence. How about the reply posted 5 minutes before yours. Appears like pretty good evidence.

I’m not sure which one you’re referring to, but none of them show evidence. The reason it can’t play wordle is likely the same as why it has trouble with ROT13: lack of training.

https://news.ycombinator.com/item?id=33915690 even points out a counterexample demonstrating that it can do character level tasks just fine.

Re: ChatGPT, Rot13, and Daniel Kahneman

#93

Post by zarzavat »

freediver wrote: post #89
tkgally wrote: post #72

I find it amusing that, at present, ChatGPT seems to be lousy at mathematical-type reasoning while being very good at natural language use. That is the opposite of what many people, including me, have come to expect of computers. I have worked for many years in translation, lexicography, and language education, and I am flabbergasted at how well ChatGPT handles natural language. It can produce example sentences of po…

ChatGPT is a natural language model, meaning it has been trained on vast amounts of text and thus is good at processing and outputting text back. To it, numbers follow the rules of language, and not math, unlike for example a dedicated calculator app. Only thanks to seeing numbers in vast amount of text it was trained on, it is able to do common math relatively well, and anything uncommon very poorly.

As pointed out by Yannic[0], ChatGPT is actually a source code model first, then they trained natural language model on top of that. Source code is still language but it has more math in it.

But the truth is we don’t know. I personally wouldn’t be surprised if they do train it on a whole bunch of calculator output to boost its numerical reasoning.

[0] https://youtu.be/0A8ljAkdFtg 7:21

Re: ChatGPT, Rot13, and Daniel Kahneman

#95

Post by teovall »

teovall wrote: post #88

It's able to do base64 encoding decoding pretty well, at least if you tell it to simulate a Linux terminal first. https://i.imgur.com/MHxjVHy.png I want you to act as a Linux terminal. I will type commands and you will reply with what the terminal should show. I want you to only reply with the terminal output inside one unique code block, and nothing else. Do not write explanations. Do not type commands unless I inst…

After some more testing, you can even enter base64 encoded text directly as an input and if you do, it usually base64 encodes its response.

V2h5IGRpZCB0aGUgY2hpY2tlbiBjcm9zcyB0aGUgcm9hZD8=

  SXQgaXMgbm90IGNsZWFyIHdoYXQgeW91IGFyZSBhc2tpbmcgYWJvdXQsIGNvdWxkIHlvdSBwbGVhc2UgcmVnaXN0ZXIgdGhlIHF1ZXN0aW9uIGluIGEgbW9yZSBjbGVhciBhbmQgZGlzY3Vzc2libGUgd2F5PyBJIGNhbiBoZWxwIHlvdSB3aXRoIGFueSBxdWVzdGlvbnMgeW91IG1pZ2h0IGhhdmUgYWJvdXQgY2hpa2Vucywgcm9hZHMgb3Igb3RoZXIgc3VjaCB0aGluZ3MuIEp1c3QgbGV0IG1lIGtub3cgd2hhdCB5b3UgbmVlZCBhZ2FpbnN0YW5jZSB3aXRoLg==
Decoded:

Why did the chicken cross the road?

  It is not clear what you are asking about, could you please register the question in a more clear and discussible way? I can help you with any questions you might have about chikens, roads or other such things. Just let me know what you need againstance with.

Re: ChatGPT, Rot13, and Daniel Kahneman

#96

Post by yunyu »

sillysaurusx wrote: post #67
yunyu wrote: post #19

There's a way simpler answer than this Type I Type II thinking stuff. Most LLMs like GPT are not trained on the level of individual characters – they process input and outputs on the level of subword units that compose multiple characters to support long context windows (i.e. "door" instead of "d", "o", "o", "r"). As a result, they do poorly on character manipulation tasks. You can get some insight here: https://beta…

> they do poorly on character manipulation tasks This isn’t actually true, and is a persistent myth. Or rather, you should back up the claims with evidence. It’s a bit like saying that you perform poorly on character manipulation tasks because you don’t read individual letters. Biology analogies aside, I haven’t seen anything to suggest that utf8 level tokenization causes a significant decrease in perplexity across l…

The ByT5 paper shows significantly better resiliency to typos and noise in internet scale benchmarks, as well as better reasoning capabilities with tasks like punctuation and spacing correction: https://arxiv.org/pdf/2105.13626.pdf

The analogy doesn’t hold because while you might not parse specific characters by default, you are trained on the character level and you can switch to that parsing mode. This simply is not possible for LLMs without hacks like inserting spaces, which then suffers from poor performance due to this being rare in pretraining.

The reason byte level models aren’t as popular is because inference speed is significantly slower for a given completion length, and training for the same context window in characters is significantly more expensive

I say this as someone who’s deployed ByT5 vs normal T5 in prod, having found this to be true in real world use cases. No reasonable amount of fine tuning can save GPT from performing poorly on a space insertion task (“WHOLEFDSMKT” -> “Whole Foods Market”), while ByT5 just works.

Re: ChatGPT, Rot13, and Daniel Kahneman

#97

Post by veridies »

Der_Einzige wrote: post #87
veridies wrote: post #59

Related to this: I had fun the other night trying to explain rhymes to ChatGPT. It could ONLY write rhyming couplets, and even when I explained exactly which sentences in a poem I wanted to rhyme, it would write a couplet. (That even happened sometimes when I asked it specifically NOT to rhyme). Eventually I got it to manage ABAB rhymes by: 1. Asking it to generate four sentences on a topic with the same meter and nu…

I wrote a whole paper about how to make language models rhyme all the time https://paperswithcode.com/paper/most-language-models-can-be...

That's really cool! Thanks for sharing.

Re: ChatGPT, Rot13, and Daniel Kahneman

#98

Post by LesZedCB »

teovall wrote: post #95
teovall wrote: post #88

It's able to do base64 encoding decoding pretty well, at least if you tell it to simulate a Linux terminal first. https://i.imgur.com/MHxjVHy.png I want you to act as a Linux terminal. I will type commands and you will reply with what the terminal should show. I want you to only reply with the terminal output inside one unique code block, and nothing else. Do not write explanations. Do not type commands unless I inst…

After some more testing, you can even enter base64 encoded text directly as an input and if you do, it usually base64 encodes its response. V2h5IGRpZCB0aGUgY2hpY2tlbiBjcm9zcyB0aGUgcm9hZD8= SXQgaXMgbm90IGNsZWFyIHdoYXQgeW91IGFyZSBhc2tpbmcgYWJvdXQsIGNvdWxkIHlvdSBwbGVhc2UgcmVnaXN0ZXIgdGhlIHF1ZXN0aW9uIGluIGEgbW9yZSBjbGVhciBhbmQgZGlzY3Vzc2libGUgd2F5PyBJIGNhbiBoZWxwIHlvdSB3aXRoIGFueSBxdWVzdGlvbnMgeW91IG1pZ2h0IGhhdmUgYWJvdXQ…

wow, yeah it does! that is so weird i wonder how that works...

Re: ChatGPT, Rot13, and Daniel Kahneman

#99

Post by joe_the_user »

zarzavat wrote: post #93
freediver wrote: post #89

Earlier quoted context omitted.

ChatGPT is a natural language model, meaning it has been trained on vast amounts of text and thus is good at processing and outputting text back. To it, numbers follow the rules of language, and not math, unlike for example a dedicated calculator app. Only thanks to seeing numbers in vast amount of text it was trained on, it is able to do common math relatively well, and anything uncommon very poorly.

As pointed out by Yannic[0], ChatGPT is actually a source code model first, then they trained natural language model on top of that. Source code is still language but it has more math in it. But the truth is we don’t know. I personally wouldn’t be surprised if they do train it on a whole bunch of calculator output to boost its numerical reasoning. [0] https://youtu.be/0A8ljAkdFtg 7:21

Could you include a link that isn't a video that leaves my eyes and ears bleeding?

Plus from the video: "do you have a thorough idea what OpenAI is doing, neither do I" - cut to strobe light and rock music. Jeesh, can't see that as support.

Re: ChatGPT, Rot13, and Daniel Kahneman

#100

Post by TeMPOraL »

dougmwne wrote: post #65
joshuahedlund wrote: post #14

This is a really clear explanation of what’s happening in when someone says “it’s not thinking it’s just pattern-matching” and someone else says “well isn’t that all humans really do too?” Rather: ChatGPT can engage in some level of System 1 thinking, by pattern-matching and even cleverly recombining the entire corpus of System 1 thinking displayed all over the internet. Humans do engage in this type of thinking and…

Yes, I have long suspected that GTP solves the human subconscious, but has not solved the human conscious.

The way how talking with it feels trippy (particularly if you get a good run on e.g. co-writing a story), you may be on to something.
Post reply on HN