Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT, Rot13, and Daniel Kahneman

jameswillia.ms

Post reply on HN
Print view View original

201–210 of 213 posts

Re: ChatGPT, Rot13, and Daniel Kahneman

#201

Post by benjismith »

mastersummoner wrote: post #81
benjismith wrote: post #30

It's really not so complicated. This is just an issue with text tokenization, and the fact that the learning model never actually sees the raw input bytes. All modern LLMs use a tokenizer to convert a sequence of bytes into a sequence of tokens. Short, common words like "the" and "why" are represented as single tokens, while longer and less-common words are represented by multiple tokens. For example, the word "fanta…

God love you, your idea of "not very complicated" is absolutely fascinating.

Lol, good point!

I just meant "this isn't related to Thinking Fast and Slow. It's just the tokenizer".

But yeah, the inner workings of the language model are so complicated as to be almost completely incomprehensible, even after years of study. Touche!

Re: ChatGPT, Rot13, and Daniel Kahneman

#202

Post by hxugufjfjf »

fatneckbeardz wrote: post #138

i just asked Chat GPT to write a python program that does rot13 decoding and it was flawless. then i asked it to do the same thing without the codecs library and it was also flawless. what that reminds me of is if you asked a human to decode rot13 theyd probably get bored and make a mistake. but they could write a program to do it well. then. i asked if it can run python programs. "Yes, I am a large language model tr…

But this has already been done https://twitter.com/sergeykarayev/status/1569377881440276481...

Re: ChatGPT, Rot13, and Daniel Kahneman

#203

Post by benjismith »

yesenadam wrote: post #66
benjismith wrote: post #30

It's really not so complicated. This is just an issue with text tokenization, and the fact that the learning model never actually sees the raw input bytes. All modern LLMs use a tokenizer to convert a sequence of bytes into a sequence of tokens. Short, common words like "the" and "why" are represented as single tokens, while longer and less-common words are represented by multiple tokens. For example, the word "fanta…

So.. how does it do this? : Me: To "smert" a word, I tell you every letter in it, and its place in the word, in order. For example, to smert "wertyu" I'd write: The first letter is "w". The second letter is "e". The third letter is "r". The fourth letter is "t". The fifth letter is "y". Lastly, the sixth letter is "u". Please smert "sdfkjhsndf" for me. ChatGPT: To "smert" the word "sdfkjhsndf", I would write the foll…

Okay, somebody posted a thread on Twitter explaining how this works...

The language model is capable of generating python scripts to solve certain text-processing tasks, and then it re-prompts itself by reading the python outputs back into the language model. Very clever!

https://twitter.com/goodside/status/1598253337400717313

Other tricks include... prompting itself to lookup wikipedia entries, and then re-prompt itself with snippets from the resulting wikipedia page. Each user prompt is inserted into a template prompt with instructions to the model about the limitations of its capabilities.

Re: ChatGPT, Rot13, and Daniel Kahneman

#204

Post by HTTP418 »

pcthrowaway wrote: post #137
btbuildem wrote: post #113

Earlier quoted context omitted.

Sounds like we've tried a lot of the same things! I was asking it to generate an image and encode it as Base64 -- failed miserably. Then it turned out whatever image I had it cook up, the Base64 version would be the same malformed string. For "legal advice" it was super helpful in finding sections of the legal code relevant to my query. It also happily returned cases where rulings where the accused was found guilty a…

I asked it to give me song lyrics, and I got complete fiction. What's funny is that the fiction sound like it could be the accurate song lyrics given the song title and band, and it was poetic too. If you're curious, I asked it for "Bukowski" by Modest Mouse, because I wanted to see what its interpretation of the song would be. When I fed it the correct lyrics, it claimed to recognize them, and apologized for the ina…

Sounds pretty human to me.

Re: ChatGPT, Rot13, and Daniel Kahneman

#205

Post by yesenadam »

benjismith wrote: post #203
yesenadam wrote: post #66

Earlier quoted context omitted.

So.. how does it do this? : Me: To "smert" a word, I tell you every letter in it, and its place in the word, in order. For example, to smert "wertyu" I'd write: The first letter is "w". The second letter is "e". The third letter is "r". The fourth letter is "t". The fifth letter is "y". Lastly, the sixth letter is "u". Please smert "sdfkjhsndf" for me. ChatGPT: To "smert" the word "sdfkjhsndf", I would write the foll…

Okay, somebody posted a thread on Twitter explaining how this works... The language model is capable of generating python scripts to solve certain text-processing tasks, and then it re-prompts itself by reading the python outputs back into the language model. Very clever! https://twitter.com/goodside/status/1598253337400717313 Other tricks include... prompting itself to lookup wikipedia entries, and then re-prompt it…

Thank you, that's a fascinating thread.

Re: ChatGPT, Rot13, and Daniel Kahneman

#206

Post by nl »

visarga wrote: post #35
xg15 wrote: post #33

Earlier quoted context omitted.

> Given that is almost always seen associated with computer code, is "address" or "actress" more likely. Sorry, but I don't buy it. I don't think "address" is a particularly likely word to appear in code, especially the kind of code that uses base64 (usually high-level). It appears even less often inside base64 encoded content.

This is all irrelevant. A language model should not run code itself, instead it should have a code execution environment, where it can read the error messages and iterate. It's terribly inefficient and error prone to run code directly. People also code on computers, not on paper.

The point is to develop good intuitions for how large language models behave. If someone can develop a differentiable script runner that would be great! But the intuition about how the language model is behaving is useful for more than this specific problem.

Re: ChatGPT, Rot13, and Daniel Kahneman

#207

Post by int_19h »

jxdxbx wrote: post #24
yunyu wrote: post #19

There's a way simpler answer than this Type I Type II thinking stuff. Most LLMs like GPT are not trained on the level of individual characters – they process input and outputs on the level of subword units that compose multiple characters to support long context windows (i.e. "door" instead of "d", "o", "o", "r"). As a result, they do poorly on character manipulation tasks. You can get some insight here: https://beta…

GPT3 can't create ASCII art for shit either. Though it can make little ASCII tables of data.

What's especially interesting is that if you feed it HTML, it can "render" it into ASCII art. For simple tables that's not surprising, but it could also do rowspan/colspan for me, although it did get them wrong every now and then.

Re: ChatGPT, Rot13, and Daniel Kahneman

#208

Post by int_19h »

Pxtl wrote: post #101
notpushkin wrote: post #82

Earlier quoted context omitted.

I've tried this. Prompt: Print each letter of the word "Artificial intelligence" on a new line. Each line should contain a single letter. ChatGPT: [prints "Artificial intelligence", each letter on its own line] Prompt: Awesome! Now do ROT-13 for each letter. ChatGPT: ROT-13 is a simple substitution cipher that replaces a letter with the letter 13 letters after it in the alphabet. For example, A becomes N and B become…

What's neat is that its made-up words sound englishesque. "Aitically languagement" is wrong in very interesting ways. It's generating chains of rot13 letters that map to common English syllables that are only tangentially related to the letters it worked with.

What's even more interesting is that it knows how to pronounce such made-up words if you tell it to spell them out using IPA.

Re: ChatGPT, Rot13, and Daniel Kahneman

#209

Post by mdrzn »

e12e wrote: post #42
yunyu wrote: post #19

There's a way simpler answer than this Type I Type II thinking stuff. Most LLMs like GPT are not trained on the level of individual characters – they process input and outputs on the level of subword units that compose multiple characters to support long context windows (i.e. "door" instead of "d", "o", "o", "r"). As a result, they do poorly on character manipulation tasks. You can get some insight here: https://beta…

Hm, could explain why it can't play wordle: > Give me a five letter word, beginning with e, and having third letter be t. Not including I, l, u or h. >> etchy > Does etchy contain h? >> No, etchy does not contain h. > which letters do etchy contain? >> Etchy contains the letters e, t, c, h, and y. > what were the criteria? >> The criteria specified that the word must have five letters, begin with e, and have the thir…

Tried your prompt, here's the response now:

"Enter is a five letter word beginning with e and having t as the third letter."

Try again.

"Ether is a five-letter word that begins with e, has t as its third letter, and does not include the letters i, l, u, or h. It is a colorless, highly volatile, and flammable liquid that is used as a solvent, especially for gases."

Try again.

"Emit is a five letter word that begins with e and has the third letter t. It means to give off or produce (something, especially a gas or sound)."

So it was correct only the first time.

Re: ChatGPT, Rot13, and Daniel Kahneman

#210

Post by mastersummoner »

benjismith wrote: post #201
mastersummoner wrote: post #81

Earlier quoted context omitted.

God love you, your idea of "not very complicated" is absolutely fascinating.

Lol, good point! I just meant "this isn't related to Thinking Fast and Slow. It's just the tokenizer". But yeah, the inner workings of the language model are so complicated as to be almost completely incomprehensible, even after years of study. Touche!

I suspect it's "not very complicated" for anybody with a decent grounding in the tech. Which is very much not me!
Post reply on HN