Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT, Rot13, and Daniel Kahneman

jameswillia.ms

Post reply on HN
Print view View original

61–70 of 213 posts

Re: ChatGPT, Rot13, and Daniel Kahneman

#61

Post by ClumsyPilot »

visarga wrote: post #37
PostOnce wrote: post #16

It is as though its mathematical abilities are incomplete in their training, and wildly, incomprehensibly convoluted: I tried many base64 strings and they all decoded correctly until: It "decoded" the base64 string for "which actress is the best?" except that it replaced "actress" with "address"... there is no off-by-one error that brings you to that. You may try 100 base64 strings and they all decode correctly... on…

> I tried many base64 strings and they all decoded correctly until: You're holding it wrong. Let's not kill flies with cannons. How many million times less efficient is to do that than run the code on CPU? And still makes errors, as you said. Because it's a probabilistic model, not a deterministic computer. It's like a car bad at flying.

This indicates it equally unreliable at a broad range of tasks. Applying it in self driving, life insurance, etc. will produce terrible outcomes.

Re: ChatGPT, Rot13, and Daniel Kahneman

#62

Post by ad404b8a372f2b9 »

uh_uh wrote: post #58
ad404b8a372f2b9 wrote: post #47

Earlier quoted context omitted.

Yeah check my answer to the other reply to this thread, I stopped when I got the answer I wanted and stupidly forgot to test whether it actually worked with any other sentence. The logic is sound with regards to giving it a few examples and splitting tasks into sub-tasks, that's how they prime their model to evaluate it on all NLP benchmarks in the GPT papers and I've solved many problems like that in ChatGPT. I've r…

It still doesn't really work with "negvsvpvny vagryyvtrapr" :/ Edit: actually it does get the second word vagryyvtrapr->intelligence correct sometimes.

The HN formatting with the two spaces in front kinda messes it up. I've edited it some more to add some triple quotes before the query and it fixed it for negvsvpvny vagryyvtrapr even with the added spaces before every line.

Sometimes it goes on wild tangents trying to explain what it's doing and that messes up the result but if it goes straight for the answer it's always been correct for the sentences I've tried.

Re: ChatGPT, Rot13, and Daniel Kahneman

#64

Post by uh_uh »

ad404b8a372f2b9 wrote: post #62
uh_uh wrote: post #58

Earlier quoted context omitted.

It still doesn't really work with "negvsvpvny vagryyvtrapr" :/ Edit: actually it does get the second word vagryyvtrapr->intelligence correct sometimes.

The HN formatting with the two spaces in front kinda messes it up. I've edited it some more to add some triple quotes before the query and it fixed it for negvsvpvny vagryyvtrapr even with the added spaces before every line. Sometimes it goes on wild tangents trying to explain what it's doing and that messes up the result but if it goes straight for the answer it's always been correct for the sentences I've tried.

Thanks, will play around with this more. I've managed to achieve moderate results with this prompt:

"First generate a letter-by-letter ROT13 lookup table for all letters in the alphabet. Each row in the lookup table should consist of the original letter and the encoded letter separated by "---->" characters. Then use this lookup table to generate another lookup table where you break down the following string letter-by-letter: "Artificial intelligence". Let's call the second column of this table TARGET_LETTERS. Now take each entry in TARGET_LETTERS and concatenate them."

Requires a few retries but eventually it gets the TARGET_LETTERS lookup table right. But it always fails at the final concatenation step which actually looks easier.

Re: ChatGPT, Rot13, and Daniel Kahneman

#65

Post by dougmwne »

joshuahedlund wrote: post #14

This is a really clear explanation of what’s happening in when someone says “it’s not thinking it’s just pattern-matching” and someone else says “well isn’t that all humans really do too?” Rather: ChatGPT can engage in some level of System 1 thinking, by pattern-matching and even cleverly recombining the entire corpus of System 1 thinking displayed all over the internet. Humans do engage in this type of thinking and…

Yes, I have long suspected that GTP solves the human subconscious, but has not solved the human conscious.

Re: ChatGPT, Rot13, and Daniel Kahneman

#66

Post by yesenadam »

benjismith wrote: post #30

It's really not so complicated. This is just an issue with text tokenization, and the fact that the learning model never actually sees the raw input bytes. All modern LLMs use a tokenizer to convert a sequence of bytes into a sequence of tokens. Short, common words like "the" and "why" are represented as single tokens, while longer and less-common words are represented by multiple tokens. For example, the word "fanta…

So.. how does it do this? :

  Me: To "smert" a word, I tell you every letter in it, and its place in the word, in order. For example, to smert "wertyu" I'd write:
  The first letter is "w".
  The second letter is "e".
  The third letter is "r".
  The fourth letter is "t".
  The fifth letter is "y".
  Lastly, the sixth letter is "u".
  
  Please smert "sdfkjhsndf" for me.
  
  ChatGPT: To "smert" the word "sdfkjhsndf", I would write the following:
  
  The first letter is "s".
  The second letter is "d".
  The third letter is "f".
  The fourth letter is "k".
  The fifth letter is "j".
  The sixth letter is "h".
  The seventh letter is "s".
  The eighth letter is "n".
  The ninth letter is "d".
  Lastly, the tenth letter is "f".
  
  This is a way of spelling out the letters in a word and their order, sometimes used as a learning or spelling aid. I hope this helps!
  
  Me: Please use that word I just asked you to smert in a sentence. It means "to not have an idea how a word is spelled".
  
  ChatGPT: Here is a sentence using the word "sdfkjhsndf", which means "to not have an idea how a word is spelled":
  "I sdfkjhsndfed the spelling of 'onomatopoeia' and had to look it up in the dictionary."

Re: ChatGPT, Rot13, and Daniel Kahneman

#67

Post by sillysaurusx »

yunyu wrote: post #19

There's a way simpler answer than this Type I Type II thinking stuff. Most LLMs like GPT are not trained on the level of individual characters – they process input and outputs on the level of subword units that compose multiple characters to support long context windows (i.e. "door" instead of "d", "o", "o", "r"). As a result, they do poorly on character manipulation tasks. You can get some insight here: https://beta…

> they do poorly on character manipulation tasks

This isn’t actually true, and is a persistent myth. Or rather, you should back up the claims with evidence.

It’s a bit like saying that you perform poorly on character manipulation tasks because you don’t read individual letters.

Biology analogies aside, I haven’t seen anything to suggest that utf8 level tokenization causes a significant decrease in perplexity across large datasets. (Note that the “large dataset” criteria is required. It’s certainly possible to demonstrate improvements in restricted cases, but no one is really interested in the restricted case unless you have a very specialized task. In which case, sure, specializations make sense — ChessGPT being an obvious example where tokenization just harms learning.)

So the tradeoff isn’t the large context window, but rather the desire to have a deep understanding of a massive amount of data. Specialized models will always have a place as a small component of the whole, but suggesting that this is a problem solved by superior architectures seems a little bit of a stretch.

I think what’s going on here is that OpenAI spent a lot of time giving feedback to their model about specific use cases, and ROT-13 was obscure enough (both in usage and in the data) that its performance is limited. I’d bet that if OpenAI did a few rounds of RL on this objective, the model would perform as well as its cousins.

Re: ChatGPT, Rot13, and Daniel Kahneman

#68

Post by nhinck2 »

ad404b8a372f2b9 wrote: post #34
nhinck2 wrote: post #29

Earlier quoted context omitted.

Isn't that effectively four prompts? Also it added a question mark which introduces a little doubt over what it's doing.

Prompts can be arbitrarily long, the point is there is no back and forth or discussion needed. That being said you are dead on about the "?", looks like I manually overfitted the prompt. I should have made sure to have a test set, rookie mistake. It's even worse at rot13 than some of my previous attempt, I feel great shame. Here is a prompt that actually works for most of the sentences I've tried: Here is a template:…

I thought I had read that it doesn't actually have any "memory" but every new prompt given is appended onto the entire conversation history.

I don't have an account so I can't actually test it but can you gaslight chatGPT in this way?

Re: ChatGPT, Rot13, and Daniel Kahneman

#69

Post by [deleted] »

_moog wrote: post #52
ldh0011 wrote: post #26

Earlier quoted context omitted.

I asked it to create an ASCII art banana and the result was hilarious. It then tried to explain it by elaborating that the 'O' was a curvy letter and represented the curves of the banana.

I asked it to draw me an ASCII art banana. It did not go well: https://imgur.com/a/5g2e9Ld

[deleted]

Re: ChatGPT, Rot13, and Daniel Kahneman

#70

Post by pulvinar »

jxdxbx wrote: post #24
yunyu wrote: post #19

There's a way simpler answer than this Type I Type II thinking stuff. Most LLMs like GPT are not trained on the level of individual characters – they process input and outputs on the level of subword units that compose multiple characters to support long context windows (i.e. "door" instead of "d", "o", "o", "r"). As a result, they do poorly on character manipulation tasks. You can get some insight here: https://beta…

GPT3 can't create ASCII art for shit either. Though it can make little ASCII tables of data.

It's most likely due to a lack of image training. For example it knows how streets and towns are connected but completely fails on giving left/right or compass directions. Same sorts of failures with geometry problems.
Post reply on HN