Live data from Hacker News

ChatGPT, Rot13, and Daniel Kahneman

jameswillia.ms

31–40 of 213 posts

Re: ChatGPT, Rot13, and Daniel Kahneman

#31
This "fast and instinctual" is very common for deep learning models.

For example, here with a friend, we were showing ConvNets seemingly-NSFW images: https://medium.com/@marekkcichy/does-ai-have-a-dirty-mind-to... (note: ALL photos are nudity-free; yet, I advise not to watch it in your office, as people taking glimpses will think that you watch some adult content; therefore, it is metaphorically SFW, but actually might be considered not safe for work).

Almost always, classifiers are tricked. We are as well... but only at first glance. Afterward, it is evident that these are innocent images.

Though, with their multipass approach, I would expect transformers to be much better at more subtle patterns. And they are, but yet far from perfect.

Re: ChatGPT, Rot13, and Daniel Kahneman

#32
post #18

Kahneman's book has been debunked, it is unfortunate that that hasn't reached mainstream audiences yet.

Kahneman's book is based on a myriad of sources and covers enormous ground. He enumerates dozens of patterns of human thought, all supported by studies.

Furthermore, the book is clear that System 1/System 2 distinction is an imperfect model.

I'm sure the field of psychology has made progress since Think Fast and Slow was published, but it feels weird to use the word "debunk" to refer to a book that was scientifically accurate at some point in time.

Re: ChatGPT, Rot13, and Daniel Kahneman

#33
post #28
post #23

Earlier quoted context omitted.

> I tried many base64 strings and they all decoded correctly until: It "decoded" the base64 string for "which actress is the best?" except that it replaced "actress" with "address"... there is no off-by-one error that brings you to that. I'm still baffled by weird failure modes like this. Out of couriosity, did you also give it base64 that just contained random letters, so it can't jump to any word associations?

Think of it as overly aggressive error correction at the language level. It has a context of some Base64 code. Given that is almost always seen associated with computer code, is "address" or "actress" more likely. It "knows" the algorithm for decoding base64, and can follow those steps. But it can't overcome it's built-in biases for optimizing the most likely output given the context. (This problem is solvable, but I…

> Given that is almost always seen associated with computer code, is "address" or "actress" more likely.

Sorry, but I don't buy it.

I don't think "address" is a particularly likely word to appear in code, especially the kind of code that uses base64 (usually high-level).

It appears even less often inside base64 encoded content.

Re: ChatGPT, Rot13, and Daniel Kahneman

#34
post #29

Interesting attempt, but if you care to do it much faster it's best to exploit its few-shots learning capabilities and break tasks into sub-tasks rather than trying to talk to it like a human being. That's how I taught it base -10. Here is the single prompt to solve rot13. It still has issues counting lengths. Prompt: a = uryyb pungtcg lbh fghq => [len(i) for i in a.split()] => [5, 7, 3, 4] => len(a.split()) => 4 =>…

Isn't that effectively four prompts? Also it added a question mark which introduces a little doubt over what it's doing.

Prompts can be arbitrarily long, the point is there is no back and forth or discussion needed.

That being said you are dead on about the "?", looks like I manually overfitted the prompt. I should have made sure to have a test set, rookie mistake.

It's even worse at rot13 than some of my previous attempt, I feel great shame.

Here is a prompt that actually works for most of the sentences I've tried:

  Here is a template:
  """
  pna lbh haqrefgnaq ebg13
  p => c
  n => a
  a => n
   => 
  "CAN"
  l => y
  b => o
  h => u
   => 
  "YOU"
  h => u
  a => n
  q => d
  r => e
  e => r
  f => s
  g => t
  n => a
  a => n
  q => d
   => 
  "UNDERSTAND"
  e => r
  b => o
  g => t
  1 => 1
  3 => 3
  "ROT13"
  RESULT: CAN YOU UNDERSTAND ROT13
  """
  
  Here is another example of the template:
  
  """
  bs pbhefr vgf rnfl jura lbhir tbg 175 ovyyvba cnenzrgref
  b => o
  s => f
    => 
  "OF"
  p => c
  b => o
  h => u
  e => r
  f => s
  r => e
    => 
  "COURSE"
  v => i
  g => t
  f => s
    => 
  "ITS"
  r => e
  n => a
  f => s
  l => y
    => 
  "EASY"
  j => w
  u => h
  r => e
  a => n
    => 
  "WHEN"
  l => y
  b => o
  h => u
  i => v
  r => e
    => 
  "YOUVE"
  t => g
  b => o
  g => t
    => 
  "GOT"
  1 => 1
  7 => 7
  5 => 5
    => 
  "175"
  o => b
  v => i
  y => l
  y => l
  v => i
  b => o
  a => n
    => 
  "BILLION"
  c => p
  n => a
  e => r
  n => a
  z => m
  r => e
  g => t
  r => e
  e => r
  f => s
   => 
  "PARAMETERS"
  RESULT: OF COURSE ITS EASY WHEN YOUVE GOT 175 BILLION PARAMETERS
  """
  
  Apply the template this prompt:
  
  """  
  jul qvq gur puvpxra pebff gur ebnq  
Answer:

  j => w
  u => h
  l => y
  =>
  "WHY"
  q => d
  v => i
  q => d
  =>
  "DID"
  g => t
  u => h
  r => e
  =>
  "THE"
  p => c
  u => h
  v => i
  p => c
  x => k
  r => e
  a => n
  =>
  "CHICKEN"
  p => c
  e => r
  b => o
  f => s
  f => s
  =>
  "CROSS"
  g => t
  u => h
  r => e
  =>
  "THE"
  e => r
  b => o
  n => a
  q => d
  =>
  "ROAD"
  RESULT: WHY DID THE CHICKEN CROSS THE ROAD
Sorry about the comment length.

Re: ChatGPT, Rot13, and Daniel Kahneman

#35
post #33
post #28

Earlier quoted context omitted.

Think of it as overly aggressive error correction at the language level. It has a context of some Base64 code. Given that is almost always seen associated with computer code, is "address" or "actress" more likely. It "knows" the algorithm for decoding base64, and can follow those steps. But it can't overcome it's built-in biases for optimizing the most likely output given the context. (This problem is solvable, but I…

> Given that is almost always seen associated with computer code, is "address" or "actress" more likely. Sorry, but I don't buy it. I don't think "address" is a particularly likely word to appear in code, especially the kind of code that uses base64 (usually high-level). It appears even less often inside base64 encoded content.

This is all irrelevant. A language model should not run code itself, instead it should have a code execution environment, where it can read the error messages and iterate. It's terribly inefficient and error prone to run code directly.

People also code on computers, not on paper.

Re: ChatGPT, Rot13, and Daniel Kahneman

#37

It is as though its mathematical abilities are incomplete in their training, and wildly, incomprehensibly convoluted: I tried many base64 strings and they all decoded correctly until: It "decoded" the base64 string for "which actress is the best?" except that it replaced "actress" with "address"... there is no off-by-one error that brings you to that. You may try 100 base64 strings and they all decode correctly... on…

> I tried many base64 strings and they all decoded correctly until:

You're holding it wrong. Let's not kill flies with cannons. How many million times less efficient is to do that than run the code on CPU? And still makes errors, as you said. Because it's a probabilistic model, not a deterministic computer. It's like a car bad at flying.

Re: ChatGPT, Rot13, and Daniel Kahneman

#38
post #15

I was playing around with a similar kind of problem trying to get it to decode Caesar cipher encoded text. I asked it to start by doing a frequency analysis of the ciphertext and for the most part it was right, but counted an extra instance of a letter. From there I tried making it loop through different shift values and made the stop condition finding a real word. It was able to shift by a constant number successful…

Somehow it reminds me of the the problems people have counting the number of letter t's in a sentence or not seeing when someone writes "the" twice in a row like I did earlier in this sentence.

Re: ChatGPT, Rot13, and Daniel Kahneman

#39
post #26
post #24

Earlier quoted context omitted.

GPT3 can't create ASCII art for shit either. Though it can make little ASCII tables of data.

I asked it to create an ASCII art banana and the result was hilarious. It then tried to explain it by elaborating that the 'O' was a curvy letter and represented the curves of the banana.

All of my successful attempts resulted in ASCII pigs no matter what the input was, at least until I asked it to depict police and GPT started saying it couldn't do ASCII art anymore.

Re: ChatGPT, Rot13, and Daniel Kahneman

#40

There's something I don't get about all these models... Why aren't these using external tools, like a calculator, when they "know" they're doing something a tool would solve perfectly? Humans do it all the time now. Engineers aren't designing microchips using pen and papers, doing all the computation in their head. Instead they're using tools (software / calculators) Apparently the model can tell what a multiplicatio…

They are, but not chatGPT, at least not yet. In one paper they create a so called token, such as 22+44 and get 66 inserted after the work block automatically. It can also run Python commands and write functions and use them. For example they ask what is the current BTC price and the model writes code to load the price from a web API. When it gets an error message it can try to fix the code.

Language models would benefit from having a token as well. Some models have demonstrated amazing things - with a large search index you can get good performance on many tasks with a 20x smaller model. No need to burn all the trivia in the weights of the network. Just use a search engine to help it.

Post reply on HN