Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I asked GPT-NeoX-20B a hundred arithmetic questions

twitter.com

Post reply on HN
Print view View original

71–80 of 235 posts

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#71

Post by jcims »

sillysaurusx wrote: post #36

If you want to play with the model, you can (with difficulty) for free at https://goose.ai/playground . You have to log in, but thankfully you can via google. The playground crashes every minute, and the defaults ruin your outputs (temperature 1, really? 0.7 to 0.8 is a necessity, with top-k 40), and they turned off autocorrect on mobile, presumably because they hate you and your family for owning an iPad, but you ca…

For what it's worth you can get access to the OpenAI playground. You might have to put yourself on a waiting list.

https://beta.openai.com/playground

I tried a couple different things to pick it apart. I typed everything except for the text after each A: (for reference 888 * 888 is 788544)

    I am a math savant, I can do arithmetic in my head. 
    I will provide an answer, then show how I arrived at 
    the answer.

    Q: What is 8+8?
    A: 16
    
    Q: How did you arrive at that answer?
    A: 8+8=16. 16 is two groups of 8.
    
    Q: What is 888+888?
    A: 1776
    
    Q: How did you arrive at that answer?
    A: 888+888=1776. 1776 is two groups of 888.
    
    Q: What is 888*2?
    A: 1776
    
    Q: How did you arrive at that answer?
    A: 888*2=1776. 1776 is two groups of 888.
    
    Q: What is 888*10?
    A: 8800
    
    Q: How did you arrive at that answer?
    A: 888*10=8800. 8800 is two groups of 888*10.
    
    Q: What is 888*888?
    A: 775648
    
    Q: How did you arrive at that answer?
    A: 888*888=775648. 775648 is two groups of 888*888.
    
    Q: What is 888 groups of 888?
    A: 775648

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#72

Post by lucb1e »

sillysaurusx wrote: post #36

If you want to play with the model, you can (with difficulty) for free at https://goose.ai/playground . You have to log in, but thankfully you can via google. The playground crashes every minute, and the defaults ruin your outputs (temperature 1, really? 0.7 to 0.8 is a necessity, with top-k 40), and they turned off autocorrect on mobile, presumably because they hate you and your family for owning an iPad, but you ca…

> You have to log in, but thankfully you can via google.

... thankfully you can also not login via Google.

Anyway it doesn't work even slightly. When I simply try 4+4= it prints some source code afterwards. A second attempt wrote some printf() call which computed 6+1 instead (but not giving the result, just the code). When I try what someone else shared below with a Q: and A: syntax I get this:

Q: What is 8 + 8? \n A: \n The answer looks confusing. \n How do you work out 8 + 8. What is the rule?

Q: What is 8+8? \n A:

I got the answer as 12, so it is 1+2.

(I added some \newlines because HN doesn't support markdown syntax for line breaks)

On the right are some parameters, including the model name. There is no GPT-3, instead there are options like GPT-NeoX 20B (don't know how different that is). Maybe that's the problem, or it needs a different ...temperature?

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#73

Post by ggrrhh_ta »

sillysaurusx wrote: post #68
ggrrhh_ta wrote: post #62

Earlier quoted context omitted.

I pointed it out above; even though it is text, the ASCII representation is just a different base for the numbers - base 2^8 - ('325' is '3' * (2^32) + '2' * (2^16) + '5' * 2^8 = 51 * 2^32 + 50 * 2^16 + 53 * 2^8); it should approximate those polynomial functions very well.

Hmm. I’m not sure what you mean. Temperature is randomness; low temp is to get the most probable least random result. It’s what chess engines do during tournaments, for example. The other parts seem unlikely. It has no knowledge of bases, except insofar as they appear in the training set. I saw this in our GPT chess work — even with strange tokenization, it learned chess notation well.

Sorry, I thought it was clear. A neural network, when untrained is just random noise that multiplies inputs by random weights over an over (+ normalization) until it reaches the output. When you train it with inputs whose outputs have are the process of applying some polynomial to those inputs, the weights can be set so that the output very closely approximates that polynomial. It never needs to know the base, and less randomness will help because the computations within the neural network match very well with the function you want to approximate. Still, it is not as simple, as outputting the correct ASCII representation is a challenge for example when carry is involved (100009999999999 + 1), however, the emergence of good arithmetic from a neural network itself should not be shocking.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#74

Post by sillysaurusx »

lucb1e wrote: post #72
sillysaurusx wrote: post #36

If you want to play with the model, you can (with difficulty) for free at https://goose.ai/playground . You have to log in, but thankfully you can via google. The playground crashes every minute, and the defaults ruin your outputs (temperature 1, really? 0.7 to 0.8 is a necessity, with top-k 40), and they turned off autocorrect on mobile, presumably because they hate you and your family for owning an iPad, but you ca…

> You have to log in, but thankfully you can via google. ... thankfully you can also not login via Google. Anyway it doesn't work even slightly. When I simply try 4+4= it prints some source code afterwards. A second attempt wrote some printf() call which computed 6+1 instead (but not giving the result, just the code). When I try what someone else shared below with a Q: and A: syntax I get this: Q: What is 8 + 8? \n A…

Well yes, if you do things in slightly different ways, you’ll get hugely different results. It’s a bit like saying you added bacteria to a controlled experiment on sanitization.

Try with temp 0.1, with exactly the text in the OP, including capitalization.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#75

Post by starfallg »

mordymoop wrote: post #65

When you toss “2241 + 19873 =” into an applet that shows you the default tokenization scheme GPT-3 uses, you get this: (224)(1)( +)( 198)(73)( =) I’ve heard it remarked before that, while tokenization is obviously an unavoidable part of a model with an architecture like GPT, this is a very silly way of tokenizing number strings for the purposes of learning or doing arithmetic. Indeed, I think a lot of GPT-3’s puzzlin…

It seems that we need another layer to tokenize according to context. I can see that breaking up a long number into 3 or 4 digits is the correct behaviour if we are dealing with phone numbers, but it'd be completely wrong if it's nearly anything else.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#76

Post by sillysaurusx »

ggrrhh_ta wrote: post #73
sillysaurusx wrote: post #68

Earlier quoted context omitted.

Hmm. I’m not sure what you mean. Temperature is randomness; low temp is to get the most probable least random result. It’s what chess engines do during tournaments, for example. The other parts seem unlikely. It has no knowledge of bases, except insofar as they appear in the training set. I saw this in our GPT chess work — even with strange tokenization, it learned chess notation well.

Sorry, I thought it was clear. A neural network, when untrained is just random noise that multiplies inputs by random weights over an over (+ normalization) until it reaches the output. When you train it with inputs whose outputs have are the process of applying some polynomial to those inputs, the weights can be set so that the output very closely approximates that polynomial. It never needs to know the base, and le…

You are clear but mistaken.

I give you points for creative thinking, but it’s important not to make inferences that “feel correct.” No matter what your gut is telling you, I would happily bet $10k that the emergence of arithmetic has nothing to do with the things you mention.

If an alternative training scheme were devised that didn’t rely on any of that, it would still result in a model that behaved more or less the same as what we see here. The properties of the training process influence the result, but they don’t cause the result — that would be like saying your vocal cords cause you to be an excellent orator. Vocal cords don’t form the ideas; the training process doesn’t form the arithmetic.

What we’re seeing is a consequence of a large training dataset. The more tasks a model can perform, the better it is at any individual task.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#77

Post by mlb_hn »

Couple things there where you can see if it improves with the prompt/formatting. E.g. with Davinci (and J a bit but didn't test too much) you can get bette results by:

  - Using few-shot examples of similar length to the targets (e.g. 10 digit math, use 10 digit few shots)

  - Chunking numbers with commas

  - Having it double check itself
and here it's not doing any of those things.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#78

Post by Isinlor »

YeGoblynQueenne wrote: post #59
Isinlor wrote: post #18

Earlier quoted context omitted.

Sure: Deep Symbolic Regression for Recurrent Sequences https://arxiv.org/abs/2201.04600 (Interactive demo: http://recur-env.eba-rm3fchmn.us-east-2.elasticbeanstalk.com... ) Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets https://arxiv.org/abs/2201.02177 Both of these models can generalize to numbers it have not seen.

As far as I can tell from a quick heuristic perusal, the "Generalization Beyond Overfitting" paper reports "generalisation" _on the validation set_. That's not particularly impressive and it's not particularly "generalisation" either. Actually, I really don't grokk this (if I may). I often see deep learning work reporting generalisation on the validation set. What's up with that? Why is generalisation on the validati…

The point of the paper is to show that NN can still learn long after fully memorizing the train dataset.

This behavior goes against current paradigm of thinking about training NNs. It is just very unexpected, similarly as double descent is unexpected from classical statistics point of view that more parameters lead to more over-fitting.

They could have split validation test set into validation and test sets, but I don't know what that would achieve in their case.

Fig. 1 center shows different train / validate splits. Fig 2. shows a swoop between different optimization algorithms if you are concerned about hyperparameters over-fitting.

But to me really interesting is the Fig 3. that shows that NN learned the structure of the problem.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#79

Post by vbuterin »

mordymoop wrote: post #65

When you toss “2241 + 19873 =” into an applet that shows you the default tokenization scheme GPT-3 uses, you get this: (224)(1)( +)( 198)(73)( =) I’ve heard it remarked before that, while tokenization is obviously an unavoidable part of a model with an architecture like GPT, this is a very silly way of tokenizing number strings for the purposes of learning or doing arithmetic. Indeed, I think a lot of GPT-3’s puzzlin…

When I do mental arithmetic my brain frequently tokenizes into digit pairs or triples if I can recognize pairs and triples that have specific properties.

"224" is actually a really nice object to recognize because it's 7 * 32, and if you can recognize other multiples of 32 it frequently gives you shortcuts. It's less useful for addition because you would need to get lucky and get a multiple of 32 (or 7) on both sides, but for multiplication and division it helps a lot.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#80

Post by vbuterin »

d--b wrote: post #24

What? You think this is poor performance? This totally blows my mind. I would never have guessed that GPT could get ANY of these right. I mean, is there a data point in the dataset used to train where you can read 2241 + 19873 = 22114? Quite unlikely... And those multiplications. It's consistently getting the number of digits right and the first two numbers correct. How the hell does this happen? Sure, it's sometimes…

What fascinates me most is that the errors are very "human-like". If you gave me multi-digit multiplication and addition problems like that, I would frequently have similar results of getting most digits right but making a mistake on one or a few of them.
Post reply on HN