Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I asked GPT-NeoX-20B a hundred arithmetic questions

twitter.com

Post reply on HN
Print view View original

211–220 of 235 posts

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#211

Post by infogulch »

moyix wrote: post #148

Hey! As the author of the gist, just wanted to clear up what seem to be a few misconceptions: - This isn't GPT-3, it's the recently-released open-source and open-weights model from EleutherAI, GPT-NeoX-20B. GPT-3 is much larger (175 billion parameters vs NeoX's 20 billion). - It's well-known that language models don't tend to be good at math by default (Gwern, among others, pointed this out back in June 2020). It see…

Would it do better if you asked it to "show its work"? I.e. work it out in long form, one step at a time, like you'd ask a school kid to do. Maybe an example prompt would look like this:

    Work out 2241 + 19873.
    02241 + 19873 ~ ____4
    02241 + 19873 ~ ___14 carry 1
    02241 + 19873 ~ __114 carry 1
    02241 + 19873 ~ _2114 carry 1
    02241 + 19873 = 22114.
I'm not sure what is the best way to represent each step including details like carry digits. And you'd have to design a separate scheme for each operation.

If these models are symbol manipulators maybe the key is to break down the task into steps that are closer to being solvable with symbol manipulation.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#212

Post by YeGoblynQueenne »

moyix wrote: post #193
YeGoblynQueenne wrote: post #189

Earlier quoted context omitted.

Don't worry, I won't yell at you :) I'm fine with "invent" actually, despite the implication of agency (I'm used to the terminology "predicate invention" [1]; although maybe I should actually re-examine the motivation behind it). I'm more interested in the representation issue. I had a look at the quoted article on CNNs earlier. I think there is a very fine line between claiming that a CNN's weights represent an algo…

> I think there is a very fine line between claiming that a CNN's weights represent an algorithm and that its weights can be _interpreted_ as an algorithm. Yeah, I agree this is an issue. It feels a bit reminiscent of Searle's Waterfall argument, and so I'm inclined to turn to Scott Aaronson's response here [1; Section 6] – basically, how much work is the interpretation itself doing? If you actually tried to use the…

Thanks for the link to Scott Aaronson's paper, that I hadn't read. I think your comment helped clarify something that bothered me with the CNN paper: I would feel more convinced if the claimed algorithm had been implemented _without_ a neural net. What the authors did was manually set the weights of another neural net. If an algorithm is discerned in the neural net's weights, then why can't it be programmed in the usual manner, in some computer language? If it can't, then that's very difficult for me to accept it as an algorithm, because I can't see what it is, exactly. The authors claim that the algorithm can be described in a few English sentences, but in truth these few English sentences are a high-level description of the behaviour of the CNN rather than a sequence of instructions (which is what I have in mind, perhaps erroneously, as an "algorithm").

I'm not necessarily asking for simplicity. I'm used to algorithms being relatively simple things that can be implemented in a few lines of code and I'd think of something more extensive as a "program" rather than an algorithm, but I appreciate that an algorithm encoded in the weights of a deep neural net could be something really big. I just want to see this algorithm written down in pseudocode at least, in a form that can be executed by a device other than a neural network (like me, following the pseudocode). I think that is the opposite of Aaronson's point actually.

I think I might be a realist:

https://youtu.be/STFcvzoxVw4?t=97

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#213

Post by jcims »

wallacoloo wrote: post #185
ceejayoz wrote: post #88

Earlier quoted context omitted.

> How did you arrive at that answer? This sort of question - "you don't remember what I just asked you?" - is one of the most glaring failings in Apple's Siri and I hope this leads to it being solved soon.

it could be interesting to gauge how entwined the “how did you arrive at that answer” process is with the answering itself. i.e. which paths do they share? even at this early a stage: is there some structure which is used to determine the operand(s) that’s leveraged in both of these prompts? is the “how did you X” answer leveraging most of the “X” circuitry and just bailing out early? or does it deviate as early as p…

My initial guess is they have nothing to do with each other. It would be like explaining why the next idea pops in your head. You can create a rational explanation but there's no way to test it.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#214

Post by Veedrac »

I fiddled with this prompt

    Calculate 721*436

    Step A: Multiplication table.
    721*0=0 721*1=721 721*2=1,442 721*3=2,163
    721*4=2,884 721*5=3,605 721*6=4,326
    721*7=5,047 721*8=5,768 721*9=6,489
    Step B: Multiply digits
    436=4 hundreds + 3 tens + 6 ones
    721*4 hundreds=2,884*100=288,400
    721*3 tens=2,163*10=21,630
    721*6 ones=4,326*1=4,326
    Step C: Add the first two
    288,400+21,630=310,030
    Then add the last one
    310,030+4,326=314356
To get GPT-3's answer to this one correct (this is its answer, but it took some trial and error with the prompt to get it to stop making small mistakes)

    Calculate 312*481

    Step A: Multiplication table.
    312*0=0 312*1=312 312*2=624 312*3=936
    312*4=1,248 312*5=1,560 312*6=1,872
    312*7=2,184 312*8=2,496 312*9=2,808
    Step B: Multiply digits
    481=4 hundreds + 8 tens + 1 ones
    312*4 hundreds=1,248*100=124,800
    312*8 tens=2,496*10=24,960
    312*1 ones=312*1=312
    Step C: Add the first two
    124,800+24,960=149,760
    Then add the last one
    149,760+312=150,072
I then tested it with two random 3-digit numbers

    Calculate 669*306

    Step A: Multiplication table.
    669*0=0 669*1=669 669*2=1,338 669*3=2,007
    669*4=2,676 669*5=3,345 669*6=4,014
    669*7=4,683 669*8=5,352 669*9=6,021
    Step B: Multiply digits
    306=3 hundreds + 0 tens + 6 ones
    669*3 hundreds=2,007*100=200,700
    669*0 tens=0*10=0
    669*6 ones=4,014*1=4,014
    Step C: Add the first two
    200,700+4,014=204,714
    Then add the last one
    204,714+306=205,020
204,714 is correct, but then it gets confused in the very last line because of the zero.

Anyway this, I think, proves that GPT-3 can do zero-shot math if it doesn't have to do it in its head all at once. Is it good for a calculator? No. Is it good for a language model trained on web text? Yes, absolutely.

E: Correcting the start of Step C in that last example to “200,700+0”, and replacing “XYZ=X hundreds + Y tens + Z ones” with “XYZ=X Y Z=X hundreds + Y tens + Z ones” allowed it to do 145*585, 961*761 and 592*555 correctly in a row, all randomly chosen, and at least the last two tried without changes to the prompt. I consider this an adequate test, and it demonstrates GPT-3's algorithm following abilities. As GPT-3 is still a tiny model, this seems important to note.

E2: To be clear this is still nowhere near 100% successful. GPT-3 still makes a lot of errors. I ran 100 tries of a slightly different prompt through the API, and got a success rate of 42%.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#215

Post by Veedrac »

infogulch wrote: post #211
moyix wrote: post #148

Hey! As the author of the gist, just wanted to clear up what seem to be a few misconceptions: - This isn't GPT-3, it's the recently-released open-source and open-weights model from EleutherAI, GPT-NeoX-20B. GPT-3 is much larger (175 billion parameters vs NeoX's 20 billion). - It's well-known that language models don't tend to be good at math by default (Gwern, among others, pointed this out back in June 2020). It see…

Would it do better if you asked it to "show its work"? I.e. work it out in long form, one step at a time, like you'd ask a school kid to do. Maybe an example prompt would look like this: Work out 2241 + 19873. 02241 + 19873 ~ ____4 02241 + 19873 ~ ___14 carry 1 02241 + 19873 ~ __114 carry 1 02241 + 19873 ~ _2114 carry 1 02241 + 19873 = 22114. I'm not sure what is the best way to represent each step including details…

I tried something like that for 3-digit multiplication with GPT-3 in another comment[1], successfully. You have to lay things out different manner than you did here, because GPT-*s have no sense of layout on a page; their byte-pair encoding destroys their ability to learn it efficiently. Further, transformers are optimized to look for things via similarity, because that's what typically occurs in text, so you're better off writing out things it can anchor off of.

There are ways to fix these issues, but BPEs micro-optimize for the primary text benchmarks that papers want good scores on so those are standard for now. I'm sure they'll get replaced eventually, once the costs outrun the wins and more scalable (alternatives to?) transformers become popular.

[1] https://news.ycombinator.com/item?id=30309302

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#216

Post by infogulch »

Veedrac wrote: post #214

I fiddled with this prompt Calculate 721*436 Step A: Multiplication table. 721*0=0 721*1=721 721*2=1,442 721*3=2,163 721*4=2,884 721*5=3,605 721*6=4,326 721*7=5,047 721*8=5,768 721*9=6,489 Step B: Multiply digits 436=4 hundreds + 3 tens + 6 ones 721*4 hundreds=2,884*100=288,400 721*3 tens=2,163*10=21,630 721*6 ones=4,326*1=4,326 Step C: Add the first two 288,400+21,630=310,030 Then add the last one 310,030+4,326=3143…

> can do zero-shot math if it doesn't have to do it in its head all at once

Very interesting! This is what I would expect. It can run a symbolic algorithm fine, just give it some scratch space to work out the intermediate results. I feel like there's a very large space to optimize the layout "algorithm" -- like how you adjusted step c -- to produce reliable results.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#217

Post by wallacoloo »

jcims wrote: post #213
wallacoloo wrote: post #185

Earlier quoted context omitted.

it could be interesting to gauge how entwined the “how did you arrive at that answer” process is with the answering itself. i.e. which paths do they share? even at this early a stage: is there some structure which is used to determine the operand(s) that’s leveraged in both of these prompts? is the “how did you X” answer leveraging most of the “X” circuitry and just bailing out early? or does it deviate as early as p…

My initial guess is they have nothing to do with each other. It would be like explaining why the next idea pops in your head. You can create a rational explanation but there's no way to test it.

my thoughts too, based on limited understanding of GPT. but the more pressure you apply towards compressing the neural network during training, the more circuitry these paths are likely to share. it would be interesting to see just how much and which parts could be folded together before you start to lose significant fidelity (though unfortunately the fidelity seems too low today to even try that).

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#218

Post by jonathankoren »

rasen58 wrote: post #181
jonathankoren wrote: post #179

Earlier quoted context omitted.

No. A million times no. It’s a language model. It doesn’t understand math at all. It doesn’t even understand language. All it did was spit out something that looks like math. It’s fancy automatic writing. I’ll concede that if you tokenized the equations correctly, you might be able to get a language model to learn arithmetic, since it’s just symbol manipulation; but to make the leap that a general text model has lear…

Isn’t your comment that you wrote here also just a bunch of “symbol manipulation“? It definitely hasn’t learned math but it definitely has learned general concepts

1) No. Because I didn’t compute anything. This is the result of cognition. There’s a difference. If you think there isn’t, the burden of proof is on you show that they’re the same, as this has never been the dominate belief either now, nor for the last thousands of years.

2) What general concept has it learned? You can’t pull any fact consistently out of these things, because they don’t actually have a model of a world. They have statistical correlations between words. There’s no logical inference. They’re just Eliza.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#219

Post by jonathankoren »

arghwhat wrote: post #16
nikolayasdf123 wrote: post #8

This just shows that this model did not learn anything. Humans do not see billions of examples to add numbers. We see just few and can apply learned notation and procedures to infinity with 100% precision. GPT-3 learned mathematical intuition. Humans can hardly learn multiplication table over months and repetitions of same examples, and that table hardly matters at all. GPT-3 is just plainly wrong objective they tryi…

I think you'd find that most people doing large number math in their head is also off by a few percent like this model. Sure, with pen and paper we can follow specific algorithms manually to very slowly get a precise result. If we wanted a computer to merely follow instructions, then I suspect that there are better ways...

You’re really lowering the bar for success here. It’s now unreasonable for a computer to correctly add two numbers together? Give me a break. It wasn’t even reasonable for a Pentium chip to incorrectly divide two numbers back in 1994.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#220

Post by schleck8 »

syntaxing wrote: post #133
schleck8 wrote: post #93

Earlier quoted context omitted.

I tried this with the new open source GPT NeoX 20B model on GooseAI playground and it suceeded too: Q: Can here you here still here understand here what here I here am here asking here if here I here put here words here in here between? A: Yes. Q: What was the word that I put in between? A: The word was "here." It's capable of other languages as well, although not quite as fluent. Q: Kannst du auch Deutsch sprechen?…

Did you try this on GooseAI? I was not able to replicate this Edit: oops just noticed you mentioned GooseAI, what settings did you use?

If I recall correctly, I dialed up the penalties for repetition and something else. Otherwise it would often generate the same sentence multiple times.
Post reply on HN