Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI language models are struggling to “get” math

spectrum.ieee.org

Post reply on HN
Print view View original

191–200 of 201 posts

Re: AI language models are struggling to “get” math

#191

Post by bloep »

p1esk wrote: post #189
bloep wrote: post #188

Earlier quoted context omitted.

What I had in mind was kind of like a reward model that is trained by on longer outputs that have a very high similarity to training examples. Something similar has been done to prevent LLMs from using toxic language. You'd simply backprop through that model like in GANs. And no it does not contradict the overall training objective completely because the criterion would be long verbatim copies and it would not affect…

Oh OK, so you mean training the model after it has already been trained on the main task, right? Like finetuning. Yes, I think the GAN-like finetuning is a good idea. Though it's less clear where the labels would come from, it seems like some sort of fingerprint would need to be computed for each generated sequence, and this fingerprint would need to be compared against a database of fingerprints for every sequence i…

You'd need something Spotify.

Another similar possibility might be to do more RL with this data, e.g. using upside-down RL. One can possibly steer this with user feedback as well.

Re: AI language models are struggling to “get” math

#192

Post by 1024core »

mjburgess wrote: post #170
1024core wrote: post #144

Earlier quoted context omitted.

See the "joke explaining" examples on Page 36 of the paper: https://arxiv.org/abs/2204.02311 and tell me if it's just "passable garbled nonsense".

> 500 bn parameter model That solution has a compressed representation of half the internet. NNs are "garabled nonesense" insofar as they try to generalise; insofar as they are search engines, they provide apparent sense by just repeating something in their database (= weights). Google, likewise, returns joke explanations.

"The Internet" is much, much bigger than 500bn floats. The latest snapshot of archive.org has 505 billion web pages itself.

Re: AI language models are struggling to “get” math

#193

Post by nl »

the_af wrote: post #179
nl wrote: post #133

Earlier quoted context omitted.

> Translating ambiguous symbols to meaning and combining them in meaningful ways is a big deal which, apparently, these AI models cannot do. Plenty of AI models do exactly this. Very clear examples include question answering models and code generation. In both cases novel, meaningful responses are generated. > They can just parrot things. That isn't true. While language models can parrot things it is generally specia…

So is in your opinion General AI solved? Because reliably turning symbols into meaning, outside narrow or special cases, is General AI. In my opinion, it's not solved. GPT-3 is not General AI, it's a more clever mechanism for parroting back text it cannot truly understand. Comparisons to ways humans confuse themselves are a red herring in my opinion: the old ELIZA program could reply like a very confused or trollish…

I don't think "general intelligence" is a bright-line, but instead is a continuum, and I don't agree with your definition (although I appreciate you do at least give a definition).

I think that in general most human decision making is just pattern matching (plenty of evidence for this - read "thinking fast and slow" for an overview).

I think the extrapolation that ML models can do is a form of intelligence. I also think that the compression and encoding of inputs into a lower dimensional space is exactly the "turning symbols into meaning" that you call for in your definition.

Re: AI language models are struggling to “get” math

#194

Post by _0ffh »

_0ffh wrote: post #122

And no wonder, as they correspond much closer to a Kahneman system 1 than system 2, where we do most of our math.

Btw, here's an example of how even a very simple zero-shot/prompt-engineering attempt to introduce a bit of system 2 reasoning into a language model can improve results.

https://arxiv.org/abs/2201.11903

Re: AI language models are struggling to “get” math

#195

Post by visarga »

spywaregorilla wrote: post #139
visarga wrote: post #132

Earlier quoted context omitted.

Brains weren't built for math either, just for surviving. And the "trying to do math in your head" is true if you use naive question answering, but if you ask "step by step" or "chain of thought", or "supporting questions", any of them will allow for flexible time steps. There are some solutions called "Language Model Cascades" that compose language models calls to simulate arbitrary complex reasoning chains includin…

> Brains weren't built for math either, just for surviving. Brains were, however, built for language processing, in addition to many other tasks. > There is no reason to think language models are unfit for math, they are fit for generating possible solutions that need to be verified somehow. This is just a dumb idea though. Guess and check based on semantically well positioned answers in the ambiguity that is the emb…

> This is just a dumb idea though.

True, but that's how "inspiration" works in humans as well: generate stupid ideas until you stumble upon a great one.

That's why I said we only need verification. It's the artist-critic model, we got the artist we need the critic. Sometimes it's easy (in games, code, math) and other times we don't have a good way to verify.

Re: AI language models are struggling to “get” math

#196

Post by spywaregorilla »

visarga wrote: post #195
spywaregorilla wrote: post #139

Earlier quoted context omitted.

> Brains weren't built for math either, just for surviving. Brains were, however, built for language processing, in addition to many other tasks. > There is no reason to think language models are unfit for math, they are fit for generating possible solutions that need to be verified somehow. This is just a dumb idea though. Guess and check based on semantically well positioned answers in the ambiguity that is the emb…

> This is just a dumb idea though. True, but that's how "inspiration" works in humans as well: generate stupid ideas until you stumble upon a great one. That's why I said we only need verification. It's the artist-critic model, we got the artist we need the critic. Sometimes it's easy (in games, code, math) and other times we don't have a good way to verify.

> True, but that's how "inspiration" works in humans as well: generate stupid ideas until you stumble upon a great one.

I don't agree with this, but even if it were true, "inspiration" is not how we do math.

> That's why I said we only need verification. It's the artist-critic model, we got the artist we need the critic. Sometimes it's easy (in games, code, math) and other times we don't have a good way to verify.

That's not even how the typical language model works.

Re: AI language models are struggling to “get” math

#197

Post by the_af »

nl wrote: post #193
the_af wrote: post #179

Earlier quoted context omitted.

So is in your opinion General AI solved? Because reliably turning symbols into meaning, outside narrow or special cases, is General AI. In my opinion, it's not solved. GPT-3 is not General AI, it's a more clever mechanism for parroting back text it cannot truly understand. Comparisons to ways humans confuse themselves are a red herring in my opinion: the old ELIZA program could reply like a very confused or trollish…

I don't think "general intelligence" is a bright-line, but instead is a continuum, and I don't agree with your definition (although I appreciate you do at least give a definition). I think that in general most human decision making is just pattern matching (plenty of evidence for this - read "thinking fast and slow" for an overview). I think the extrapolation that ML models can do is a form of intelligence. I also th…

Thanks for the conversation.

We obviously disagree. I don't think we are near general AI (and yes, I know the objection that everyone who says this is simply moving the goalposts-- regardless, I'm unconvinced). I think GPT et al are very interesting tricks, but still not general AI; and that the path to it doesn't lie in this direction.

I subscribe to the view we think of the human mind as a pattern matching computer simply because this is the current major tech, much like people in the past thought of "humors" or "steam machines". I think some of the analogies are useful, to a point, but I don't think there's hard evidence the mind is like a neural net (irony notwithstanding) or a pattern marching GPT-like algorithm.

Re: Thinking Fast and Slow, I see there are serious doubts about the validity of the book's foundations and conclusions, and that it's been challenged.

Re: AI language models are struggling to “get” math

#198

Post by nl »

the_af wrote: post #197
nl wrote: post #193

Earlier quoted context omitted.

I don't think "general intelligence" is a bright-line, but instead is a continuum, and I don't agree with your definition (although I appreciate you do at least give a definition). I think that in general most human decision making is just pattern matching (plenty of evidence for this - read "thinking fast and slow" for an overview). I think the extrapolation that ML models can do is a form of intelligence. I also th…

Thanks for the conversation. We obviously disagree. I don't think we are near general AI (and yes, I know the objection that everyone who says this is simply moving the goalposts-- regardless, I'm unconvinced). I think GPT et al are very interesting tricks, but still not general AI; and that the path to it doesn't lie in this direction. I subscribe to the view we think of the human mind as a pattern matching computer…

> I don't think we are near general AI

Nor do I. But I don't agree with your definition of general AI at all.

I think by your definition we are well on the way towards it. I'd note that you didn't address the idea that compression of input concepts into a lower dimensional space is exactly the "reliably turning symbols into meaning" idea you suggest.

The fact the latent representations of concepts can be manipulated in ways that make logical sense is a good indication that the symbols have meaning. The classic Word2Vec experiments showing how the relationships Paris->France ~= London->England and King - Man + Woman = Queen show this well. Modern large language models are much more complicated of course, but the principles remain.

> I subscribe to the view we think of the human mind as a pattern matching computer simply because this is the current major tech, much like people in the past thought of "humors" or "steam machines". I think some of the analogies are useful, to a point, but I don't think there's hard evidence the mind is like a neural net (irony notwithstanding) or a pattern marching GPT-like algorithm.

I don't think the computational approach is an interesting question. No one thinks brains operate like software neural networks (not sure what "pattern marching GPT-like algorithm" means - it is just a neural network). That doesn't matter because all computational methods are ultimately equivalent.

I think outcomes on metrics like benchmarks is important, and while some benchmarks have issues that some approaches exploit I think things like Chollet's "On the Measure of Intelligence" (https://arxiv.org/abs/1911.01547) are reasonable frameworks for discussion of progress.

> Re: Thinking Fast and Slow, I see there are serious doubts about the validity of the book's foundations and conclusions, and that it's been challenged.

This in inaccurate. There are some reasonable criticisms of the studies on the priming affect (one chapter of the book). These and Kahneman's response (he accepts the criticisms) are outlined in https://replicationindex.com/category/thinking-fast-and-slow... and https://retractionwatch.com/2017/02/20/placed-much-faith-und...

These issues don't detract from the overall theme of the book about the two systems of decision making (rational and reflex) and how often we use the reflex decision system but convince ourselves we are using the rational system.

If you can point at any additional doubts about the books conclusions I'd appreciate a reference.

Re: AI language models are struggling to “get” math

#199

Post by the_af »

nl wrote: post #198
the_af wrote: post #197

Earlier quoted context omitted.

Thanks for the conversation. We obviously disagree. I don't think we are near general AI (and yes, I know the objection that everyone who says this is simply moving the goalposts-- regardless, I'm unconvinced). I think GPT et al are very interesting tricks, but still not general AI; and that the path to it doesn't lie in this direction. I subscribe to the view we think of the human mind as a pattern matching computer…

> I don't think we are near general AI Nor do I. But I don't agree with your definition of general AI at all. I think by your definition we are well on the way towards it. I'd note that you didn't address the idea that compression of input concepts into a lower dimensional space is exactly the "reliably turning symbols into meaning" idea you suggest. The fact the latent representations of concepts can be manipulated…

> I don't think the computational approach is an interesting question.

Great, so we agree then!

> No one thinks brains operate like software neural networks

A bold claim.

> (not sure what "pattern marching GPT-like algorithm" means - it is just a neural network).

Yet you understood me perfectly.

Anyway, you are more well read than me on this subject, yet we both agree we are not near general AI.

Re: AI language models are struggling to “get” math

#200

Post by nl »

the_af wrote: post #199
nl wrote: post #198

Earlier quoted context omitted.

> I don't think we are near general AI Nor do I. But I don't agree with your definition of general AI at all. I think by your definition we are well on the way towards it. I'd note that you didn't address the idea that compression of input concepts into a lower dimensional space is exactly the "reliably turning symbols into meaning" idea you suggest. The fact the latent representations of concepts can be manipulated…

> I don't think the computational approach is an interesting question. Great, so we agree then! > No one thinks brains operate like software neural networks A bold claim. > (not sure what "pattern marching GPT-like algorithm" means - it is just a neural network). Yet you understood me perfectly. Anyway, you are more well read than me on this subject, yet we both agree we are not near general AI.

>> I don't think the computational approach is an interesting question.

> Great, so we agree then!

Maybe? I think all forms of computation are equivalent, so if the brain is implemented in the same way as a neural network is uninteresting. They can do the same thing (which is interesting).

> yet we both agree we are not near general AI.

Sure. My disagreement is with the idea with your original statement:

"Translating ambiguous symbols to meaning and combining them in meaningful ways is a big deal which, apparently, these AI models cannot do. They can just parrot things."

Modern AI systems can do this, and do not just parrot things.

They are capable of novel outputs.

This is because the models have sufficient "understanding" to manipulate "things" (latent vector representations, which you call symbols) to output novel but meaningful outputs.

Post reply on HN