Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI language models are struggling to “get” math

spectrum.ieee.org

Post reply on HN
Print view View original

51–60 of 201 posts

Re: AI language models are struggling to “get” math

#51

Post by hey_over_here »

PaulHoule wrote: post #2

Ashby strikes again. Current sequence models don't have the right structures to represent math. Even if they use floating point internally, they can't really float the point because the nonlinearity in the model has a certain scale. A system that processes language can take advantage of the human desire for closure https://www.eurogamer.net/blood-in-the-gutter to fool people into thinking it is more capable than it r…

Mwell, the article claims, and points to work that also claims, that large language models can actually be made to perform arithmetic well. They need fine-tuning, verification, chain of thought prompting and majority voting to be combined but the linked Google blog says that Minerva hit 78.5% accuracy (on the GSM8K benchmark).

For me the problem is that we can look at the output and say if it's right or wrong, but we know what language models do, internally: they predict the next token in a sequence. And we know that this is no way to do arithmetic, in the long run, even though it might well work over finite domains.

Which is to say, I'm just as skeptical as you are, and probably even more, but I think it's useful to separate the claim from what has actually been demonstrated. Google claims its Minerva model is "solving maths problems" but what it's really doing is predicting solutions to problems like the ones it's been fine-tuned on, and those problems are problems stated at least partly in natural language, not "naked" arithmetic operations. In the latter, language models are still crap because they can't use the context of the natural language problem statement to help them predict the solution.

Btw, "chain of thought prompting" if I remember correctly is a process by which an experimenter prompts the language model with a sequence of intermediary problems. So it's not so much the model's chain of thought, as the experimenter's chain of thought and the experimenter is asking the model to help him or her complete their chain of thought. I have a fuzzy recollection of that though.

Re: AI language models are struggling to “get” math

#52

Post by mjburgess »

dimmuborgir wrote: post #39
mjburgess wrote: post #27

How much of this is just "AI is bad at everything", but in the math case, it's easier for the lay person to tell . It's all just passable garbled nonesense that the reader (goes to lengths) to interept based on their prior knowledge, which is not expressed in the syntax of what these systems output. In the case of mathematics, we're far less willing to "BS away" the interpretive failures. But if we were equally deman…

AI is bad at music also. Even the state of the art transformer models can't produce more than a few seconds of coherent melodic phrases.

I think if we replaced "AI" with "taking averages over subsets of historical examples", then there'd be no mystery for when "AI" will be good or bad at anything.

Would we expect a discrete melodic structure to be expressible as averages of prior music? No.

Re: AI language models are struggling to “get” math

#53

Post by aaaaaaaaaaab »

burlesona wrote: post #29

I genuinely wonder if we will find there are some inherent tradeoffs to knowledge and understanding such that if we ever have machines that can “think like humans” they would in practice run into human-like cognition limits: ie such machines would be “bad at math” in the same way humans are “bat at math” compared to conventional computers.

Even if neural networks were fundamentally incompatible with conventional computation, I don't see why you couldn't augment a neural network with a conventional ALU to do the numerical computations. This is exactly what humans do with pencil and paper - it's just a bit too slow.

Re: AI language models are struggling to “get” math

#54

Post by enord »

sharemywin wrote: post #10
sharemywin wrote: post #3

Earlier quoted context omitted.

computers already do math. language models just need to translate problems into code of some kind that can be run to get the answer. executive function/planning is probably the biggest problem at this point for ai.

The point I'm trying to make is LLMs don't need to do everything just be the glue to other systems.

Wait what? Glue as in extract high level semantic representations from _syntatic probabilities_ and pass on to appropriate domain specific tools?

This is the glaring hole in LLMs, a paradoxical semantic incoherence despite impressive sentenial and gramatical coherence.

As glue it is so thin as to be potable.

Re: AI language models are struggling to “get” math

#55

Post by yeasurebut »

dimmuborgir wrote: post #39
mjburgess wrote: post #27

How much of this is just "AI is bad at everything", but in the math case, it's easier for the lay person to tell . It's all just passable garbled nonesense that the reader (goes to lengths) to interept based on their prior knowledge, which is not expressed in the syntax of what these systems output. In the case of mathematics, we're far less willing to "BS away" the interpretive failures. But if we were equally deman…

AI is bad at music also. Even the state of the art transformer models can't produce more than a few seconds of coherent melodic phrases.

That’s what a musician does. They make short loops and loop them.

This reads like someone who knows sheet music and theory but does not listen to music. It’s repetition of short phrases over and over.

I’m not really sure what people expect of general AI trained on human generated outputs. It can’t make up anything anything “net new” only compose based upon what we feed it.

I like to think AI is just showing us how simple minded we really are and how our habit of sharing vain fairy tales about history makes us believe we’re masters of the universe.

Re: AI language models are struggling to “get” math

#56

Post by PaulHoule »

zozbot234 wrote: post #46
lupire wrote: post #19

Earlier quoted context omitted.

Floating point isn't relevant here. The problem is that human language is approximate and correct math is not, so pattern matching on prose text is doomed. AI trained on exact math does a lot better. But that's not fully generic so fails the weird GPT goal of modeling all of human intelligence through prose. That's not how people solve math at all. GPT's "Superficially plausible but wrong" math is actually pretty goo…

> GPT's "Superficially plausible but wrong" math is actually pretty good match for non-expert bad-at-math average human behavior. Relevant blog post: https://www.greaterwrong.com/posts/YhgjmCxcQXixStWMC/artific... "The best experts in the field estimate it will be at least a hundred years before calculators can add as well as a human twelve-year-old."

I like Yudkovsky parodying himself there although I still don't know if he has a sense of humor or not.

Re: AI language models are struggling to “get” math

#57

Post by hey_over_here »

CommieBobDole wrote: post #50
hey_over_here wrote: post #44

Earlier quoted context omitted.

> It's a language model; why would we expect it do math or try to somehow shoehorn math into the model? Language models can do math, or anyway arithmetic. That's because language models are trained to predict the next token in a sequence and an arithmetic operation can be represented as a sequence of tokens. For example, see Figure 3.10 on page 22, here: https://arxiv.org/abs/2005.14165 The only problem is that langu…

Right, but what you're describing is 'not being able to do math'. Like, if I've memorized a multiplication table and can give you any result that's on the table but can't multiply anything that wasn't on the table, I can't do multiplication.

It depends on how you see it. I agree with you, generally, but in the limit, if you memorised all possible instances of multiplication, then yes, you could certainly be said to know multiplication.

I've not just come up with that off the top of my head, either. In PAC-Learning (what we have in terms of theory, in machine learning) a "concept" (e.g. multiplication) is a set of instances and a learning system is said to learn a concept if it can correctly label each of a set of testing instances by membership to the target concept with arbitrary probability of error. Trivially, a learner that has memorised every instance of a target concept can be said to have learned the concept. All this is playing fast and loose with PAC-Learning terminology for the sake of simplification.

The problem of course is that some concepts have infinite sets of instances, and that is the case with arithmetic. On the other hand, it's maybe a little disingenuous to require a machine learning system to be able to represent infinite arithmetic since there is no physical computer that can do that, either.

Anyway that's how the debate goes on these things. I'm on the side that says that if you want to claim your system can do arithmetic, you have to demonstrate that it has something that we can all agree is a recognisable representation of the rules of arithmetic, as we understand them. For instance, the axioms of Peano arithmetic. Which though is a bit unfair for deep learning systems that can't "show their work" in this way.

Re: AI language models are struggling to “get” math

#58

Post by PaulHoule »

sva_ wrote: post #17
PaulHoule wrote: post #13

Earlier quoted context omitted.

There has been big progress in automated theorem proving lately https://en.wikipedia.org/wiki/Automated_theorem_proving you just don't hear about it much because the technology is not so fashionable today. Also it is more clear what the limits are, I mean, Turing, Godel, Tarski and all of those apply to neural networks as well any other formal system but people mostly forget it. Knuth wrote a really fun volume of The…

> There has been big progress in automated theorem proving lately It doesn't seem like there has been much progress for anything but FOL?

It's clear that commonsense reasoning needs to deal with modals, counterfactuals, defaults, temporal logic, etc.

It's not hard to add some extensions to logic for a particular application but a very hard problem to develop a general purpose extended logic.

I look at the logic-adjacent production rules systems which never really standardized some of the commonly necessary things such as agendas, priorities, defaults, etc.

Re: AI language models are struggling to “get” math

#59

Post by dimmuborgir »

yeasurebut wrote: post #55
dimmuborgir wrote: post #39

Earlier quoted context omitted.

AI is bad at music also. Even the state of the art transformer models can't produce more than a few seconds of coherent melodic phrases.

That’s what a musician does. They make short loops and loop them. This reads like someone who knows sheet music and theory but does not listen to music. It’s repetition of short phrases over and over. I’m not really sure what people expect of general AI trained on human generated outputs. It can’t make up anything anything “net new” only compose based upon what we feed it. I like to think AI is just showing us how si…

Those models are not trained on short loops. They are trained on whole songs just like image generation models are trained on whole images. And yet they struggle to repeat sections, modulate to a different key, create bridges, intros and outros. After a few seconds of hallucinating a melodic line they simply abandon the idea and migrate to another one. There is no global structure whatsoever.

Re: AI language models are struggling to “get” math

#60

Post by yshrestha »

lupire wrote: post #21
yshrestha wrote: post #9

Language models can generate a Python function that does the math perfectly. I bet you would get better results if you tweaked the prompt to say "Generate a Python program that solves X math problem" and then just ran the resulting Python script. It does not need to be AGI to be useful.

You mean "generate a Python function that calls a library that does math perfectly, right?

Exactly! Hey it gets the job done :)

Software is just a tall wedding cake of abstractions built on top of abstractions.

Post reply on HN