AI language models are struggling to “get” math
spectrum.ieee.org
AI language models are struggling to “get” math
1–10 of 201 posts
Re: AI language models are struggling to “get” math
#2Ashby strikes again.
Current sequence models don't have the right structures to represent math. Even if they use floating point internally, they can't really float the point because the nonlinearity in the model has a certain scale.
A system that processes language can take advantage of the human desire for closure
https://www.eurogamer.net/blood-in-the-gutter
to fool people into thinking it is more capable than it really is. Math isn't like that.
Re: AI language models are struggling to “get” math
#3Ashby strikes again. Current sequence models don't have the right structures to represent math. Even if they use floating point internally, they can't really float the point because the nonlinearity in the model has a certain scale. A system that processes language can take advantage of the human desire for closure https://www.eurogamer.net/blood-in-the-gutter to fool people into thinking it is more capable than it r…
computers already do math. language models just need to translate problems into code of some kind that can be run to get the answer.
executive function/planning is probably the biggest problem at this point for ai.
Re: AI language models are struggling to “get” math
#4Are there any general purpose models that are good at learning math? I mainly know basic feed-forward neural nets, but I don't think they do well outside their training region. Math, of course, has an infinite training region.
Re: AI language models are struggling to “get” math
#5Ashby strikes again. Current sequence models don't have the right structures to represent math. Even if they use floating point internally, they can't really float the point because the nonlinearity in the model has a certain scale. A system that processes language can take advantage of the human desire for closure https://www.eurogamer.net/blood-in-the-gutter to fool people into thinking it is more capable than it r…
computers already do math. language models just need to translate problems into code of some kind that can be run to get the answer. executive function/planning is probably the biggest problem at this point for ai.
I'm not so sure about that. Of course computers can do arithmetic operations, but this is not the same as solving math problems, proving theorems, etc.
Even mathematical objects are approximated up to an approximation error in a computer (like a differentiable manifold or a real number).
Re: AI language models are struggling to “get” math
#6Are there any general purpose models that are good at learning math? I mainly know basic feed-forward neural nets, but I don't think they do well outside their training region. Math, of course, has an infinite training region.
From my (limited) experience with the advanced ML models, they can "do basic math", but they make amateur mistakes with basic things - which indicates they don't actually know addition, but they are good at looking at patterns in existing language.
I would assume that state-of-the-art ML models could "convert a word problem into an equation", then feed that equation into a 30 year-old graphing calculator to "do the math"
The fact that no one has done this is an indicator that "there are more important things to work on", and it is just a matter of time that someone connects the two together
Re: AI language models are struggling to “get” math
#7Earlier quoted context omitted.
computers already do math. language models just need to translate problems into code of some kind that can be run to get the answer. executive function/planning is probably the biggest problem at this point for ai.
I'm not so sure about that. Of course computers can do arithmetic operations, but this is not the same as solving math problems, proving theorems, etc. Even mathematical objects are approximated up to an approximation error in a computer (like a differentiable manifold or a real number).
Online Integral Calculator
Solve integrals with Wolfram|Alpha
https://www.wolframalpha.com/calculators/integral-calculator...
Re: AI language models are struggling to “get” math
#8Ashby strikes again. Current sequence models don't have the right structures to represent math. Even if they use floating point internally, they can't really float the point because the nonlinearity in the model has a certain scale. A system that processes language can take advantage of the human desire for closure https://www.eurogamer.net/blood-in-the-gutter to fool people into thinking it is more capable than it r…
computers already do math. language models just need to translate problems into code of some kind that can be run to get the answer. executive function/planning is probably the biggest problem at this point for ai.
That's not a bad approach, necessarily.
There is a fairly simple program in
https://www.amazon.com/Paradigms-Artificial-Intelligence-Pro...
that solves word problems using the methods of the old AI. The point is that is is efficient and effective to use real math operators and not expect to fit numbers through the mysterious bottleneck of neural encoding.
Re: AI language models are struggling to “get” math
#9Language models can generate a Python function that does the math perfectly.
I bet you would get better results if you tweaked the prompt to say "Generate a Python program that solves X math problem" and then just ran the resulting Python script.
It does not need to be AGI to be useful.
Re: AI language models are struggling to “get” math
#10Ashby strikes again. Current sequence models don't have the right structures to represent math. Even if they use floating point internally, they can't really float the point because the nonlinearity in the model has a certain scale. A system that processes language can take advantage of the human desire for closure https://www.eurogamer.net/blood-in-the-gutter to fool people into thinking it is more capable than it r…
computers already do math. language models just need to translate problems into code of some kind that can be run to get the answer. executive function/planning is probably the biggest problem at this point for ai.
The point I'm trying to make is LLMs don't need to do everything just be the glue to other systems.