I think this is a silly question, you could track AI's doing very simple maths back in 1960 - 1970's
It's just the worrisome linguistic confusion between AI and LLMs.
Can AI do maths yet? Thoughts from a mathematician
51–60 of 364 posts
Re: Can AI do maths yet? Thoughts from a mathematician
#52I am fairly optimistic about LLMs as a human math -> theorem-prover translator, and as a fan of Idris I am glad that the AI community is investing in Lean. As the author shows, the answer to "Can AI be useful for automated mathematical work?" is clearly "yes." But I am confident the answer to the question in the headline is "no, not for several decades." It's not just the underwhelming benchmark results discussed in…
To give a sense if scale: It’s not that o3 failed to solve that red blue rectangle problem once: o3 spent thousands of gpu hours putting out text about that problem, creating by my math about a million pages of text, and did not find the answer anywhere in those pages. For other problems it did find the answer around the million page mark, as at the ~$3000 per problem spend setting the score was still slowly creeping…
If the trajectory of the past two years is any guide, things that can be done at great compute expense now will rapidly become possible for a fraction of the cost.
Re: Can AI do maths yet? Thoughts from a mathematician
#53Earlier quoted context omitted.
If constrained by existing human knowledge to come up with an answer, won’t it fundamentally be unable to push human knowledge forward?
I don't think many expect AI to push knowledge forward? A thing that basically just regurgitates consensus historic knowledge seems badly suited to that
O1 was able to find the math problem in a recently published paper, so yes.
Re: Can AI do maths yet? Thoughts from a mathematician
#54I may be wrong, but I think it a silly question. AI is basically auto-complete. It can do math to the extent you can find a solution via auto-complete based on an existing corpus of text.
> AI is basically
Very many things conventionally labelled in the 50's.
You are speaking of LLMs.
Re: Can AI do maths yet? Thoughts from a mathematician
#55It's fascinating that this has run into the exact same problem as the Quantum research. Ie, in the quantum research to demonstrate any valuable forward progress you must compute something that is impossible to do with a traditional computer. If you can't do it with a traditional computer, it suddenly becomes difficult to verify correctness (ie, you can't just check it was matching the traditional computer's answer. I…
>in the quantum research to demonstrate any valuable forward progress you must compute something that is impossible to do with a traditional computer This is factually wrong. The most interesting problems motivating the quantum computing research are hard to solve, but easy to verify on classical computers. The factorization problem is the most classical example. The problem is that existing quantum computers are not…
> This is factually wrong. The most interesting problems motivating the quantum computing research are hard to solve, but easy to verify on classical computers.
You parent did not talk about quantum computers. I guess he rather had predictions of novel quantum-field theories or theories of quantum gravity in the back of his mind.
Re: Can AI do maths yet? Thoughts from a mathematician
#56It's fascinating that this has run into the exact same problem as the Quantum research. Ie, in the quantum research to demonstrate any valuable forward progress you must compute something that is impossible to do with a traditional computer. If you can't do it with a traditional computer, it suddenly becomes difficult to verify correctness (ie, you can't just check it was matching the traditional computer's answer. I…
If constrained by existing human knowledge to come up with an answer, won’t it fundamentally be unable to push human knowledge forward?
Reasoning is essentially the creation of new knowledge from existing knowledge. The better the model can reason the less constrained it is to existing knowledge.
The challenge is how to figure out if a model is genuinely reasoning
Re: Can AI do maths yet? Thoughts from a mathematician
#57> FrontierMath is a secret dataset of “hundreds” of hard maths questions, curated by Epoch AI, and announced last month.
The database stopped being secret when it was fed to proprietary LLMs running in the cloud. If anyone is not thinking that OpenAI has trained and tuned O3 on the "secret" problems people fed to GPT-4o, I have a bridge to sell you.
Re: Can AI do maths yet? Thoughts from a mathematician
#58Re: Can AI do maths yet? Thoughts from a mathematician
#59So here's what I'm perplexed about. There are statements in Presburger arithmetic that take time doubly exponential (or worse) in the size of the statement to reach via any path of the formal system whatsoever. These are arithmetic truths about the natural numbers. Can these statements be reached faster in ZFC? Possibly—it's well-known that there exist shorter proofs of true statements in more powerful consistent sys…
> There are statements in Presburger arithmetic that take time doubly exponential (or worse) in the size of the statement to reach via any path of the formal system whatsoever.
This is a correct statement about the worst case runtime. What is interesting for practical applications is whether such statements are among those that you are practically interested in.
Re: Can AI do maths yet? Thoughts from a mathematician
#60Earlier quoted context omitted.
Then much of human research and development is also fundamentally impossible.
Only if you think current "AI" is on the same level as human creativity and intelligence, which it clearly is not.
I think current "AI" (i.e. LLMs) is unable to push human knowledge forward, but not because it's constrained by existing human knowledge. It's more like peeking into a very large magic-8 ball, new answers everytime you shake it. Some useful.