Can AI do maths yet? Thoughts from a mathematician
31–40 of 364 posts
Re: Can AI do maths yet? Thoughts from a mathematician
#32At this stage I assume everything having a sequencial pattern can and will be automated by LLM AIs.
At this stage I hope everything that needs to be reliable won't be automated by LLM AIs.
Re: Can AI do maths yet? Thoughts from a mathematician
#33Who is the author?
Re: Can AI do maths yet? Thoughts from a mathematician
#34It's fascinating that this has run into the exact same problem as the Quantum research. Ie, in the quantum research to demonstrate any valuable forward progress you must compute something that is impossible to do with a traditional computer. If you can't do it with a traditional computer, it suddenly becomes difficult to verify correctness (ie, you can't just check it was matching the traditional computer's answer. I…
How much of this could be resolved if its training set were reduced? Conceivably, most of the training serves only to confuse the model when only aiming to solve a math equation.
Re: Can AI do maths yet? Thoughts from a mathematician
#35Earlier quoted context omitted.
If constrained by existing human knowledge to come up with an answer, won’t it fundamentally be unable to push human knowledge forward?
Then much of human research and development is also fundamentally impossible.
Only if you think current "AI" is on the same level as human creativity and intelligence, which it clearly is not.
Re: Can AI do maths yet? Thoughts from a mathematician
#36It's fascinating that this has run into the exact same problem as the Quantum research. Ie, in the quantum research to demonstrate any valuable forward progress you must compute something that is impossible to do with a traditional computer. If you can't do it with a traditional computer, it suddenly becomes difficult to verify correctness (ie, you can't just check it was matching the traditional computer's answer. I…
If constrained by existing human knowledge to come up with an answer, won’t it fundamentally be unable to push human knowledge forward?
Depends on your understanding of human knowledge I guess? People talk about the frontier of human knowledge and if your view of knowledge is like that of a unique human genius pushing forward the frontier then yes - it'd be stuck. But if you think of knowledge as more complex than that you could have areas that are kind of within our frontier of knowledge (that we could reasonably know, but don't actually know) - taking concepts that we already know in one field and applying them to some other field. Today the reason that doesn't happen is because genius A in physics doesn't know about the existence of genius B in mathematics (let alone understand their research), but if it's all imbibed by "The Model" then it's trivial to make that discovery.
Re: Can AI do maths yet? Thoughts from a mathematician
#37Earlier quoted context omitted.
If constrained by existing human knowledge to come up with an answer, won’t it fundamentally be unable to push human knowledge forward?
I don't think many expect AI to push knowledge forward? A thing that basically just regurgitates consensus historic knowledge seems badly suited to that
But apparently these new frontier models can 'reason' - so with that logic, they should be able to generate new knowledge?
Re: Can AI do maths yet? Thoughts from a mathematician
#38It's fascinating that this has run into the exact same problem as the Quantum research. Ie, in the quantum research to demonstrate any valuable forward progress you must compute something that is impossible to do with a traditional computer. If you can't do it with a traditional computer, it suddenly becomes difficult to verify correctness (ie, you can't just check it was matching the traditional computer's answer. I…
>in the quantum research to demonstrate any valuable forward progress you must compute something that is impossible to do with a traditional computer
This is factually wrong. The most interesting problems motivating the quantum computing research are hard to solve, but easy to verify on classical computers. The factorization problem is the most classical example.
The problem is that existing quantum computers are not powerful enough to solve the interesting problems, so researchers have to invent semi-artificial problems to demonstrate "quantum advantage" to keep the funding flowing.
There is a plethora of opportunities for LLMs to show their worth. For example, finding interesting links between different areas of research or being a proof assistant in a math/programming formal verification system. There is a lot of ongoing work in this area, but at the moment signal-to-noise ratio of such tools is too low for them to be practical.
Re: Can AI do maths yet? Thoughts from a mathematician
#39There was a little more information in that reddit thread. Of the three difficulty tiers, 25% are T1 (easiest) and 50% are T2. Of the five public problems that the author looked at, two were T1 and two were T2. Glazer on reddit described T1 as "IMO/undergraduate problems", but the article author says that they don't consider them to be undergraduate problems. So the LLM is already doing what the author says they would be surprised about.
Also Glazer seemed to regret calling T1 "IMO/undergraduate", and not only because of the disparity between IMO and typical undergraduate. He said that "We bump problems down a tier if we feel the difficulty comes too heavily from applying a major result, even in an advanced field, as a black box, since that makes a problem vulnerable to naive attacks from models"
Also, all of the problems shows to Tao were T3
Re: Can AI do maths yet? Thoughts from a mathematician
#40Earlier quoted context omitted.
If constrained by existing human knowledge to come up with an answer, won’t it fundamentally be unable to push human knowledge forward?
Depends on your understanding of human knowledge I guess? People talk about the frontier of human knowledge and if your view of knowledge is like that of a unique human genius pushing forward the frontier then yes - it'd be stuck. But if you think of knowledge as more complex than that you could have areas that are kind of within our frontier of knowledge (that we could reasonably know, but don't actually know) - tak…
I was referring specifically to the parent comments statements around current AI systems.