>The first assumption is wrong because to really solve a mathematical problem, providing a mere answer (even if formally certified) is not sufficient. What is missing is an intelligible proof that human mathematicians can understand and use to advance the aims of mathematics. This makes a bad assumption that humans need to be the one to advance the aims of mathematics. LLMs could be what advances the aims of mathemat…
Formal proof only emerged early in the 20th century, and the standard became that in theory a proof should be formalizable to answer any skepticism, but the real goal in Euclid's time and ours has been to communicate why a theorem is true to your fellow humans. There were a few theorems that are only known via computer proof, like the Four Color Theorem, but this has always been regarded as disappointing or even controversial, and the fact that there hasn't been any conceptual breakthrough has meant that we didn't learn anything other than the sheer fact that the Four Color Theorem is true. Theorems that produce understanding, on the other hand, typically produce many new ideas that lead to more theorems.
The purpose of scholarship is understanding. This is just as true for science as it is for math. If AI produces a unified theory of fundamental physics, but it's just an opaque blob, physicists will find it just as unsatisfying.