Earlier quoted context omitted.
Mathematicians are more than just number crunchers. Perhaps the most important part of a mathematicians job is formally defining problems. Even if the next generation of ChatGPT can solve any math problem you give it satisfactory, you still need a mathematician to ask the question. Presumably this AI won’t be omnipotent either, and there will be some limit to its mathematical prowess. I suspect most human effort will…
LLMs stand to be much better at research mathematics than humans. Breakthroughs in mathematics often come from taking a concept from one area of mathematics and applying it to another area. Humans find this difficult because it's hard to be a research-level mathematician in more than a few areas, a human life isn't long enough to learn that much. Humans learn depth-first. LLMs learn breadth-first. You can go ask Chat…
Computers were invented in large part for this purpose. The hope was to have a machine that could spit out interesting theorems. It turned out that generating theorems was trivial (you can just enumerate them) but generating interesting theorems was a task of an entirely different nature. To this day, computers are frequently used as tools by maths researchers but cannot do their own independent maths research, i.e. invent and prove their own interesting theorems. This is surprising if you think of research maths as ultimately a game of symbol manipulation, but I think what it tells us is that that is not what research maths is.
The discovery that in trying to make a maths machine we made a computation machine instead, and it wasn't inherently any good at maths, made the difference between computation and maths very clear. What we have now is algorithms that are able to learn patterns in symbol use from large corpuses of text, and generate new text by recombining those patterns. Personally, I'm pretty sure this will be a demonstration that that isn't what research maths is either.
If LLMs do turn out to be capable of independently inventing and proving interesting new theorems (every word of that is important) I will have to throw away most of what I have come to think about what intelligence is, and I will do so without complaint because it will have been shown to have been wrong. Indeed, I'll do it with joy, because I've always believed AGI is possible and the biggest thing I've always wanted out of its invention is a better understanding of what mathematics is. This will have been provided.
However, I don't expect them to. Usually when I mention this people bring up various examples of attempts that have been made to make AIs "do maths" in some way but I'm aware of what has been done so far and none of it comes close to AI "independently inventing and proving interesting new theorems" or looks like its heading towards that any time soon.