A misalignment of AI in mathematics

The declaration literally opens by writing that AI companies (or really anybody) saying, “Hey, let’s see if this powerful reasoning engine can solve an open problem in mathematics” is “detrimental” to the “science” of mathematics. Full stop. So I don’t think it’s being misrepresented. If it’s a science, why isn’t progress good in itself? You can’t have it both ways.

The online comments I’ve read that side with the letter explain that the problem is AI proofs are inscrutable and useless, but the reality is using computers to brute-force things has long been part of proofs in one way or another, and AI proofs range in legibility up to 100% (like pointing out a proof exists in a long-lost paper, or writing something that is basically correct and just needs a human reviewer to fix it up). People are probably defending the declaration inaccurately, but I think that’s because the real argument or arguments are unclear.

Is it that students won’t learn math if there’s ChatGPT? That could be discussed.

To a non-mathematician (MIT physics and CS ‘06), the letter sounds like, “We have a fun job. Sometimes there are no practical applications of the work, so it’s just kind of like a sport, er I mean science. If someone solves a problem that has stumped mathematicians for decades or centuries, we worship them as a great mathematician. It’s a status thing. So we don’t like some guy with a computer coming along and solving our problems. The way things unfold with all the ideas coming from humans, over time, maybe it’s slower, but it’s nice.”

I don’t think anyone can really stop someone from using a computer capable of solving unsolved problems to solve unsolved problems, and there are always going to be mathematicians who DO think it’s fun to try to figure out what a computer is doing (if the computer isn’t already explaining it in English, which it generally can), and make progress that way, and it obviously will lead to mathematical advances in human understanding, from my point of view. So the whole thing is a non-issue that no one can do anything about anyway.

Many comments seem to be misinterpreting this, but all this seems to be suggesting is that the AI companies should not be treating mathematical milestones as benchmark targets but need to release the cumulative findings in a human interpretable and digestible way that enriches and develops the field. See Thurston quote on Tao's banner:

> We are not trying to meet some abstract production quota of definitions, theorems and proofs. The measure of our success is whether what we do enables people to understand and think more clearly and effectively about mathematics.

I agree insofar as this may fall on deaf ears of AI folks, but that the goals are differ is exactly the point of pointing out misalignment. To talk about this may be more useful for coalescing professional societies than for conversing directly with AI advancers.

I've had a similar experience.

LLMs doing math has made math far more accessible to me than it was pre-LLM.

Not just using existing math, but developing new math. (For example, I developed an alternative to NURBS surfaces for geometric kernels using LLMs.)

It's really annoying that anyone, anywhere is talking about halting math progress with AI.

Thanks for putting it in words. I feel the same about SOTA coding models. I get visibly frustrated by this and often find myself using curse words at Astra.

You're right it's very unhuman to miss obvious opportunities you mentioned. And what's why I think these models would continue sucking as long as they don't have human level general intelligence. They may get marginally better at these tasks but I don't think we can expect them to connect dots like humans before AGI.