A misalignment of AI in mathematics

Why do we believe that we cannot train models which could explain the jargon in more human terms when current LLMs can perfectly explain the most complicated codebases?

Tao is not someone who is anti-AI for the sake of being anti-AI. He has been advocating for the usefulness of AI in maths for a long time, to the point that people have started calling him a shill for the commercial companies.

And everyone agrees that there are plenty of use cases to be had; helping with less interesting tasks like easing literature review, efficiently delving into existing work, doing review, whether on your own work or that of others, prototyping algorithms in areas where computation is useful, but also more in hands-on aspects of maths like validating potential proof directions by getting quick feedback on veracity of lemmas, etc., and, on very rare occasions, being able to one-shot the problem you care about.

The point he is trying to make here is much more subtle than "AI bad", and it's probably easy to miss if you have never engaged with research in maths: it's that the particular approach that large commercial companies have opted to take to produce marketing material can be a net negative. There is not doubt that -- even if you ignore the rampant plagiarism that has been reported across multiple problems now, the unethical attempts to oust authors, the outrageous attempts to scoop researchers instead of collaborating with them and building on existing projects -- it's nifty to have a machine that can help you figure out if a proposition is true or not. But just figuring out as much was never the point. When people have built problem lists, it's because some problems are more likely than others to provide new insight, and that insight is the target. And to than end, a poorly written paper with inadequate references and a pile of Lean is not valuable at all. Yes, now we know with higher certainty that Fermat's Last Theorem is true, but everyone expected that already.

One place where "just" answering the question can be a net negative is because the current incentive structure is set up in such a way that going in afterwards, trying to reclaim and extract the insights from a brute force solution, is considered less valuable work than that of coming up with a solution in the first place. That's a problem of incentives, and something Tao himself has addressed in e.g. his ICM talk, and that's something that we'll want to do something about. Until a better structure appears, though, if any given commercial provider of large language models really wants to help out with maths research and not just make more pre-IPO marketing material by competing with their customers, they could do so by using their magic machines to help build insight instead.

I'm outraged at mathematicians. There, write the article.