LLMs can't contribute good code to some of the good OSS math libraries, How is it even solving these problems?
It is able to contribute code, but maybe not good code.
It's the same in math: it's able to solve problems, but not necessarily in a good way with a human readable code.
Math papers are a lot like software:
- theorems are like API
- lemmata like internal/private function API
- definitions are like types
- the proofs are the implementation
The proofs of ChatGPT are not necessarily readable or maintainable.