Mathematicians issue warning as AI rapidly gains ground

I found myself thinking about this issue when I was experimenting with an MCP server to handle tuning some precision parameters for scientific simulations. Claude did a much better job than I used to do when I was a fresh PhD student, yet being given tasks like that was how I learned, so it almost felt like pulling the ladder up after myself.

In the sciences, I think this is less of a problem because the PhD to scientist pipeline is pretty normalized, labs are used to the idea of having to let younger people take longer on problems that experienced people could solve much faster. But this doesn't seem to be as normalized elsewhere.

I personally do wonder (worry) about where all of this pans out and what society looks like post generative llms. But at the same time there is a particular flavor of amusement that I can't help feeling watching folks simultaneously balance, "llms produce nothing of value" and "llms are so harmful and dangerous to our culture that we need to start policing use within our community"

Where that harm essentially stems from devaluing hard earned skills within the community. And while I do not take joy in the displacement of labor, never in my wildest dreams could I have anticipated how harsh and irrational of a reaction to the equity of these skills could be. Which, I would like to point out, though hard earned were earned under the tremendous privilege to pursue these goals in the first place.

Llms are an amplifier of an individuals intuition and taste. That these supposed pillars of the community are not bravely exploring how to push and wrangle these bounds, and instead are retracting into conservative stances under the guise of human centric morality is (IMHO) demonstrative of lack of confidence and creativity within these fields more generally.

I believe that this lack of creativity and imagination is how we find ourselves in the personal fable you're noting: the experts are so myopic that they can't even imagine how they're field can be disrupted until it's disrupted outside of their control, and feel the need to control rather than explore.

Your distinction between the practical and the theoretical is important. Practicality is important - everything we do is a matter of practicality of means or method, even how we pursue theoretical ends - but two points.

First, there is more to life than the practical. Some truths are known for their own sake, even if they also tell us about still more profound truths (also known for their own sake) or may have incidental practical relevance and consequences in some other context.

Second, while the theoretical terminus is the truth for its own sake, the practical terminus is always something other than itself. Well, what is that "something else"? You can't have an infinite regress of practicality. The meaning of a proximate, practical end is always other than itself. The practical requires an end beyond itself to justify it.

I agree that most people don't seem to inquire much about such ultimate ends. Their thoughts are confined to the proximate. Of course, how have they determined what the proximate should be? Something for people to contemplate.

Where science is concerned, it depends. On the one hand, there are fields that are certainly more theoretically oriented. It's not "the game" that motivates theory - that would make it mere recreation, with the truth taking a backseat - but the truth. (For this reason, I hesitate to call Erdos theoretically motivated. AFAICT, he was motivated by the challenge of problem solving and not the truth, insight, and understanding to be gained which would have been merely incidental and instrumental for him.)

However, I would also say a good chunk of science is motivated by a background motivation of technology production and the mastery of nature. Think Francis Bacon who viewed science as an instrument of power and showed a preference for the "how" over the "what" (τόδε τι) or the "why" (τὸ διότι). This set the tone for a great deal of modern science. A great deal does less explaining and more predictive modeling, because predictive modeling can be sufficient for control. Indeed, a truly theoretical causal account and understanding of a thing's nature can be less useful as a practical instrument than a merely predictive model.

Now, AI is a practical tool. I think they can be enormously useful as research aids, even in theoretical contexts, provided that one

1. understands their nature;

2. understands the purpose of the theoretical activity undertaken.

What is their nature? Well, they're statistical models that can unearth interesting and useful correlations and patterns. But they are not reasoning and knowing things. Their results are generated mechanically and mindlessly. Knowing this means taking their results with a healthy skepticism and a critical eye.

What about the purpose of theory? By analogy, think of a student in school who uses AI to complete all his assignments. Has he satisfied the purpose of those assignments? No, because the purpose of the assignments isn't to produce the effect - the solutions - per se, but to learn something. Theoretical work is like that; it's purpose is to understand and to grasp some truth. An AI can be used to assist this process, just as a calculator or a search engine can, but if you use it in a manner that circumvents that purpose instead of supporting it, then you're not achieve that purpose and wasting your time. What's the point?