> Mathematics produces not only a body of results, but also understanding, clarity, and judgment among the communities of mathematicians who have shaped them, often in the context of their own autonomously guided research. This expert knowledge is essential, both to effectively use mathematics, and to continue to articulate new and significant research questions. In a word, the job of the mathematics department is no…
> Outsourcing the work deprives you of who you become by writing it. Just because AI can do something that resembles work should not mean outsourcing work to it. Mathematicians should not outsource their work to AI just like programmers should not outsource programming to AI. Humans working with AIs in a tight loop means intellectual work becomes more high-level and creative, but a human should always own the work, v…
The idea that we just "trust everyone to carefully check and learn from AI output" as our barrier to human skillsets eroding is never going to work.
There is an Anthropic engineering post on HM front page that addresses this exact issue:
"... supervise the agent’s behavior via a human-in-the-loop. Claude Code previously protected against agents taking unintended actions by asking users for permission at each turn. Theoretically that works, but we’ve found the approach to be fallible. Our telemetry showed users approved roughly 93% of permission prompts. The more approvals a user sees, the less attention they pay to each, becoming over time much less diligent in their supervision. "