Lots of people talk about cheating here. That is not the framing I put in my mind. Or rather, if the students are cheating anyone it's themselves.
LLMs exist and are extremely useful in academia. I am currently riding a productivity wave, where I can spit out formalised proofs – not quite at the pace at which I can think of the theorems to prove, but almost. The thing braking my progress is that I have to build all the existing theory too, not just my own new stuff.
But for students, who do not yet know anything, it is a danger. From observing my students, it seems to take a certain amount of willpower to do the exercises without letting ChatGPT give them (or "explain them") the answer. And reading an explanation is not the same thing as working it out yourself. You just don't learn doing it, by not doing it!
Some students have this will power, some students don't. And it is also situational. Some students have stuff going (work or other factors) which makes it really hard too.
Yes, they can of course keep asking ChatGPT later in life, but they will not have the fundamental understanding which is required to even use LLMs for the advanced stuff. ChatGPT cannot explain you advanced concepts if you don't understand the fundamentals!
So, to test the students' understanding, we are back to having on-site exams, with the old-school supervision. And more students will fail, because there is now a way to solve exercises without learning.