Professor Malan said students would be warned of the pitfalls of the AI, saying they should “always think critically” when presented with information. I want to believe that this will actually happen, but my experience with people blindly trusting what a tool (not even an AI-based one) says, including but not limited to students, suggests that's not going to happen.
My experience with LLMs so far is that they very confidently give you an answer and very convincingly explain that answer but for certain portion of the time that answer is very wrong. You can even get it to break its reasoning down into little convincing but, none the less, incorrect steps. If I was running this course I would make that "artefact" part of the process. i.e. I would tell students that the chatbot will…
You won‘t have to. Students are extraordinarily skilled at fining endless pitfalls on their own.