Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI boosted homework scores, then exam scores dropped: study

economist.com

Post reply on HN
Print view View original

381–390 of 409 posts

Re: AI boosted homework scores, then exam scores dropped: study

#381

Post by kzz102 »

nilamo wrote: post #356
kzz102 wrote: post #157

Earlier quoted context omitted.

It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic: "At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generat…

I'm curious what happened in the source, that a copy/paste exposed such odd artifacts. "e§ort" == effort "Öve" == not sure. Maybe "over five"?

It was copied from a pdf, maybe LaTeX generated. All the ligatures are wrong -- not sure where the problem lies, the pdf itself, pdf reader, or the clipboard on Linux.

Re: AI boosted homework scores, then exam scores dropped: study

#382

Post by rzerowan »

Seems if any assignments are to be handed out they should be structured in such a way that takes into account the potential of LLM use. Additionally to make the output (hardcopy? handwritten?oral presentation?) in such a way that even if LLms are used at the very least some part will be forcibly undertaken by the students in preparation of finals/real life.

Open book exams existed for a long time and even take home assignments where the net was usable to complete the tasks. Difference was one actually had to do the reseach and learn something along the way regarding research skills or the topic itself.

LLms just make the automation and laziness with a lower barrier to polished output.

Re: AI boosted homework scores, then exam scores dropped: study

#383

Post by lelanthran »

asdfman123 wrote: post #279
kzz102 wrote: post #157

Earlier quoted context omitted.

It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic: "At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generat…

Personally in my own work it's pretty obvious how it's an amplifier. You waste far less time trying to find an answer to something that confuses you. You might say "it's good to learn research skills" and that's true to an extent, but tutors have always made people better students. And AI is a tutor you can message at any time, day or night, for free.

> Personally in my own work it's pretty obvious how it's an amplifier.

The answer is obvious: you're doing homework type activity (learning is not a goal), not exam type activity (learning is the goal).

Re: AI boosted homework scores, then exam scores dropped: study

#384

Post by grey-area »

ndriscoll wrote: post #373
grey-area wrote: post #369

Earlier quoted context omitted.

LLMs are not a shortcut to mastery, in fact the opposite most of the time and their errors are qualitatively different from human errors.

I don't think anyone is claiming they're a shortcut to mastery. Personal tutors aren't a shortcut to mastery either. Both simply have the advantage of being able to deeply know where the student's knowledge frontier is, so they can tailor their answers to that, make connections and draw analogies to things the student has mastered (while pointing out distinctions), etc. The student still needs to put in the work.

LLMs are not tutors and do not deeply know anything.

Re: AI boosted homework scores, then exam scores dropped: study

#385

Post by CivBase »

dominicq wrote: post #347
CivBase wrote: post #287

Earlier quoted context omitted.

You put way too much faith in kids' desire to learn and their ability to make responsible choices. > Homework should be optional material for self study, evaluated by teachers if you choose to do it but never forced. My wife is a teacher and her school does this. The result is that nobody does homework. The kids who need the extra practice don't get it and they fall into a spiral of failure and apathy. This is especi…

Homework is no longer useful in the era of AI. Options are: treat kids like they are able to make responsible choices, and accept that much more of them will fail; or, keep the charade of grading AI-"assisted" homework and push kids farther along the track, even though they definitely don't know enough. I much prefer the first. School shouldn't be impossible, but you shouldn't have a charade of schooling. Obviously,…

It's not necessarily useless. If you make assignments short, provide some time in school to work on it, and do not allow students to use devices with access to general-purpose "AI", then there is a good chance most students will do the work like they're supposed to. Some will still cheat, but it's foolish to reorganize everything around the few who stubbornly refuse to learn.

Re: AI boosted homework scores, then exam scores dropped: study

#388

Post by ndriscoll »

grey-area wrote: post #384
ndriscoll wrote: post #373

Earlier quoted context omitted.

I don't think anyone is claiming they're a shortcut to mastery. Personal tutors aren't a shortcut to mastery either. Both simply have the advantage of being able to deeply know where the student's knowledge frontier is, so they can tailor their answers to that, make connections and draw analogies to things the student has mastered (while pointing out distinctions), etc. The student still needs to put in the work.

LLMs are not tutors and do not deeply know anything.

In the sense that they are not programmers and do not deeply know e.g. performance engineering, sure. But I can still give one a program and ask it to optimize it, and it will turn it into a zero allocation streaming SIMD implementation that's orders of magnitude faster than where it started. It may not "know" how to do any of that, but it does it anyway.

Re: AI boosted homework scores, then exam scores dropped: study

#389

Post by stephbook »

tancop wrote: post #163

All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates. We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional mat…

> If you try and force kids to study when they don't want to "for their own good" you're not actually helping them. Everyone needs to find their own path. Let people fail.

This simply does not work for most, as pupils are too immature. You put sunscreen on children, keep them away from the street and brush their teeth. Having everyone re-learn the same stupid lesson is wasteful, it's better to institutionalize this knowledge in the form of mandatory participation.

Re: AI boosted homework scores, then exam scores dropped: study

#390

Post by zeryx »

I full heartedly recommend anyone who wants to learn, to use the Socrates skill, and then build a daily 1 hour curriculum that you will tackle. It's painful and not pleasant, but it works quite well when to don't have a teacher and want to learn something really hard and complicated
Post reply on HN