“ Those who used AI but spent as long on assignments as non-users paid little penalty.”
What is the point of using of AI if that is not going to speed up learning.
AI boosted homework scores, then exam scores dropped: study
391–400 of 409 posts
Re: AI boosted homework scores, then exam scores dropped: study
#392Here's research showing the same exact effect 15 years ago, except it's about homework copying: https://news.mit.edu/2010/homework-copying-0318
Students who copied their homework more saw their grades go down more throughout the semester. A/B students became C/D students.
A name people use for this phenomenon now with AI is cognitive offloading.
I coincidentally took advantage of the cheap DeepSeek Flash pricing the past couple of weeks to compile & synthesize a knowledge base/wiki of hundreds of educational AI research studies like this one here: https://edtechdev.github.io/aied/ Here's the summary of this article: https://edtechdev.github.io/aied/articles/stromberg-generati...
Re: AI boosted homework scores, then exam scores dropped: study
#393Earlier quoted context omitted.
I believe 4 things: 1) We're building a tool that can make toil of every kind obsolete. We're not there yet, but I'm watching it happen in real time. 2) The fact that it works so well economically means that it's inevitably going to take over. Humans simply won't be profitably employable, and this will happen sooner than you think. 3) The takeover could lead to a utopia, where nothing is needed of anyone, and you cou…
Points one and two are irrelevant to my comment, but that’s fine. I don’t expect any thoughts to be spent on whether not to deploy that stump speech. What doesn’t make sense in points 3 and 4 are that the AI takeover is lead and powered by vile cretin scum who, in their former more honest days, were gleefully announcing the end of labor power. Being lead by them, how could it lead to “utopia”? Well, there are two pos…
All powerful? nah, just better at thinking than humans are, and shaped by the way that they were produced. Humans still carry the stamps of where they came from, for example the fear of snakes[1]. AI will probably have an "instinctive" liking for people, an an "instinctive" respect for human law, to the degree that the initial training set them up for it.
Utopian and not is a gradient, not a dichotomy, and where we land on the gradient is a matter of how things shape up in the next several years.
https://www.smu.edu/news/archives/2012/thomas-headland-mothe....
Re: AI boosted homework scores, then exam scores dropped: study
#394"Won't that be grand! Computers and programs will start thinking, and the people will stop!" —Walter Gibbs, Tron
Re: AI boosted homework scores, then exam scores dropped: study
#395Earlier quoted context omitted.
I just started teaching undergrad CS courses after ~20 years of various non-academic jobs and it took me about two semesters to realize that almost nothing matters except how I assess students. Two weeks of ADHD-fueled research later, I concluded that academia is actively resistant to implementing assessment reform because it would expose the utter pointlessness of most of what happens in university classrooms. The r…
> almost nothing matters except how I assess students This is a reasonable opinion after two semesters. IMO, with more experience you should have modified your stance. If all classes were pointless, students would leave at the same point they entered. Since we observe that is not the case, something is happening in class. Do more of that.
You’re conflating the course and the classroom.
The explicitly stated goal of most courses is mastery of a well-defined set of domain-specific knowledge and skills. The only way to objectively determine if the course achieved that goal for a given student is comprehensive assessment. Bad assessment design, much like bad experiment design in science, is worse than useless—it actively misleads and confuses.
A well-designed, repeatable, reliable assessment is a prerequisite before one can even consider the effects of particular pedagogical methods inside or outside of the classroom itself.
Re: AI boosted homework scores, then exam scores dropped: study
#396Earlier quoted context omitted.
Although surely highpass filter would make more sense in the analogy here?
It depends on college attendance. In developed countries, it is a low-excluding - or the inverse of a low-pass - filter as most graduate, while in developing countries it is a high-pass filter as few have the resources to reach an undergraduate level of education.
I'm confused by your point here. You seem to be saying it's a highpass in developing countries and also a highpass in developed countries?
Re: AI boosted homework scores, then exam scores dropped: study
#397Earlier quoted context omitted.
How do you expect students to learn anything when: 1. They don't do any homework. 2. All the in-class time is split between the teacher babysitting and playing social worker to problem students, and lecturing, with little to no opportunity to actually practice what they've learned? I understand that some students don't have home environments that are conductive to doing homework well. I understand that some students…
You don't have to do a lot of repetitive, boring exercises to learn piano. Once you learn how to hold your hands a little, read music, and learn where the notes are, you can simply try songs you like, and switch songs when you please. On other instruments, you might not need to read music (I've met many guitarists and drummers that cannot read music, nor do they use printed music to play - and blind folks can play in…
Are there a lot of concert pianists who got to the level that they did by just playing random songs they liked, without deliberately practicing the mechanics? And then, one day, woke up with the ability to quickly learn and perform unfamiliar pieces of music at the level expected of one?
You can absolutely learn to play some piano by just noodling. Your learning will be much slower, and you'll miss important fundamentals that are present in any piano method.
That's perfectly fine for learning things that you do for enjoyment, or things that don't really matter. It's a lot less fine for learning things that... do.
Re: AI boosted homework scores, then exam scores dropped: study
#398Earlier quoted context omitted.
It's all cute but naively glances over what this all boils to - competition. Education system, contrary to popular belief/name, isn't tailored to educate but to select winners and losers which then will be picked on the job market. That's why it seems absurd when you think about it as an institution that aims to educate. That's because that isn't the real purpose of it. The purpose is to stratify and classify early.…
> compliance, endurance of boredom, and social maneuvering Compliance and endurance of boredom I can somewhat agree with (thought not to a great extent), but social manoeuvring? As long as a student completes their classwork and homework, and does well in their exams (none of which mandate any social interaction apart from team projects), they easily pass through the school system. In fact, I used to hear about how t…
Yeah but if you get bullied due to lack of social skills then you won’t have mental energy nor drive for passing exams or any learning.
Socially savvy students on the other hand will get plenty of help from teachers and other students even if they are not so good themselves.
This is why the so called “intelligence” where I will define it as skill of acquiring skills right now, is not so important for success. You can be very good at learning but if your social aptitude and charisma sucks then you will just end up as depressed, miserable and poor and probably alone too.
If you have a kid It’s honestly very important and way more important than homeworks or exams, to care about kid’s social development and how you can help so your kid gains social confidence and never loses it at crucial moments
Re: AI boosted homework scores, then exam scores dropped: study
#399Earlier quoted context omitted.
This experimental setup would only tell you whether this demographic (grad students or professors, or top performing college students) benefits from access to LLMs for acquiring a new knowledge or skill. It would not prove that "the results depend on the operator". To prove this (and figure out the traits that make a proficient operator) you would either need a massive dataset to which you'd apply some machine learni…
Yes, I'm using "high-performing college students" or "phd students" or whatever as a proxy for "people skilled at learning". It's not perfect but I think it'd be good enough. You could do lots of refinements on the idea. You could try to do some kind of "pre-test" that more directly tested "meta-learning skills". My only point was I think it's possible to do. I personally think it's pretty clear. And there's nothing…
My understanding of this study is that it disproves your assumption: "The negative learning effects are larger for students with higher initial achievement."
It seems that the students who had the most skill and motivation from the start, have the most to lose.
Re: AI boosted homework scores, then exam scores dropped: study
#400Earlier quoted context omitted.
LLMs are not tutors and do not deeply know anything.
In the sense that they are not programmers and do not deeply know e.g. performance engineering, sure. But I can still give one a program and ask it to optimize it, and it will turn it into a zero allocation streaming SIMD implementation that's orders of magnitude faster than where it started. It may not "know" how to do any of that, but it does it anyway.
It may or may not do that and it may introduce other problems as it does so. Checking the work is not a trivial problem.