Live data from Hacker News

AI boosted homework scores, then exam scores dropped: study

economist.com

171–180 of 409 posts

Re: AI boosted homework scores, then exam scores dropped: study

#171

Earlier quoted context omitted.

This is my thoughts as well! It makes smarter people smarter and dumb people dumber!

It helps smart people be barely more effective and helps dumb (more importantly, people who do not value effort, people who are lazy, people who are self absorbed) people shit out endless streams of worthless tokens that can swamp out everything. Raising the noise floor like this only makes it that much harder to find "Smart" people, which we were already doing terrible at. I use Claude every single day, but this is…

> Raising the noise floor like this only makes it that much harder to find "Smart" people,

It does give us a new heuristic, though: people who are willing to completely cut generative AI out of their lives (cold-turkey, if you ever started using it) are a much smaller group of, predominantly thoughtful, people. You do have to give up Claude to be part of this group, but from what you say, that's no great loss, and no longer being deafened is worth it.

This has considerable advantages over conventional elitism, because the barrier-to-entry is negative in almost all cases.

The one exception I've found is assistive tech, where the state-of-the-art is so poor that vibecoded slop is genuinely an improvement over the state-of-the-art, and in many cases the tooling simply isn't available to make your own assistive tech (unless you want to bootstrap an entire networked computing environment, which isn't very helpful when you want to do your online banking and do not, in fact, work at your bank).

But there are not many principled exceptions where you could seriously argue that the trade-off is worth it. Take mathematics, for example, which we often see touted as a "good use-case" of generative AI. The primary advantage of generative AI in mathematics is being able to search though a vast corpus of ivory towers and inconsistent terminology (without proper attribution) to locate and connect ideas that can help solve problems. The deficiency this is addressing is elitism, inadequate communication, and inadequate indexing within academic mathematics. This problem is entirely created by the academic mathematicians, and has been known for nearly a century (per https://en.wikipedia.org/w/index.php?title=Nicolas_Bourbaki&...):

> Bourbaki was founded in response to the effects of the First World War which caused the death of a generation of French mathematicians; as a result, young university instructors were forced to use dated texts. While teaching at the University of Strasbourg, Henri Cartan complained to his colleague André Weil of the inadequacy of available course material, which prompted Weil to propose a meeting with others in Paris to collectively write a modern analysis textbook.

To my knowledge, this is the only organised project to clean up and improve mathematical communication. Everything else (Metamath, Mizar, AFP, Lean) is yet another ivory tower. The Wikipedia article on this topic (https://en.wikipedia.org/wiki/Mathematical_knowledge_managem...) risks deletion as non-notable, that's how little anyone's actually trying. They made their own bed, and generative AI will only provide a brief respite from having to lie in it. (I was surprised how many other "compelling" use-cases evaporated when I applied this razor to them: the sibling comment https://news.ycombinator.com/item?id=49392265 points out one such.)

Vibe-coding assistive tech which doesn't yet exist, as a temporary scaffold to improve the quality-of-life of yourself and others in a social world dominated by non-essential access barriers is, to my knowledge, the only exception to this principle that can be justified. If you treat people who make other excuses, or who don't even bother with excuses, as not worth listening to, you lose little – and doubly-so, if you make your stance clear, so that others know the "cost" of gaining your attention.

Re: AI boosted homework scores, then exam scores dropped: study

#172

Guess what this is probably fine. AI helps people do their work on a daily basis and probably makes them memorize less things. That’s how the world is moving. You don’t need to be able to memorize something for an exam to show mastery of a topic. And we shouldn’t organize our society around memorizing things for exams.

> to show mastery of a topic

How do you show mastery of a topic where you neither understand fundamentals nor have skills to do a task without help?

Mastery is about demonstrating skills. Skills take practice, hands down.

AI can help us build better tools, provide tutoring, answer broad ranging questions, help relate concepts, and do reasoning for us. All of those things can be good in that they can help us acquire skills faster, e.g. with better practices and exercises. But it cannot replace actual time spent doing a thing.

Wanna learn knitting? You're going to have to knit some stuff.

Re: AI boosted homework scores, then exam scores dropped: study

#173
post #163

All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates. We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional mat…

> If you try and force kids to study when they don't want to "for their own good" you're not actually helping them. Everyone needs to find their own path. Let people fail. this rhetoric is pretending to be an alternative to coercion. IMO the ideas you are talking about are well trodden and are still coercion nonetheless.

Coercion explains it perfectly. A lot of kids see no future because of the environment they live in, so they have exactly zero motivation to study

Re: AI boosted homework scores, then exam scores dropped: study

#174
post #163

All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates. We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional mat…

> F students will realize they have to study, start actually learning and then pass on the second time

Do you have evidence for this beyond your friends (who were accepted into college)?

Re: AI boosted homework scores, then exam scores dropped: study

#175
post #163

All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates. We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional mat…

> If you try and force kids to study when they don't want to "for their own good" you're not actually helping them. Everyone needs to find their own path. Let people fail. this rhetoric is pretending to be an alternative to coercion. IMO the ideas you are talking about are well trodden and are still coercion nonetheless.

I could be mistaken but my interpretation is that "studying for your own good" is being contrasted with "studying so you don't fail". If you actually fail students, they'll have a second motivation beyond trusting some authority figure's advice. Namely consequences.

Re: AI boosted homework scores, then exam scores dropped: study

#176
post #163

All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates. We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional mat…

> If you try and force kids to study when they don't want to "for their own good" you're not actually helping them. Everyone needs to find their own path. Let people fail. this rhetoric is pretending to be an alternative to coercion. IMO the ideas you are talking about are well trodden and are still coercion nonetheless.

You are missing the point. Getting a certificate for all of the programs you passed and still graduating could completely change the course of a person’s life. Community college has certificates and associates degrees to let people start working when they reach a practical limit in their general education.

Re: AI boosted homework scores, then exam scores dropped: study

#177
post #163

All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates. We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional mat…

> Homework should be optional material for self study

At Caltech, homework was assigned but had no bearing on your grade. The grades were based on the midterm and final exams.

But not mastering the homework usually resulted in flunking the exams. There were "retch" sessions after each homework assignment that was staffed by a grad student, and the purpose was to help the students understand the homework problems. I knew only one person (Hal Finney) who was so smart he didn't need to do the homework.

I learned the hard way that the path to success was:

1. never miss a lecture, no matter what

2. take notes by hand during lecture

3. do the homework on time, and make sure you understand every problem. Take advantage of the retch sessions.

And that worked for me.

Re: AI boosted homework scores, then exam scores dropped: study

#178
post #163

All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates. We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional mat…

It's all cute but naively glances over what this all boils to - competition.

Education system, contrary to popular belief/name, isn't tailored to educate but to select winners and losers which then will be picked on the job market.

That's why it seems absurd when you think about it as an institution that aims to educate. That's because that isn't the real purpose of it. The purpose is to stratify and classify early.

Schools do not operate in a vacuum; they serve as credentialing gatekeepers for a hyper-competitive capitalist job market. If everyone could easily retake exams until they got an A, grades would lose their primary utility for employers and universities: differentiation. Society relies on schools to provide a neat hierarchy of candidates that for one reason or another thrived in difficult environment of adolescent schooling.

The system often prioritizes compliance, endurance of boredom, and social maneuvering over actual critical thinking precisely because those traits align with corporate hierarchies.

Re: AI boosted homework scores, then exam scores dropped: study

#179

For some of the classes my kids were in, the lessons and homework time were reversed. Learning the material (using the text book and videos) was what they had to do at night and class time was spent working through problems or applying the material in some way.

This is typical at the college/university level - when I attended 20-25 years ago, we were expected to learn the material through readings outside of class, and then class time was spent on discussion, application, questions, and working through problems. IMHO there's a lot to recommend this style. A lot of students learn better when they can wrestle with the material at their own pace, on their own time, in their ow…

Although obvious, I think it's worth stating that this method mostly (only?) makes sense for higher education, where students are conscious, involved and engaged in their curriculum.
Post reply on HN