Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI boosted homework scores, then exam scores dropped: study

economist.com

Post reply on HN
Print view View original

181–190 of 409 posts

Re: AI boosted homework scores, then exam scores dropped: study

#181

Post by yorwba »

tancop wrote: post #163

All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates. We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional mat…

The study was done in China, where making the class easier to improve pass rates isn't really a thing, as the Gaokao operates more like a stack ranking where it doesn't matter much how well you did in an absolute sense, only that not too many others who did better than you are competing for the same spot. Unlimited retakes are possible in theory, except each costs you a year of your life and except for some extreme cases of repeat test-takers, most people are going to give up after just one or two bad results. Homework is definitely not optional, but going home from school is. (Due to the hukou system, children often have to go to school far from where their parents work, so boarding schools with teacher-supervised self-study are common.) I don't see how holding back for individual classes is supposed to work considering scheduling conflicts.

Re: AI boosted homework scores, then exam scores dropped: study

#182

Post by 331c8c71 »

necovek wrote: post #168
shuwix wrote: post #62

Education sustem is not about the actual knowledge, but about training various aspects of the brain during kids development. So even memorizing literature have benefits, even though it doesn't bring any career use. If you skip this, well your brain won't develop as much at period of life it's able to do so.

I see this as a memorize-vs-understand fallacy many repeat. It is really a combination: you need to memorize a large corpus of information to operationalize knowledge — understanding a bunch of theorems in mathematics does not help much (even if you are able to prove them when you see them) if you can't remember the boundary conditions they hold under. Or having good understanding of foreign language grammar won't he…

> if you can't remember the boundary conditions they hold under.

At the outskirts of one's knowledge this is inevitably the case. My impression it is still useful to know that a certain implication is possible - and one can look up the exact conditions.

Re: AI boosted homework scores, then exam scores dropped: study

#183

Post by magicalhippo »

nxc18 wrote: post #162
magicalhippo wrote: post #4

Earlier quoted context omitted.

> Homework scores once predicted exam performance As with most training the journey is the point, not the destination. That said, I think smart use of AI could help. It could explain concepts in a way that might help you understand better, it could probe your knowledge in a more dynamic way by tailoring questions, and so on. This requires the AI be restrained by some harness, not free to write down the answers for yo…

> help. It could explain concepts in a way that might help you understand better People say that all the time, but does anyone really think a lack of good explanations for things is a limiting factor in 2026? Or even 2010?

Back when I was studying there were topics which I had a really hard time grasping. Having a personal tutor that I could ask specific questions and have a back and forth with would have helped a lot I think. At least the few times I did have such an opportunity that was definitely the case.

Re: AI boosted homework scores, then exam scores dropped: study

#184

Post by mettamage »

ezst wrote: post #169
Nevin1901 wrote: post #165

Earlier quoted context omitted.

School has always been about creating compliant workers, not educating people. They will double down on "performing the right things" and "morals" while the economy will continue on its K shaped path as AI gets more capable.

Could it be that you vastly overgeneralize? While there's no perfect education system, many are much better than the nightmare you describe.

Yea, also the programs are too different. My psychology bachelor felt like a tea party. My computer science master was 60 hours per week during the hardest courses (which were supposed to go for 20 hours per week - so I could only do one course if it was at this level). My game design master felt more like we were dumped into an art school masquerading as a psychology/CS master but it really was art school. My information science bachelor felt like the only "normal" study program.

All of this was at university of which 3 of them were at the same university.

Especially the artsy game design program definitely did not feel like it was preparing me to be a cog in some giant corporate wheel.

Re: AI boosted homework scores, then exam scores dropped: study

#185

Post by mpalczewski »

When you get out into the real world, what will matter is how effectively you use AI. When you are done with school, you can take the "exam" using AI. The problem is that teaching is trying to teach skills that are no longer relevant and is always behind what the latest going on in the industry. Every time without fail you got interns, who after two years at stanford would learn more in the 3 months on a google internship than in those two years at stanford. They learn useless shit.

My own experience is that college is fun. You can spend the time wisely, but school stuff, is not what makes you money.

Make student loans dischargeable in bankruptcy and make colleges under write them. The problem will get solved quickly.

Re: AI boosted homework scores, then exam scores dropped: study

#186

Post by WalterBright »

WalterBright wrote: post #177
tancop wrote: post #163

All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates. We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional mat…

> Homework should be optional material for self study At Caltech, homework was assigned but had no bearing on your grade. The grades were based on the midterm and final exams. But not mastering the homework usually resulted in flunking the exams. There were "retch" sessions after each homework assignment that was staffed by a grad student, and the purpose was to help the students understand the homework problems. I k…

I've given this advice many times, and the ones who followed it did well. I only added "leave your laptop in your dorm room" for modern times.

The general pattern at Caltech was 2 hours of study for every hour of lecture. Which was quite a shock to me.

Re: AI boosted homework scores, then exam scores dropped: study

#187

Post by jedberg »

oblio wrote: post #155
magicalhippo wrote: post #4

Earlier quoted context omitted.

> Homework scores once predicted exam performance As with most training the journey is the point, not the destination. That said, I think smart use of AI could help. It could explain concepts in a way that might help you understand better, it could probe your knowledge in a more dynamic way by tailoring questions, and so on. This requires the AI be restrained by some harness, not free to write down the answers for yo…

> This requires the AI be restrained by some harness How would this magic harness look like and why would anyone use it?

"I need help with my homework, but I don't want answers, just guidance on how to get to the answer".

Put that in your agents.md.

Re: AI boosted homework scores, then exam scores dropped: study

#188

Post by slaymaker1907 »

tancop wrote: post #163

All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates. We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional mat…

Unlimited retakes would be an enormous amount of work for professors/TAs/teachers.

Optional homework is often a disaster. At best, students would do it right before an exam and the goal of education is not to just pass exams. They’d probably still get a lower score than if they did the homework when they were supposed to.

What I think is better is to have a due date, but just make the maximum 10% each day it is late. So after 2 days, the highest score you could receive would be 80%.

I liked that system because it gave some flexibility with deadlines while still encouraging you to turn things in on time.

Re: AI boosted homework scores, then exam scores dropped: study

#189

Post by jedberg »

nxc18 wrote: post #162
magicalhippo wrote: post #4

Earlier quoted context omitted.

> Homework scores once predicted exam performance As with most training the journey is the point, not the destination. That said, I think smart use of AI could help. It could explain concepts in a way that might help you understand better, it could probe your knowledge in a more dynamic way by tailoring questions, and so on. This requires the AI be restrained by some harness, not free to write down the answers for yo…

> help. It could explain concepts in a way that might help you understand better People say that all the time, but does anyone really think a lack of good explanations for things is a limiting factor in 2026? Or even 2010?

When I was in college, we had office hours with the prof and the TAs, we had group study sessions, and private tutors. Or you could ask your friends. All of those required going somewhere at a specific place and time and asking someone else to give up their time for you.

AI gives you a way to get the same help without asking another human. Say of that what you will, but not everyone was comfortable asking other humans for help even back then.

Re: AI boosted homework scores, then exam scores dropped: study

#190

Post by Barbing »

grey-area wrote: post #156
the_real_cher wrote: post #11

Earlier quoted context omitted.

This is my thoughts as well! It makes smarter people smarter and dumb people dumber!

There is zero evidence for this. LLMs make significant mistakes frequently and smart people have no way of judging those mistakes outside their domain expertise. They are also sycophantic and great at being an echo chamber which makes people feel smart even if they are not. So I think the burden of proof is on you to prove that they somehow amplify intelligence, it seems highly unlikely.

No evidence but somewhat of a counterpoints:

Smart people know LLMs confabulate and tell them they’re Absolutely Right! Smart people don’t want to be embarrassed by trusting the hallucination machine and revealing their gullibility to others.

Post reply on HN