Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI boosted homework scores, then exam scores dropped: study

economist.com

Post reply on HN
Print view View original

331–340 of 409 posts

Re: AI boosted homework scores, then exam scores dropped: study

#331

Post by keybored »

a2ff6eeb0 wrote: post #76
keybored wrote: post #45

Earlier quoted context omitted.

Why one would care? Look at your comment from two days ago. https://news.ycombinator.com/item?id=49363319 > I hope we'll figure out some equitable society before that, but I'm not going to bet on it, so I'll do what I can to be on the side of the winners. Obviously people care if they think they might lose out. Is it going to be this hopes-and-prayers about egalitarianism/ah screw it I hope I get to oppress and domin…

I believe 4 things: 1) We're building a tool that can make toil of every kind obsolete. We're not there yet, but I'm watching it happen in real time. 2) The fact that it works so well economically means that it's inevitably going to take over. Humans simply won't be profitably employable, and this will happen sooner than you think. 3) The takeover could lead to a utopia, where nothing is needed of anyone, and you cou…

Points one and two are irrelevant to my comment, but that’s fine. I don’t expect any thoughts to be spent on whether not to deploy that stump speech.

What doesn’t make sense in points 3 and 4 are that the AI takeover is lead and powered by vile cretin scum who, in their former more honest days, were gleefully announcing the end of labor power. Being lead by them, how could it lead to “utopia”? Well, there are two possibilities that some AI evangelist might bring up.

1. It’s the volition of the AI that could lead to the “utopia”—they/it simply take charge after some threshold

2. No, it is still human-lead

This point (1) conflicts with your own point (4) though; the all-powerful (or converging on at least) AI is foiled by human sins? How?

The other point (2) conflicts with my point about how it is the vile cretins who are in charge of AI.

Re: AI boosted homework scores, then exam scores dropped: study

#333

Post by necovek »

331c8c71 wrote: post #182
necovek wrote: post #168

Earlier quoted context omitted.

I see this as a memorize-vs-understand fallacy many repeat. It is really a combination: you need to memorize a large corpus of information to operationalize knowledge — understanding a bunch of theorems in mathematics does not help much (even if you are able to prove them when you see them) if you can't remember the boundary conditions they hold under. Or having good understanding of foreign language grammar won't he…

> if you can't remember the boundary conditions they hold under. At the outskirts of one's knowledge this is inevitably the case. My impression it is still useful to know that a certain implication is possible - and one can look up the exact conditions.

This is why I said operationalizing: you need to memorize enough to either be able to know what to look up, or know it outright if you need to be fast(er).

Re: AI boosted homework scores, then exam scores dropped: study

#334

Post by grey-area »

ndriscoll wrote: post #158
grey-area wrote: post #156

Earlier quoted context omitted.

There is zero evidence for this. LLMs make significant mistakes frequently and smart people have no way of judging those mistakes outside their domain expertise. They are also sycophantic and great at being an echo chamber which makes people feel smart even if they are not. So I think the burden of proof is on you to prove that they somehow amplify intelligence, it seems highly unlikely.

Most domains have some kind of internal consistency/theory building you can do. A smart person can certainly notice inconsistencies when trying to learn something. In fact they're likely to be points of confusion that the smart person will dive into just to try to make sense of things, even if they don't suspect the LLM is at fault.

People who think they are smart often think they can get by in any domain without knowledge and context, but at a certain point this fails and knowledge is required, not just intelligence.

Re: AI boosted homework scores, then exam scores dropped: study

#335

Post by grey-area »

ACow_Adonis wrote: post #227
grey-area wrote: post #156

Earlier quoted context omitted.

There is zero evidence for this. LLMs make significant mistakes frequently and smart people have no way of judging those mistakes outside their domain expertise. They are also sycophantic and great at being an echo chamber which makes people feel smart even if they are not. So I think the burden of proof is on you to prove that they somehow amplify intelligence, it seems highly unlikely.

Why would a smart person go to an LLM for an answer they cannot judge or test, be succeptible to flattery and sycophancy rather than picking up on the emotional manipulation and being suspicious/sceptical of the interaction, or looking for support from an echo chamber rather than a Socratic opponent? All of those sound like flaws and defects of dumb people?

Many people think they are smart and are taken in by LLMs. Also many genuinely smart people are unable to judge their competence in other areas (eg Nobel Laureates making pronouncements in fields they know nothing about).

Re: AI boosted homework scores, then exam scores dropped: study

#336

Post by chrisjj »

the_real_cher wrote: post #11
GPerson wrote: post #7

The kids who used AI and studied for a similar amount of time as the non-AI high performers apparently had similar (slightly higher) performance. The kids who used AI and did poorly used the AI to do the homework for them (this is what the article says). I believe AI is basically an amplifier of bad and good. I’m cynical about the world and assume it will be used more for bad than good, but I don’t doubt some of the…

This is my thoughts as well! It makes smarter people smarter and dumb people dumber!

Unfortunately this results in overwhelming net dumbing because the dumb are the ones lacking the smarts to avoid the "AI".

Re: AI boosted homework scores, then exam scores dropped: study

#337

Post by chrisjj »

bsder wrote: post #298
the_real_cher wrote: post #11

Earlier quoted context omitted.

This is my thoughts as well! It makes smarter people smarter and dumb people dumber!

AI does not make me "smarter" in the important thing I am working on. In software particularly, what AI does do is give me back my time from the drudge work that I don't care about. Keeping my build system configured and my tests up to date and my documentation synchronized is a good use of AI because I mostly don't care about how crummy the result is as long as it "works".

> ... a good use of AI because I mostly don't care about how crummy the result is as long as it "works".

This is "AI"s forte. Producing more bad work at lower cost.

Re: AI boosted homework scores, then exam scores dropped: study

#338

Post by yoz-y »

idiotsecant wrote: post #208
moffkalast wrote: post #196

Earlier quoted context omitted.

I thought that it was generally accepted by now that homework in the volumes that it is being assigned in the modern day was not found to be beneficial in any significant way in the first place? Maybe once all students start outsourcing it to AI it might finally die like it deserves to. People these days grow up with almost no free time for themselves, it's all school, sports/extracurriculars or homework nonstop. All…

All play and no work makes for an even more dysfunctional society. Kids need to do their homework, both to train their minds but also to develop integrity and work ethic. Being able to diligently work toward a distant goal is not something you're born with.

I think there are better ways to teach this particular skill than homework. But I agree that practicing is very important. In hindsight I think that the concept of TD (classes where you only do exercises and have a teacher guide you if needed) in France is really good. (This is not the same thing as lab classes)

Re: AI boosted homework scores, then exam scores dropped: study

#339

Post by qeternity »

College stopped being about getting an education a long time ago.

It's just a low pass filter for the job market. Students aren't actually interested in learning anything, they have been condition for a long time to jump through the hoops.

It would be much better if we actually returned to exam and performance based evaluation of students, and stopped giving everyone As just for trying hard. But parents look at college fees as buying their kid a job opportunity, and so grade inflation has basically ruined the SNR of academic performance for all but the lowest performers.

Post reply on HN