Now, what this tells me is that LLM AIs can supply the knowledge of the moment for the prepared human person, and the person does not need to learn that knowledge, for a extremely large class of problems. Which is fantastic for capitalism and business in general, and very very bad for humanity in general.
AI boosted homework scores, then exam scores dropped: study
351–360 of 409 posts
Re: AI boosted homework scores, then exam scores dropped: study
#352All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates. We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional mat…
It's all cute but naively glances over what this all boils to - competition. Education system, contrary to popular belief/name, isn't tailored to educate but to select winners and losers which then will be picked on the job market. That's why it seems absurd when you think about it as an institution that aims to educate. That's because that isn't the real purpose of it. The purpose is to stratify and classify early.…
> compliance, endurance of boredom, and social maneuvering
Compliance and endurance of boredom I can somewhat agree with (thought not to a great extent), but social manoeuvring? As long as a student completes their classwork and homework, and does well in their exams (none of which mandate any social interaction apart from team projects), they easily pass through the school system. In fact, I used to hear about how the school system disproportionately benefited studious, shut-in students over street-wise, socially-savvy ones.
Re: AI boosted homework scores, then exam scores dropped: study
#353Earlier quoted context omitted.
> But cheating isn't new. People have been cheating in school for centuries. It's easier now, but the consequences were always the same. At some point the chickens come home to roost and you pay the piper. My concern is that with LLMs and above average bullshitting abilities, the chickens might never come home to roost.
I think that is fine, certain things are becoming less important to learn and certain things more important. With the pace of change it is likely to continue shifting.
I really don't see how anything but social skills are rising in prominence. Perhaps twenty years from now everything will depend on your connections as any "hard" problems will be readily solved by AI (seeing as few problems are ever really novel).
Re: AI boosted homework scores, then exam scores dropped: study
#354Earlier quoted context omitted.
I meet a lot of noj-college people and let's just say, "just a low pass filter" is still A LOT of value.
Although surely highpass filter would make more sense in the analogy here?
It depends on college attendance. In developed countries, it is a low-excluding - or the inverse of a low-pass - filter as most graduate, while in developing countries it is a high-pass filter as few have the resources to reach an undergraduate level of education.
Re: AI boosted homework scores, then exam scores dropped: study
#355Earlier quoted context omitted.
Most domains have some kind of internal consistency/theory building you can do. A smart person can certainly notice inconsistencies when trying to learn something. In fact they're likely to be points of confusion that the smart person will dive into just to try to make sense of things, even if they don't suspect the LLM is at fault.
People who think they are smart often think they can get by in any domain without knowledge and context, but at a certain point this fails and knowledge is required, not just intelligence.
We're talking about learning here, i.e., building that knowledge and context. People also refer to more than one source, and I think you overstate the degree to which LLMs make mistakes in current year. Human teachers also make mistakes, and it's typical for a bright student who's paying attention to correct even small details.
Re: AI boosted homework scores, then exam scores dropped: study
#356The kids who used AI and studied for a similar amount of time as the non-AI high performers apparently had similar (slightly higher) performance. The kids who used AI and did poorly used the AI to do the homework for them (this is what the article says). I believe AI is basically an amplifier of bad and good. I’m cynical about the world and assume it will be used more for bad than good, but I don’t doubt some of the…
It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic: "At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generat…
I'm curious what happened in the source, that a copy/paste exposed such odd artifacts.
"e§ort" == effort
"Öve" == not sure. Maybe "over five"?
Re: AI boosted homework scores, then exam scores dropped: study
#357To the surprise of no one who’s doing any teaching these days.
Re: AI boosted homework scores, then exam scores dropped: study
#358Should have let the kids use AI during exam too, later in life they will be solving tasks with AI not manually. Just make the test questions difficult enough even with AI. I see our current approach like a driving school training its students at fast running.
The goal of education is not and should not be creating human robots for a specific type of job.
Re: AI boosted homework scores, then exam scores dropped: study
#359Earlier quoted context omitted.
> Depends who's using it. Like many tools the force multiplier depends on the operator. How would you prove/disprove this assumption without falling into a True Scotsman fallacy?
Fair question. In theory you could do an experiment where the subjects were grad students or professors, or top performing college students. Give them some fixed amount of time to understand some new subject on which they'll be tested, only 1 group has access to an LLM with appropriate context for the learning task, etc. That's just one idea...
This experimental setup would only tell you whether this demographic (grad students or professors, or top performing college students) benefits from access to LLMs for acquiring a new knowledge or skill.
It would not prove that "the results depend on the operator". To prove this (and figure out the traits that make a proficient operator) you would either need a massive dataset to which you'd apply some machine learning to figure out the correlations, or at least you'd need a hypothesis on the traits you want to test.
Do left-handed people perform better with LLMs? Analytical thinkers? Dyslexic people?
It's not enough to just claim "some individuals will perform better with these tools" without any notion of who does, and whether it's possible to become one of these people, and what is the expected gain in this group.
Otherwise, sure, we could sell unsecured chainsaws as a tool for making ice sculptures and observe that "some people" are indeed more proficient at not getting their face cut off ; but if the tool is being marketed to schools, such a vague statement is not helpful in making a case for it.
Re: AI boosted homework scores, then exam scores dropped: study
#360[flagged]