"You have a 27% 'AI' issue in here" • Hacker News Board

Re: "You have a 27% 'AI' issue in here"

#201

Post by OkayPhysicist » Wed, Oct 04, 2023, 9:32 PM UTC

HDThoreaun wrote: post #13

nsxwolf wrote: post #7

Earlier quoted context omitted.

Use ChatGPT to write and administer 300 personalized oral exams.

The issue with different exams for each students is that it’s next to impossible to make them equally difficult which ruins the curve.

You've missed the point of curves. Curves exist as a different solution to the problem of "we don't know whether everybody understands the material or not, and we don't know how to evaluate whether they do". It's like saying "The problem with electric cars is that they don't fit the equipment for our emissions test"

If you give a pre-written test, you basically need to curve it because you don't really know how well the test aligns with your expectations of the class's understanding of the material. So you say "typically, about 20% of the class doesn't understand the course at all, 20% have a very good grasp..." etc, and then fit the written test results to the expected curve.

A one-on-one communication with a student allows you to directly gauge their understanding, allowing you to assign a grade. It doesn't matter how well or poorly the rest of the class understands the material, this student, sitting in front of me, has a "B" level understanding.

Re: "You have a 27% 'AI' issue in here"

#202

Post by yunwal » Wed, Oct 04, 2023, 9:34 PM UTC

9dev wrote: post #191

fnordpiglet wrote: post #107

Earlier quoted context omitted.

I find the “liberal arts” thing weird. First, as a scientist with an engineering degree, I definitely feel if you cheat through school you’re only cheating yourself. In the hard sciences a lack of knowledge and ability is pretty easy to suss out. The degree gets you the interview, the network might open a door or two, but the education is how you get the job. And once you’ve got it, if you didn’t learn anything, you…

> The degree gets you the interview, the network might open a door or two, but the education is how you get the job. What a great way to respond. I couldn’t agree more.

Unfortunately it’s not true for most positions in the tech industry, including the highest paying ones. All of these are game-able. Spend a month running through leetcode questions and learning some jargon, and it won’t matter whether you cheated your way through school, you’ll know enough to pass the interview.

Even on the job people can often get away with not providing much value if they have a bad/apathetic/misaligned manager.

Re: "You have a 27% 'AI' issue in here"

#203

Post by OkayPhysicist » Wed, Oct 04, 2023, 9:36 PM UTC

RandomLensman wrote: post #31

gustavus wrote: post #3

AI Detection services are the oiliest of snake oils. Seems like education is going to need to change drastically in the next few years. Maybe we'll go all the way back to oral examinations again.

Wouldn't just things were people sit in a room without or limited tools to write the exam work, too?

Or you can just trust your students. Most upper-division courses I took at university gave take-home exams. "Here's the test, come back with it in a couple days, don't talk about your answers with other students".

Re: "You have a 27% 'AI' issue in here"

#204

Post by fluoridation » Wed, Oct 04, 2023, 9:38 PM UTC

dmm wrote: post #73

jonathanstrange wrote: post #55

This is apparently unpopular but my take on this is that if teachers want students to write something on which AI can get high scores, then whatever they're teaching is obsolete and will be replaced by AIs anyway. Of course, not mentioning AI as co-author remains cheating. But I think the best course of action is to simply allow students to use AI while mentioning it (and the prompts they've used). I'm in philosophy…

> if teachers want students to write something on which AI can get high scores, then whatever they're teaching is obsolete Doing trivial things is often the first step in learning how to do non-trivial things. No one has ever needed a five paragraph essay. It's a completely artificial construct that, hopefully, teaches people to structure an essay. Electronic calculators have existed for decades but we're still teach…

>we're still teaching kids arithmetic.

I remain unconvinced that it's a worthwhile use of time.

Re: "You have a 27% 'AI' issue in here"

#205

Post by Meekro » Wed, Oct 04, 2023, 9:40 PM UTC

nicbou wrote: post #111

HDThoreaun wrote: post #4

Earlier quoted context omitted.

I doubt oral examinations come back in general. The undergrad model relys on massive classes, I don’t see how you can do oral exams in those 300 person classes without massive cheating as the first students tell everyone else what to expect.

One thing I don't understand is that despite automating more and more of the work away, we seem to have fewer and fewer humans available for anything that's left. I have to scan my own groceries, I can't ever talk to an actual human at any business, I order my food from a machine. Where is everyone?

I've wondered about this myself. It's not just grocery scanning-- my parents used to get a bunch of bills in the mail every month, and send out a bunch of checks to pay them. It's stunning to think about how many people were needed at every step of that pipeline to make the whole thing work. All of that's automated now, and you can find similar things in many industries.

That should have freed up lots of people to handle the special cases that fall through the cracks of the automation, but they're nowhere to be found.

Re: "You have a 27% 'AI' issue in here"

#206

Post by em-bee » Wed, Oct 04, 2023, 9:43 PM UTC

oceanghost wrote: post #110

garciasn wrote: post #9

I am an Eagle Scout and was awarded it in 1995. As part of that process, I had to write up a detailed document outlining my project including how it was conceived, the work, and the outcome. It was rejected on my initial submission because the reviewers felt I it wasn’t written by a sophomore in HS. My parents rewrote the entire thing in what they described as “baby talk” and it was then accepted. The irony is not lo…

I had a similar experience in the 5th grade. We had a creative writing assignment (write x many pages), and I wrote a short story about "space spiders" that spun giant webs between planets and moons to catch something. I can't recall. The only thing more mundane and unimaginative than that two pages was my teacher, who accused me of plagiarizing it.

and i bet the teacher could not tell you which story you were supposed to have this plagiarized from.

in a way it's a compliment that your teacher thought the story was so good, that they didn't believe it was your own.

i have had this happen in sports. i was not very sportive and tended to pick easier activities when there was a choice. there was this school sports day where the whole school was doing activities outside, and one of those activities was running for half an hour, one hour or two hours. usually i'd pick the half hour to be done with it, but this one day, my brother, a few friends and me decided to run in a group and do the two hours.

doing things in a group can be a lot easier as everyone encouraged each other. we also didn't race but it was a relaxed slow jog.

at the end we would get a pass indicating that we had completed the activity. when i showed that to my teacher, he didn't believe me.

there was no consequence of that, but if necessary i would have had three witnesses.

in hindsight the disappointment was that the teacher did not understand or recognize my capacity. i don't think he knew that i had been biking for hundreds or even thousands of km during school holidays.

Re: "You have a 27% 'AI' issue in here"

#207

Post by vineyardmike » Wed, Oct 04, 2023, 9:44 PM UTC

csours wrote: post #133

It feels more and more like we need to rethink what scoring schoolwork means. Worksheets and exams can show how familiar you are with the material, or they can be gamed or useless in some other way. But the best they can ever do is show familiarity. What I really want as a consumer of someone's grade or academic history is how well they WORK in the subject area, along with how well they work with other people, not ju…

I like your direction of thought, but I think we should make sure we don’t over correlate education with vocational utility. School isn’t always about work, and not all education should be about work or jobs or whatever.

That’s why I really like that you included how well they work with others.

Personally, I never felt like school was about memorizing facts (maybe some history classes…), but I also had two parents with advanced degrees who repeatedly told me that school was about learning to think and learning to learn (and learning to time manage). I feel like the vocational focus a lot of people have on education really hurts some fields of study - the humanities gets a lot of hate both on HN and in the media.

Ideally, classes with projects and essays should push people to think and show understanding of the subject not facts, like you said. The infinitely mocked “why did the author say the curtains were green” tropes actually show this - understanding why authors use certain imagery requires understanding intent, historic context, deeper understanding, etc. especially when applied to new works. I think people like cut and dry fact memorization and objective grading, and this doesn’t fit that cleanly.

Re: "You have a 27% 'AI' issue in here"

#208

Post by vineyardmike » Wed, Oct 04, 2023, 9:53 PM UTC

nicbou wrote: post #111

HDThoreaun wrote: post #4

Earlier quoted context omitted.

I doubt oral examinations come back in general. The undergrad model relys on massive classes, I don’t see how you can do oral exams in those 300 person classes without massive cheating as the first students tell everyone else what to expect.

One thing I don't understand is that despite automating more and more of the work away, we seem to have fewer and fewer humans available for anything that's left. I have to scan my own groceries, I can't ever talk to an actual human at any business, I order my food from a machine. Where is everyone?

As you may have heard, in the US at least, there’s record low unemployment. There’s literally not enough people to do those jobs. It’s complex why…

1. Demographic changes means more retirees, and smaller workforce. Literally less people to go around.

2. With low unemployment, everyone’s doing something else. Something harder to automate, or otherwise more valuable. It’s not like people are sitting around unemployed (except retirees).

What are they doing? It’s hard to know without combing through data in this moment, but people are surely avoiding minimum wage and low skill customer service jobs if alternatives exist in their area. Anecdotally, I hear people don’t like working in anything customer facing due to rude customers. I know someone in TX working at Amazon warehouses at an hourly rate much higher than retail workers rate. They used to work at a gym in rural California.

Until we get rising low-skill immigration to cover more of the low-wage work, expect service work to be understaffed. Or rising birth rates, or a drop in white collar workers.

Re: "You have a 27% 'AI' issue in here"

#209

Post by [deleted] » Wed, Oct 04, 2023, 10:14 PM UTC

csours wrote: post #133

It feels more and more like we need to rethink what scoring schoolwork means. Worksheets and exams can show how familiar you are with the material, or they can be gamed or useless in some other way. But the best they can ever do is show familiarity. What I really want as a consumer of someone's grade or academic history is how well they WORK in the subject area, along with how well they work with other people, not ju…

[deleted]

Re: "You have a 27% 'AI' issue in here"

#210

Post by tatersolid » Wed, Oct 04, 2023, 10:27 PM UTC

yohannparis wrote: post #60

Miraste wrote: post #21

Earlier quoted context omitted.

>if they cheat they are only cheating themselves anyways. I've heard people say this a lot. It's a very old-school liberal arts perspective that doesn't fit with the modern world. In practice, for most students, education is a third-place benefit of university behind 1. the certificate and 2. networking.

Again, this is a north-americas point of view where most universities and colleges are "private" businesses.

As I recall from my kid’s college search, the majority of US college students attend state schools or community colleges. While there are very many private colleges in number, they are each usually much smaller than state schools in enrollment. Private schools are also vastly outnumbered by community colleges, even though community colleges are on average smaller in enrollment..