College students are learning hard lessons about anti-cheating software • Hacker News Board

Re: College students are learning hard lessons about anti-cheating software

#61

Post by jccalhoun » Mon, Nov 30, 2020, 4:35 PM UTC

tomp wrote: post #26

jccalhoun wrote: post #17

I'm a professor in the humanities so I can write tests that make it harder to cheat than some other fields. I cringe whenever I hear someone using these invasive kinds of software. There has to be a better way than making students install spyware.

> I can write tests that make it harder to cheat than some other fields This might be my prejudice speaking (I studied math) but isn't this basically because grading is that much more subjective? So it would cut both ways, harder to cheat but also less objective / impartial assuming no cheating.

Not necessarily. I do a lot of questions that are here is a situation. What concept does this illustrate? Now sure, they can look up the definition of the concepts but they will still need to be able to identify how the concept relates to the situation.

Re: College students are learning hard lessons about anti-cheating software

#62

Post by jjk166 » Mon, Nov 30, 2020, 4:40 PM UTC

RegnisGnaw wrote: post #22

jjk166 wrote: post #18

Earlier quoted context omitted.

What stopped people from having someone else do their take home tests before the pandemic? On a more constructive note, it seems pretty simple to require some portion of responses to reference the lectures such that only someone who attended the class in this semester would be able to correctly answer, for example "use the method we discussed in the first half of lecture 3, use the third of the four numbers I told yo…

>> What stopped people from having someone else do their take home tests before the pandemic? Nothing except these tests were not take home prior to COVID. >> On a more constructive note, it seems pretty simple to require some portion of responses to reference the lectures such that only someone who attended the class in this semester would be able to correctly answer, for example "use the method we discussed in the…

> Nothing except these tests were not take home prior to COVID.

Take home tests were definitely a thing before covid.

> So you are expecting the student to remember 100% of what they heard in the online classes? Or more likely the student would write down these details and handle it to the cheater to use during the take home exam. Or the cheater would "do X". In most cases the one doing the helping is not a current student, so it doesn't help.

I expect students to take notes for their open book exams. If you're competent enough to identify all the material needed for the cheater to do an open book test, congratulations you have the knowledge to pass the test. If you just record every piece of information possible, congratulations you have spent way more time and effort than it would have taken to just learn the material.

If you don't attend any classes, do X. You're not being penalized for doing that, it just warrants scrutiny. If this is an entry level course that an intelligent person could teach themselves the material, then it probably doesn't matter if you cheated or not, you'll be found out in higher level courses. If this is a high level course, your department would probably have a good idea of how capable you are from past performance. If the guy struggling to get a C in introductory physics gets a perfect score on his quantum final without going to class once, that is super suspicious.

Re: College students are learning hard lessons about anti-cheating software

#63

Post by caymanjim » Mon, Nov 30, 2020, 4:40 PM UTC

College students are easy to manipulate because they're young and naive and still used to taking orders from control figures like their parents. They need to realize the power they wield and simply refuse to participate in this intrusive monitoring. They don't have to play the game, and the school can't exist without their tuition.

Re: College students are learning hard lessons about anti-cheating software

#64

Post by throwaway0a5e » Mon, Nov 30, 2020, 4:42 PM UTC

klodolph wrote: post #51

throwaway0a5e wrote: post #40

Earlier quoted context omitted.

I don't think you meant to do this but you seem to have inadvertently made a very damn good argument for "guns don't kill people people kill people. People are getting screwed because they are at the long end of an unbroken chain of crap. Crappy organizations buy crappy software and crappy professors take the results seriously. The fact that there exists a crappy tool that flags all the black people as cheaters (or w…

This argument is fallacious. Your argument assumes that “blaming the gun” is condoning the users, and this is a false dilemma. There’s really no room for absolutism, where blame is assigned to one source rather than distributed among many contributing factors. Imagine how dangerous air travel would still be if, after an accident, investigators looked for only one cause to blame, and tacitly condoned anything else the…

>This argument is fallacious.

Re-read my comment. My point is that fault is distributed sufficiently that accountability seems to evaporate. This is a systemic or organizational problem. Anti-cheating software is just the form this specific instance has taken.

>Your argument assumes that “blaming the gun” is condoning the users

Well until now you weren't throwing even the slightest bit of shade in their direction.

>and this is a false dilemma.

And you've created a false middle ground.

>Imagine how dangerous air travel would still be if, after an accident, investigators looked for only one cause to blame, and tacitly condoned anything else they came across in their investigations.

What's the difference between "dangerous harmful cheating software" and "cheating software that's being shoehorned into use cases in which it was never expected to be used"?

That's why you don't blame the (metaphorical) gun. Everything is just tools.

The FAA doesn't go off half cocked about the evils of grade-2 fasteners because once upon a time an engineer thought a grade-2 would be enough when he should have used a grade-5. I can't believe I have to defend (invasive to the point of being unethical) anti cheating software but these sorts of software tools are just tools and can be used either wisely or poorly. The software doesn't know or care how it's being used. In an industrial setting they can (and are, same underlying tech different companies) be used to design more effective interfaces to reduce errors (which I think we would all agree is a net positive contribution to the world).

Re: College students are learning hard lessons about anti-cheating software

#65

Post by wccrawford » Mon, Nov 30, 2020, 4:47 PM UTC

klodolph wrote: post #19

wccrawford wrote: post #6

IMO, the problem isn't the software. It's how humans are using the software. The software itself shouldn't have any control over the student's grades. A person should have to review the flags and actually find some wrongdoing. Not just push 'yes' and walk away.

This is an echo of the “guns don’t kill people, people kill people” argument that has been going around for ages. Let’s say you’re only using the software to flag suspicious behavior, and bringing in humans to make the final decision. What happens when (inevitably) the software disproportionally flags people with dark skin because it is not trained to recognize dark-skinned faces? Or when the software disproportional…

I will agree that certain biases are almost certainly baked into the software and that they will disadvantage anyone (and any situation) that isn't considered 'normal' by the software's creators.

If your argument is that they should be paying a person to sit here and watching a class of students while they're doing the tests, I'm not against that. They probably should.

But humans will always attempt to make their own work easier and less time-consuming, and this is a tool for that. Eventually, something like this is going to exist for distance learning. This is unlikely to be the final configuration of that tool, but it's a step on that road, no matter how much people don't like it.

What's needed are proper controls on the usage of the tool. And proper training. And proper oversight.

If your argument is for something else instead of the above, then I don't know what your solution would be. "Don't have school" and "don't worry about cheating" aren't acceptable.

Re: College students are learning hard lessons about anti-cheating software

#66

Post by gruez » Mon, Nov 30, 2020, 4:51 PM UTC

iandanforth wrote: post #52

A few points: Their "powerful AI engine" is almost certainly just humans. It might have a few off-the-shelf components like face detection but most of what they claim to do is just so easy to outsource that almost all companies do it. If there is any delay between the system observing a suspect behaviour and the student being told to correct it then they are definitely using humans. An institution using a service lik…

>Ryan Calo (Prof of Law, UW) once presented a fascinating bit of research on trying to automate something as simple as fining someone for speeding. [...]

Most of the problems you've listed only exist because of expectations caused by inconsistent/lenient enforcement by human police officers. People don't seem to have a problem with strictly enforced rules in finance. eg. "your bank account can't below zero without triggering a overdraft".

Re: College students are learning hard lessons about anti-cheating software

#67

Post by Balgair » Mon, Nov 30, 2020, 4:55 PM UTC

pdkl95 wrote: post #53

> At the self-described “heart” of the company’s monitoring software is Monitor AI, a “powerful artificial intelligence engine” that collects facial detection data [...] to identify “patterns and anomalies associated with cheating.” > "... people who have some sort of facial disfigurement have special challenges; they might get flagged because their face has an unexpected geometry.” So this company is extrapolating "…

> This is just phrenology ...

It's probably worse. I'd bet that in addition to humans with non-'normal' heads, it would also flag the blind, deaf, sufferers of Tourette syndrome, certain muslim women, bearded men, humans of african descent, humans of asian descent, etc.

> even if the calipers[1] are replaced with a bunch of linear algebra.

I love this line. If your head does not fit the space of eigen-heads, throw an error.

Re: College students are learning hard lessons about anti-cheating software

#68

Post by rescbr » Mon, Nov 30, 2020, 5:52 PM UTC

RegnisGnaw wrote: post #37

parzivalm wrote: post #25

The thing that has always frustrated me about anti-cheating/anti-plagiarism software is that it almost always only hurts people that did something by accident or unintentionally. When I took the one kubernetes certification exam as part of an old job, you used software like this and at one point I leaned too close to the screen and the proctor couldn't see my face and that got me flagged. People that want to cheat or…

>> People that want to cheat or game the system will find ways, it isn't hard to do. As someone who taught a college, that's a flawed statement. Let me propose this to you. You teach Calculus 1000 and there is an end of term exam. Your normal end of term exam is one where everyone sits in the same room, proctors are looking for cheats, things checked, etc. Instead this year you tell your students that this year it wi…

I had my fair share of uncheatable open book exams, calculators allowed back in college. One I had on Calculus (or was it Linear Algebra?) was way harder than a comparable closed book exam. You really needed to know the subject to pass vs just memorizing a couple of formulas and plugging them in the right place.

Another professor devised an exam that used your student ID as a variable of the first question, and subsequent questions used the previous answers as inputs. Impossible to cheat.

Re: College students are learning hard lessons about anti-cheating software

#69

Post by tgv » Mon, Nov 30, 2020, 6:10 PM UTC

klodolph wrote: post #11

tgv wrote: post #5

Earlier quoted context omitted.

That's no reason to lower the bar even further.

That’s a curious way to frame the problem. The harm from these anti-cheating measures is evident.

The one I replied to gives as reason: you can't stop them all. Let's just give everyone a MSc, shall we?

Re: College students are learning hard lessons about anti-cheating software

#70

Post by alistairSH » Mon, Nov 30, 2020, 6:15 PM UTC

querez wrote: post #44

parzivalm wrote: post #25

The thing that has always frustrated me about anti-cheating/anti-plagiarism software is that it almost always only hurts people that did something by accident or unintentionally. When I took the one kubernetes certification exam as part of an old job, you used software like this and at one point I leaned too close to the screen and the proctor couldn't see my face and that got me flagged. People that want to cheat or…

> One of the issues is also just academia being so hyper-focused on thinking everyone is there to become an academic which is not the case for the vast majority of people I'm not sure I follow why this is bad. I think academic rigor and being a decent scholar, as well as being able to parse and produce research are good things (and in my mind, those are the corner stones of being an academic). Did misunderstand you?…

in germany (and likely other european countries) we have a two tier system for higher education, consisting of universities and "applied universities", with the latter focusing on applied skills and the former focusing on research

While it isn't codified, we effectively have this in the US as well.

Most of the "brand name" universities, plus the flagship state universities, conduct research and grant various doctoral degrees.

Then we have the middle-tier colleges (state and private) that grant masters (often only professional degrees like nursing, MBAs, etc).

And thousands of Baccalaureate-only and 2-year community colleges.

Also, in the US, "university" generally indicates a post-graduate degree granting institution. And "college" usually refers to a 2-year and Baccalaureate-only school. But, also not codified and there are notable exceptions (ex: The College of William & Mary is a top-notch full university who's name pre-dates the convention).