Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Coping with Copilot

sigarch.org

Post reply on HN
Print view View original

31–40 of 463 posts

Re: Coping with Copilot

#31

Post by russfink »

spcebar wrote: post #11

Cheating is getting easier by the day, but the bottom line is that if the student is cheating, they're probably not absorbing the coursework (or if they are cheating and absorbing the coursework, there might be something wrong with the coursework). The most effective policy might not be to try and mechanically prevent cheating, but to explain that, this is information that you're going to need if you actually want to…

Adjunct instructor here. Before I came into the program, cheating was on the rise. When given programming assignments, students would go to Geeks for geeks and download solutions, some of them mostly correct, and submit those for credit. One remedy was to try to make programming assignments harder and harder. Eventually, they got so hard that we were asking people new to computer science to implement a full arbitrary length integer calculator using nothing but a single tape Turing machine (in Java, not BF).

Eventually, some of us came to the realization that you can never prove that somebody is cheating. People have been known to hire tutors to do their assignments for them. There is just no end.

As a result, we evaluate students on four dimensions. Programming assignments,* homework assignments, but also class discussions and group participation. Those last two count for a small, but non-trivial percentage of the grade, and are usually enough to separate and identify those who understand what they are doing, from those who are just "following along" solutions that they find on the internet.

* one addition to programming assignments includes an analysis write up. Tell me in human words what is happening. Why do you see that effect. What is the running time. And separately, comment your code to tell me how it works. Those two parts count significantly towards the grade.

Any other professors here, what have you found that works?

Re: Coping with Copilot

#32

Post by niek_pas »

GrumpyYoungMan wrote: post #28
zitterbewegung wrote: post #4

Just add a requirement where the student has to explain how the code works. To be honest there should be a class called language model querying because for simple tasks it does well but eventually you will have to debug the output (not just copilot but GPT3 and dall-e).

Much like reverse engineering, it's easier to come up with an explanation when you have someone else's working solution to examine than it is to come up with a working solution from scratch on your own. So that doesn't really help.

Agreed, and, in addition, having every single student explain their program would be a significant burden on teaching resources.

Re: Coping with Copilot

#33

Post by muzani »

eru wrote: post #14
muzani wrote: post #10

This reminds me of when my school teacher insisted that real engineers didn't use calculators. They'd calculate their sines and cosines from a table. 3 years later, the education ministry decided that was BS and let everyone bring calculators to math exams. Surprisingly, math scores didn't go up much.

We never did much calculation in math exams. That's what physics class was for.

It was 8th or 9th grade for me when the switch happened. We were still doing areas and perimeters.

A calculator actually helped me understand physics a lot better, especially things like slopes.

Re: Coping with Copilot

#34

Post by ren_engineer »

spcebar wrote: post #11

Cheating is getting easier by the day, but the bottom line is that if the student is cheating, they're probably not absorbing the coursework (or if they are cheating and absorbing the coursework, there might be something wrong with the coursework). The most effective policy might not be to try and mechanically prevent cheating, but to explain that, this is information that you're going to need if you actually want to…

yeah Copilot really changes nothing here, all these basic questions can be found with a quick Google search. The reason Copilot is so good at these types of questions is lazy professors assigning the same basic stuff so there is a ton of training data.

instead of writing an article crying about it, the professor could try making some unique questions to test knowledge of the underlying concepts

Re: Coping with Copilot

#35

Post by adrian17 »

Extra nitpick regarding generated code: the `quicksort_random_pivot` implementation isn't what people commonly mean by "quicksort", as in: it's not in-place. This means that some properties like space complexity are also going to be different than what's usually expected of quicksort.

Re: Coping with Copilot

#37

Post by jefftk »

zitterbewegung wrote: post #27
jefftk wrote: post #23

Earlier quoted context omitted.

> there should be a class called language model querying Do you think the skills taught in that class would be relevant for very long?

Hypothetically if they became irrelevant wouldn't that imply that we have models that could just understand natural language?

Not necessarily. The skills could become irrelevant just because the models advance and you need new and different skills to query them effectively.

Re: Coping with Copilot

#39

Post by meltyness »

My proudest moment was in an embedded systems course, the teacher asked for a function to generate fibonacci linearly in assembly, there was a reaalllllly long pause, and I barked the answer across the classroom. I vividly remember some stunned, disgusted glances.

Re: Coping with Copilot

#40

Post by duxup »

spcebar wrote: post #11

Cheating is getting easier by the day, but the bottom line is that if the student is cheating, they're probably not absorbing the coursework (or if they are cheating and absorbing the coursework, there might be something wrong with the coursework). The most effective policy might not be to try and mechanically prevent cheating, but to explain that, this is information that you're going to need if you actually want to…

At that point should we count grades at all, other than a basic evaluation for the student’s personal use?
Post reply on HN