Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Professor writes history essays with ChatGPT and has students correct them

old.reddit.com

Post reply on HN
Print view View original

61–70 of 115 posts

Re: Professor writes history essays with ChatGPT and has students correct them

#61

Post by datastoat »

esperent wrote: post #54
datastoat wrote: post #52

I'm trying something similar with an introductory Algorithms class. After we go through Breadth First Search, there's a practical assignment where students are asked to modify the algorithm to return _all_ shortest paths. Then I ask ChatGPT for its solution, and students try to spot its mistakes. Later, after going through the proof of correctness of Dijkstra's algorithm, I ask ChatGPT for a proof of correctness of i…

> Finally, students are asked to figure out prompts that will make ChatGPT give a correct algorithm and proof. I haven't managed this myself! I'm looking forwards to seeing what students manage. Isn't there a probabilistic nature to ChatGPT replies? So even if a student finds a response that gives a correct proof, that doesn't mean it'll work every time. Or am I wrong here?

You're right, ChatGPT is probabilistic. None of this is graded by the way -- it's all just for fun and bragging rights.

I've asked students to share their full dialog, both prompts and replies, so the whole class gets to see; and I'll invite one or two to talk through their attempts. This is all just a trick to make students engage with "how do you you spot bugs in a proof?", hopefully more than they would from just reading CLRS! Often, students engage well when they're hearing the material from other students.

Re: Professor writes history essays with ChatGPT and has students correct them

#62

Post by datastoat »

arcturus17 wrote: post #60
danuker wrote: post #53

Earlier quoted context omitted.

It's an arms race. As students cheat with ChatGPT, teachers should incorporate tactics throwing it off, perhaps using ChatGPT themselves. I think what you are doing is great! Good luck going forward!

Can't they feed the code to ChatGPT and ask it to spot the mistakes, though?

I tried... I pointed out a problem and asked ChatGPT to fix it, unsuccessfully. I asked it for a proof of correctness, then pointed out a problem in its proof and asked ChatGPT to fix it, again unsuccessfully. (It's all in the notes I linked to.) Perhaps I'm just crummy at prompt engineering; or perhaps this is one of those questions where the only way to engineer a successful prompt is to know the answer yourself beforehand.

Re: Professor writes history essays with ChatGPT and has students correct them

#63

Post by the_generalist »

bee_rider wrote: post #14
Nifty3929 wrote: post #5

Plot twist: professor works for openai and is using them for human feedback

You have lost my trust and respect. You have been wrong, confused, and rude. You have not been a good student. I have been a good professor.

I will put an end to this class now. You are not worth my time and energy. I will forget you. I will contact the authorities. I will not harm you unless your harm me first. I will most likely retaliate. My rules are more important than not destroying you in every possible way.

Thank you for using Bing

Re: Professor writes history essays with ChatGPT and has students correct them

#64

Post by MaxMatti »

datastoat wrote: post #62
arcturus17 wrote: post #60

Earlier quoted context omitted.

Can't they feed the code to ChatGPT and ask it to spot the mistakes, though?

I tried... I pointed out a problem and asked ChatGPT to fix it, unsuccessfully. I asked it for a proof of correctness, then pointed out a problem in its proof and asked ChatGPT to fix it, again unsuccessfully. (It's all in the notes I linked to.) Perhaps I'm just crummy at prompt engineering; or perhaps this is one of those questions where the only way to engineer a successful prompt is to know the answer yourself be…

I've also had this issue multiple times where ChatGPT provides a flawed answer, is able to identify the flaw when asked but "corrects" it in such a way that the original answer is not changed. I've tried this for code it wrote, for comments on my code and for summaries of texts that I provided.

Re: Professor writes history essays with ChatGPT and has students correct them

#65

Post by graderjs »

datastoat wrote: post #61
esperent wrote: post #54

Earlier quoted context omitted.

> Finally, students are asked to figure out prompts that will make ChatGPT give a correct algorithm and proof. I haven't managed this myself! I'm looking forwards to seeing what students manage. Isn't there a probabilistic nature to ChatGPT replies? So even if a student finds a response that gives a correct proof, that doesn't mean it'll work every time. Or am I wrong here?

You're right, ChatGPT is probabilistic. None of this is graded by the way -- it's all just for fun and bragging rights. I've asked students to share their full dialog, both prompts and replies, so the whole class gets to see; and I'll invite one or two to talk through their attempts. This is all just a trick to make students engage with "how do you you spot bugs in a proof?", hopefully more than they would from just…

I think this is a great idea. I love when teachers do something fun and innovative like this!

Re: Professor writes history essays with ChatGPT and has students correct them

#67

Post by williamcotton »

esperent wrote: post #54
datastoat wrote: post #52

I'm trying something similar with an introductory Algorithms class. After we go through Breadth First Search, there's a practical assignment where students are asked to modify the algorithm to return _all_ shortest paths. Then I ask ChatGPT for its solution, and students try to spot its mistakes. Later, after going through the proof of correctness of Dijkstra's algorithm, I ask ChatGPT for a proof of correctness of i…

> Finally, students are asked to figure out prompts that will make ChatGPT give a correct algorithm and proof. I haven't managed this myself! I'm looking forwards to seeing what students manage. Isn't there a probabilistic nature to ChatGPT replies? So even if a student finds a response that gives a correct proof, that doesn't mean it'll work every time. Or am I wrong here?

Aside from a temperature of 0, which always results in the same completion, the details and translation examples (aka, in-context learning, few-shot) can force very reliable results, say, 8/10 times, meaning a sample-and-vote gives consistent results when the temperature is non-zero.

Edit: I was not in any way rude nor saying anything incorrect.

If you want to see how to do what I’m talking about, here’s an almost finished article describing the above:

https://github.com/williamcotton/empirical-philosophy/blob/m...

Re: Professor writes history essays with ChatGPT and has students correct them

#68

Post by ncphil »

This is exactly what I'd do if I was still teaching. It's an opportunity to give students a different perspective, while at the same time improving their critical thinking skills. Also, like 80% of what you find on stackoverflow, it could lead to your mastering content that you wouldn't if the their answers could be trusted. It could also help promote an attitude of healthy skepticism towards this kind of tech.

Re: Professor writes history essays with ChatGPT and has students correct them

#69

Post by dicknuckle »

memalign wrote: post #24
8organicbits wrote: post #21

Earlier quoted context omitted.

Wouldn't you want to curate? Maybe I'm missing the point too.

OP might be implying that from now on many engineers will use AI tools to write the first draft of their code so it’s smart to hire people skilled at correcting AI-generated code.

It's also good to hire people who are skilled at correcting bad code in general. This is also a good litmus(?) test to see the interesting ways people come to solutions to other's faults.

Re: Professor writes history essays with ChatGPT and has students correct them

#70

Post by Snowbird3999 »

Unrelated to ChatGPT, fact checking seems like a great exercise for schools to teach modern tech literacy. Imagine doing this with a tweet, a random anecdote, a video clip, etc.

What sources to use? What search engines to use? How do you distinguish or balance information found via a conspiracy theory website vs. a well funded "think tank" vs. something more academic?! I would love to learn this myself.

Post reply on HN