Live data from Hacker News

Professor writes history essays with ChatGPT and has students correct them

old.reddit.com

71–80 of 115 posts

Re: Professor writes history essays with ChatGPT and has students correct them

#71
I think this is great idea.

Actually, anytime somebody finds a way to engage students to think in a different way than just trying to memorise facts is a win, IMO. I think it is lamentable that students are asked to rely more and more on recalling facts at the time where it is becoming less and less important compared to critical thinking. This is lazy way to do something about structural education system issues. I can't even call it "solving", because it is not really solving any problems, just ensuring purely educated students.

One of my programming interview question is a flawed piece of code implementing an algorithm. The algorithm is there in its entirety, so there is no need to actually recall the details of it. The task is to point out and fix functional flaws, then critique and fix non-functional problems.

I find this gives me much better appreciation for the candidate compared to a standard hit or miss test where the candidate is asked to implement an algorithm and they may or may not have the luck to know or come up with an answer.

It is also practically impossible to prepare for or cheat even if you somehow get to know from recruiters what you can expect (they do ask people and they do prepare candidates...)

A long time ago I had people cheat by using materials on remote interviews (which I did way before Covid). Now I invite them to use external materials or even ask questions ("This task does not require anything outside the standard library. I don't care if you remember it, if you can describe what the function does I will gladly tell you the name and describe how it works exactly.")

Re: Professor writes history essays with ChatGPT and has students correct them

#72

Earlier quoted context omitted.

Wouldn't you want to curate? Maybe I'm missing the point too.

OP might be implying that from now on many engineers will use AI tools to write the first draft of their code so it’s smart to hire people skilled at correcting AI-generated code.

"many engineers will use AI tools to write the first draft of their code"

I fail to see how ChatGPT would be useful for capable engineers. Is boilerplate code actually a time suck for anyone?

Re: Professor writes history essays with ChatGPT and has students correct them

#73

Earlier quoted context omitted.

> Finally, students are asked to figure out prompts that will make ChatGPT give a correct algorithm and proof. I haven't managed this myself! I'm looking forwards to seeing what students manage. Isn't there a probabilistic nature to ChatGPT replies? So even if a student finds a response that gives a correct proof, that doesn't mean it'll work every time. Or am I wrong here?

Aside from a temperature of 0, which always results in the same completion, the details and translation examples (aka, in-context learning, few-shot) can force very reliable results, say, 8/10 times, meaning a sample-and-vote gives consistent results when the temperature is non-zero. Edit: I was not in any way rude nor saying anything incorrect. If you want to see how to do what I’m talking about, here’s an almost fi…

Upvoted you, don't mind the false flagging

Re: Professor writes history essays with ChatGPT and has students correct them

#74

I'm trying something similar with an introductory Algorithms class. After we go through Breadth First Search, there's a practical assignment where students are asked to modify the algorithm to return _all_ shortest paths. Then I ask ChatGPT for its solution, and students try to spot its mistakes. Later, after going through the proof of correctness of Dijkstra's algorithm, I ask ChatGPT for a proof of correctness of i…

[deleted]

Re: Professor writes history essays with ChatGPT and has students correct them

#75

Unrelated to ChatGPT, fact checking seems like a great exercise for schools to teach modern tech literacy. Imagine doing this with a tweet, a random anecdote, a video clip, etc. What sources to use? What search engines to use? How do you distinguish or balance information found via a conspiracy theory website vs. a well funded "think tank" vs. something more academic?! I would love to learn this myself.

My history teacher in high school had us watch the dark side of the moon with the credits cut off https://en.m.wikipedia.org/wiki/Dark_Side_of_the_Moon_(2002_... Definitely a mind-bending experience at the time.

Re: Professor writes history essays with ChatGPT and has students correct them

#76

I think this is great idea. Actually, anytime somebody finds a way to engage students to think in a different way than just trying to memorise facts is a win, IMO. I think it is lamentable that students are asked to rely more and more on recalling facts at the time where it is becoming less and less important compared to critical thinking. This is lazy way to do something about structural education system issues. I c…

Perhaps this style of interview will be a great antidote to the elitecode arms war hellscape we accidentally created.

Re: Professor writes history essays with ChatGPT and has students correct them

#77

Earlier quoted context omitted.

> Finally, students are asked to figure out prompts that will make ChatGPT give a correct algorithm and proof. I haven't managed this myself! I'm looking forwards to seeing what students manage. Isn't there a probabilistic nature to ChatGPT replies? So even if a student finds a response that gives a correct proof, that doesn't mean it'll work every time. Or am I wrong here?

You're right, ChatGPT is probabilistic. None of this is graded by the way -- it's all just for fun and bragging rights. I've asked students to share their full dialog, both prompts and replies, so the whole class gets to see; and I'll invite one or two to talk through their attempts. This is all just a trick to make students engage with "how do you you spot bugs in a proof?", hopefully more than they would from just…

You can set a ceiling on temperature or just simply dictate a low setting to insure repeatable performances by the LLM. This could be on your reading list as well: https://sites.google.com/view/automatic-prompt-engineer

Re: Professor writes history essays with ChatGPT and has students correct them

#78
post #37

I like this angle. There was a great article on ACOUP about the weaknesses when using ChatGPT for writing essays (including many out and out errors). This is definitely a novel approach to using ChatGPT's weaknesses as a tool to aid learning.

I also remember to have seen this idea of testing students mentioned some time ago here in a HN comment - when ChatGPT was still young. Nice thing.

Re: Professor writes history essays with ChatGPT and has students correct them

#79

Earlier quoted context omitted.

I tried... I pointed out a problem and asked ChatGPT to fix it, unsuccessfully. I asked it for a proof of correctness, then pointed out a problem in its proof and asked ChatGPT to fix it, again unsuccessfully. (It's all in the notes I linked to.) Perhaps I'm just crummy at prompt engineering; or perhaps this is one of those questions where the only way to engineer a successful prompt is to know the answer yourself be…

I've also had this issue multiple times where ChatGPT provides a flawed answer, is able to identify the flaw when asked but "corrects" it in such a way that the original answer is not changed. I've tried this for code it wrote, for comments on my code and for summaries of texts that I provided.

Reminds me of children trying to speak a word properly, but repeatedly making a mistake in the same way.

Re: Professor writes history essays with ChatGPT and has students correct them

#80
post #76

I think this is great idea. Actually, anytime somebody finds a way to engage students to think in a different way than just trying to memorise facts is a win, IMO. I think it is lamentable that students are asked to rely more and more on recalling facts at the time where it is becoming less and less important compared to critical thinking. This is lazy way to do something about structural education system issues. I c…

Perhaps this style of interview will be a great antidote to the elitecode arms war hellscape we accidentally created.

My initial motivation was just low amount of information I was getting about the candidate. The point of interview is to get to know each other. To figure out if I would like to work with the person.

But the interview is very small amount of time to figure this out and there is very little information when all you are getting is the candidate staring at the screen until they come up with an answer. On the other hand I can't just have a chit chat with the person -- there are people who are good at talking but can't do shit. I resolved I will not hire a person that can't actually demonstrate they can do things but it is upon me to make this work.

So I pretty much dropped questions that only probe knowledge and decided to focus exclusively on questions/tasks that require more mutual engagement. Like development tasks that require the candidate to work with me to produce a solution. Or open ended technical questions that prompt discussion, etc.

Post reply on HN