reduced Turing test: Writing homework at a good enough quality that an overworked teacher grades it and doesn't notice it's AI
AI Homework
181–190 of 296 posts
Re: AI Homework
#182Earlier quoted context omitted.
Given that most people never check the source of what they read, this is really scary. Because now everyone has the ability to write and say things that sound plausible and likely to be convincing, and the truth will be harder to access.
I think everyone has been able to tell convincing lies for quite some time before language models even existed.
but now its almost zero effort activity.
Re: AI Homework
#183One of my subjects at highchool was technical drawing. The teacher was a quite old man who was set on his ways. We would have to draw 3D pieces by hand, using our squares, bevels and compasses, and lining everything with an ink pencil afterwards, and he gave us a lot of homework to do, which he examined for imperfections. I was incredibly bad a it. I would start a drawing, only to discover halfway through it that it…
How easy do you think it is to spot when AI is being used?
In general, we don't know how to do this in a way that is robust to all the trivial ways to fool these techniques.
I can just fine-tune a large scale model on a small downstream task, or use creative choices of decoding settings (high temperature, alternative decoders like contrastive/typicality sampling), to fool the existing methods.
Re: AI Homework
#184Earlier quoted context omitted.
One short sentence isn't enough. If you play with it a bit, there are definitely patterns. Here's one I did over the weekend: 99.98% fake. The Battle of Agincourt was a significant event in English and French history, as it was a major victory for the English army against the French during the Hundred Years' War. The battle took place on October 25, 1415, near the town of Agincourt in northern France. The English arm…
Are there any articles explaining how this works? What exactly is the classifier keying in on that indicates LLM text so reliably?
This report gets into it a bit.
Re: AI Homework
#185>In the case of AI, don’t ban it for students — or anyone else for that matter; leverage it to create an educational model that starts with the assumption that content is free and the real skill is editing it into something true or beautiful; only then will it be valuable and reliable.
This was entirely the premise of Rainbow's End by Vernor Vinge, himself an SDSU professor at the time.
Re: AI Homework
#186Earlier quoted context omitted.
How easy do you think it is to spot when AI is being used?
I asked ChatGPT how to write texts so it can be identified as human generated. The answer: "One way to make it clear that a text is not generated by AI is to include a joke or riddle that is difficult for a machine to understand or generate. For example, you could include a joke that relies on wordplay or a complex pun. Here is an example: Why was the math book sad? Because it had too many problems."
Obviously we should not anthropomorphize a model, but this is just downright funny because it's so obviously wrong that it appears as almost malicious.
Good thing ChatGPT doesn't know any complex puns or witty wordplay!
Re: AI Homework
#187Nothing quite beats a low key oral exam. The trick is that they don't scale.
These days I give take home video exams. The students record themselves explaining their code for 5 minutes, which is about as long as it takes me to hand-grade a written exam. I can even play the video at 2x to cut the time in half. This scales at least to the size of my class (100 students on average, 3-4 hours isn't bad for grading an assignment).
Oral exams in person have the advantage that the students can not read out an AI-generated solution (or a solution purchased from somebody else). They have to come up with their own solution on the spot. For video recordings, they can cheat as much as they like.
Re: AI Homework
#188Earlier quoted context omitted.
While I can (unfortunately) see this happening - it seems incredibly futile and pointless to try and fight such an arms race. When AI-supported writing becomes the norm having students write essays without AI-assistance will be like trying to force students to do complex arithmetic without calculators. Universities and schools need to accept that writing essays without AI support will simply not be a useful skill any…
>Universities and schools need to accept that writing essays without AI support will simply not be a useful skill anymore It’s kind of mind-blowing that anyone could think this. You’re posting on a website for essays. Small essays, generally speaking, if we restrict ourselves to the comments section, but essays nonetheless. You yourself just wrote one, because you wanted to express your thoughts on an issue and you w…
Feel free to express your thoughts. But I browse for content. And if AI has better content than you, then I prefer AI.
Note that if we're talking about triangulating human sentiment, it becomes a different story.
Re: AI Homework
#189Earlier quoted context omitted.
I disagree. Senior professionals might have a better understanding because they have spent a lot of time drawing, but the specific tool is not relevant. It is like saying that playing tennis with a wooden racket gives you a better "feel" for the ball and therefore is preparatory to using modern rackets. No one does this because it is not true. A similar line of thinking was popular in jiu jitsu, where veterans said t…
That is definitely true, that knowing and understanding a tool is not important to actually succeeding with the tool. But it does help a confused person understand why tools are made the way they are. In Fact, Modern programs in CAD do not follow the tools and principles that AutoCAD emphasizes. And those programs have charted a different course where the history and legacy of technical drawings has less relevance. (…
I would say that there is an overemphasis, although understandable, on the usefulness of what happened or was used before the current state, position or technology.
For example, with regard to technical drawing, we might say that using the pencil, the square, and the drafting machine gives an understanding of drawing that the use of the computer cannot. But, for some reason, we don't say that using your finger and sand, or a rock and string to draw a circle helps even more. And we don't say that simply because the people who used rock and string (the good old days) are long dead.
The same goes for clothing, for which we say the 1990s, or perhaps the late 1960s, were better, but no one says that between 1900 and 1910 people dressed with such great taste.
The same goes for romantic partners ("if he hadn't dumped you and made you cry for a year and a half, you wouldn't have found your wonderful wife/husband"), and many other things.
It is very human to rationalize the circumstantial.
Re: AI Homework
#190Something I've noticed about chat ais vs direct search is that because a chat ai is a blackbox, I can't dig into an answer's source at all. With a search, I can read the wikipedia sources or I know the forum, or the poster. But with an AI, it's a dead end and it sucks. I can ask people how they know something, but AI training data is invisible. It doesn't know how it knows something. There is this oracle with a 10% c…
> because a chat ai is a blackbox, I can't dig into an answer's source at all.
It's not quite the same thing, but I've been impressed by the results when you ask ChatGPT to back up its answers or provide concrete examples for things it claims or to explain a point. While it doesn't manage all the time, it has surprised me multiple times with how good it is at this.