Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Professor denounces mass AI fraud on an exam at Brown

english.elpais.com

Post reply on HN
Print view View original

521–530 of 771 posts

Re: Professor denounces mass AI fraud on an exam at Brown

#521

Post by bryanrasmussen »

rtpg wrote: post #257
QuadmasterXLII wrote: post #96

Earlier quoted context omitted.

I’m 30 and “we can’t do tests in paper” seems _insane_. Just how metastatic has ed tech been in what, 9 years since my undergrad?

I had to do "write code on paper" stuff as part of french engineering school entrance exams. It's fine (tho annoying when you lose points to "typos"), but it limits what kinds of problems you can reasonably put on the exam. You'll definitely lean a bit more into theoretical stuff than practicals. Which is fine for some courses, I think a bit less interesting in other courses. Remember, the hand written code is also h…

>It's fine (tho annoying when you lose points to "typos"),

this would also end up valorizing particular ways of thinking or developing, so those whose development methodology is more exploratory until you get a certain point where you get an overarching view of how everything should interact and then go into a quicker iterative mode would be penalized.

I guess it's ok it does this, some things after all must be valorized whatever is chosen, although I've known many people who are quite good using this way of working who would then be downgraded.

Re: Professor denounces mass AI fraud on an exam at Brown

#522

Post by ValentinPearce »

QuadmasterXLII wrote: post #96
walrus01 wrote: post #80

Earlier quoted context omitted.

> In the AI era, tests are going to have to be in person and hand written. Forcing hand written should really not be necessary. It would be very cheap in terms of old computing hardware to set up a test room with old desktop PCs that have wired only NICs (with a network connection that goes to a switch in the same room with no uplink, connected to a decent size laser printer only), running something like lubuntu and…

I’m 30 and “we can’t do tests in paper” seems _insane_. Just how metastatic has ed tech been in what, 9 years since my undergrad?

I've sat exams a few years after university (for a tentative career change) and I can tell you I'd forgotten how genuinely tiring writing so much by hand was. I've made sure to write more regularly since, just in case I change from software engineering to something that requires more handwriting.

Re: Professor denounces mass AI fraud on an exam at Brown

#523

Post by nneonneo »

recursivedoubts wrote: post #57

In the AI era, tests are going to have to be in person and hand written. I have written an article on how I have adjusted my classes to the situation: https://htmx.org/essays/universities-and-ai/ Ironically, I think the AI era may make university degrees a better signal of the intellectual abilities of students due to the presence of pre-computer infrastructure like large lecture halls, industrial-scale copiers, etc.

Using a special computer works too. I do my exams with our institution's Computer-Based Testing Facility, a bank of computers with fixed software and firewall rules that only permit connections to the exam site.

As a result, I've been able to challenge students to solve interactive software security challenges on the midterm and final with automatic grading - something that would have been impossible with pen and paper.

Scalability is really the major challenge. We're rolling out more CBTF rooms and rolling out access to other departments due to demand, but it's definitely more resource-intensive than pen-and-paper. One possibility is to treat CBTFs as computer labs when not actively administering exams (or maybe even vice-versa), something we're looking into doing.

Re: Professor denounces mass AI fraud on an exam at Brown

#524

Post by laszlojamf »

I had a couple of courses that had open-book exams, where you could bring _any_ material you wanted. Any books, notes, equipment, really anything you wanted. The exam was still hard, because it focused on defining problems. Knowing what question to ask. This is still the hardest problem, even with LLMs. My guess is that most teachers don't like open-book exams, because they are hard to make. But maybe LLMs could help with this too. There is no putting the genie back in the bottle anymore, and restrictions are only going to be effective for dumb cheaters. You have to create exams which make the smart cheaters do actual work.

Re: Professor denounces mass AI fraud on an exam at Brown

#525

Post by nairboon »

rocqua wrote: post #504
sgustard wrote: post #311

Earlier quoted context omitted.

"best not to blame the students" There are dozens for every one who didn't get the opportunity to attend an Ivy. The penalty for cheating should be automatic expulsion. I'll note as a hiring manager the college degree won't be worth shit unless the school can show their students have any integrity.

Sure, blame the people not the system. Our whole education system is setup as a skinner box for grades. People finding tricks for better grades are massively rewarded. Study methods are optimised for recall within a month. And educators are accountable only to the measured outcomes. Very little of the designed system of education is aimed at teaching. Most of it is aimed at measuring and certifying. Treating students…

Yes, unfortunately that is exactly how most education systems are designed. A lot of it is also historical baggage (at least in the European school tradition), where states were faced with the issue of educating the masses, which required a lot of standardization and thus also grades. Although nowadays, educational science has long established the detrimental effects of grades, they are still very widespread. Grades are institutionalized nowadays; you have generations of students who excelled in this Skinner box and became teachers themselves, thus perpetuating the grade box.

Fortunately, there are a few alternatives, schools without grades, that don't focus on short-term recall but long-term understanding, intrinsic motivation, autonomy, and self-actualization, like the Ecole d'Humanité in the Swiss Alps: https://ecole.ch/

Re: Professor denounces mass AI fraud on an exam at Brown

#526

Post by nneonneo »

gchallen wrote: post #62
recursivedoubts wrote: post #57

In the AI era, tests are going to have to be in person and hand written. I have written an article on how I have adjusted my classes to the situation: https://htmx.org/essays/universities-and-ai/ Ironically, I think the AI era may make university degrees a better signal of the intellectual abilities of students due to the presence of pre-computer infrastructure like large lecture halls, industrial-scale copiers, etc.

There are limits to what you can assess on timed assessments, and there are students whose performance on such assessments is not a good signal of their intellectual ability. In addition, at many institutions such tests are given infrequently and can be worth a significant component of a student's overall grade, increasing both student stress levels and the tendency for such assessments to measure short term knowledg…

Speed is an asset, and I think it's an underrated one. Timed assessments are, in part, a speed challenge; students who understand the material more thoroughly can apply it faster and more accurately, giving them more time to complete the exam and to review it.

Yes, students can raise their score by cramming, but won't be as fast or as fluent as a student who has learned and internalized the material over the term.

Re: Professor denounces mass AI fraud on an exam at Brown

#527

Post by sayamqazi »

onion2k wrote: post #435
sgustard wrote: post #311

Earlier quoted context omitted.

"best not to blame the students" There are dozens for every one who didn't get the opportunity to attend an Ivy. The penalty for cheating should be automatic expulsion. I'll note as a hiring manager the college degree won't be worth shit unless the school can show their students have any integrity.

The penalty for cheating should be automatic expulsion. Universities need to stay ahead of how people actually work. Framing the use of AI as 'cheating' gives a university a reputation that it isn't keeping up with industry, which has a wildly negative impact on admissions. You can't expel people for 'working like they do in the real world'.

> 'working like they do in the real world'.

But in real life I am allowed to look at any material available while solving a problem yet school exams dont allow it.

Re: Professor denounces mass AI fraud on an exam at Brown

#528

Post by nneonneo »

jdshaffer wrote: post #279
randunel wrote: post #214

Earlier quoted context omitted.

Nice way to paywall a comment on HN.

Is it paywalled? I honestly didn't realize that. Sorry. I'll edit the comment and change to an open URL and paste the two main ideas here: FORCED SKILL-BUILDING GOALS Where my original purpose of giving homework assignments was to help students build their skills through active engagement, I can no longer trust that all my students will have done it by hand. But I still want my students to feel the need to improve th…

Worth pointing out - modern multimodal LLMs (properly called VLMs, etc.) can easily take pictures as input and describe them in text. In fact, the CLIP model - one of the predecessors of modern VLMs - is entirely designed around being able to caption images with text.

That said - requiring students to hand-write answers is reasonably effective. It's a lot more boring to hand-copy text out of an LLM answer than write it yourself, and it makes the "cheating" significantly more visceral.

Re: Professor denounces mass AI fraud on an exam at Brown

#529

Post by StrauXX »

nneonneo wrote: post #523
recursivedoubts wrote: post #57

In the AI era, tests are going to have to be in person and hand written. I have written an article on how I have adjusted my classes to the situation: https://htmx.org/essays/universities-and-ai/ Ironically, I think the AI era may make university degrees a better signal of the intellectual abilities of students due to the presence of pre-computer infrastructure like large lecture halls, industrial-scale copiers, etc.

Using a special computer works too. I do my exams with our institution's Computer-Based Testing Facility, a bank of computers with fixed software and firewall rules that only permit connections to the exam site. As a result, I've been able to challenge students to solve interactive software security challenges on the midterm and final with automatic grading - something that would have been impossible with pen and pap…

My university does that, it works quite well. The devices net-boot either into a locked down exam OS or regular Debian, depending on the current need.

Re: Professor denounces mass AI fraud on an exam at Brown

#530

Post by ElProlactin »

nneonneo wrote: post #523
recursivedoubts wrote: post #57

In the AI era, tests are going to have to be in person and hand written. I have written an article on how I have adjusted my classes to the situation: https://htmx.org/essays/universities-and-ai/ Ironically, I think the AI era may make university degrees a better signal of the intellectual abilities of students due to the presence of pre-computer infrastructure like large lecture halls, industrial-scale copiers, etc.

Using a special computer works too. I do my exams with our institution's Computer-Based Testing Facility, a bank of computers with fixed software and firewall rules that only permit connections to the exam site. As a result, I've been able to challenge students to solve interactive software security challenges on the midterm and final with automatic grading - something that would have been impossible with pen and pap…

This is just...depressing. BSL-4-like rooms just to test university students.
Post reply on HN