Live data from Hacker News

Suspecting AI cheating, Ivy League prof ordered in-person final; scores fell 50%

arstechnica.com

61–70 of 168 posts

Re: Suspecting AI cheating, Ivy League prof ordered in-person final; scores fell 50%

#61

The article, the teacher, and the general academic community skips the hard question when it comes to AI and that's whether these exams are testing knowledge that is still worth internalizing in the same way? Academia has a long history of lagging behind acceptance of new cognitive tools where they claim to want to defend the students, but instead defend the assignments of the past at the expense of the students. Cal…

I have a few thoughts related to this, and maybe I can get them out and ties them back together at the end.

1a. Yes, college isn’t right for everyone, and testing the traditional way with paper and pencil certainly disadvantages some students who would be star performers in a real world setting but are not a good fit for college classes. Think “Good Will Hunting” type people. They do exist.

1b. However, there are certainly more people who only imagine themselves as Will Hunting type people and that they are just too smart for college, but the reality is that they are dumb, or didn’t learn the material. For every person who fails a test because they’re a genius who is a bad fit in the system, there are at least 10 idiots who imagine themselves geniuses, and they would have passed the class if only that PhD professor with all his book smarts had actually written the right kind of exam. Traditional schooling and testing doesn’t work for the extreme upper tail of intelligence, but it exists because it did quite well at educating and sorting the masses to support the Industrial Revolution.

1c. If college were only about the knowledge, you could learn most of it with internet access and a library card for much cheaper. The vast majority of people are in college for the credential, and the institution has to protect the signal of the credential.

2a. Most college assignments and exams are not a good reflection of full time employment. College credentials serve mainly as a networking aid and a signal to employers that you are compliant and competent enough to follow a professor’s instructions, and will likely be a compliant and competent employee, although the instructions might be different. If it is the AI who is the one who followed the professor’s instructions you water the signal down, and employers don’t want that. It is irrelevant that a student can copy and paste things into ChatGPT, and that ChatGPT can get the answer right on this test. That isn’t what college is supposed to signal.

2b. A problem is that I literally can’t write a test in most subjects now that I would expect a student to complete that can’t be completed by ChatGPT better and faster. I teach undergraduate math and a while ago we thought that since GPT-4o could get a C in calculus that we’d just raise the standard. Now Fable and GPT-5.5 can cruise to an A in literally every math course in our catalog, and they can also catch every tiny issue in an exam written by a human. But I have to teach these undergraduate subjects so that some students can go on to PhD studies so they can contribute to the field. If we just stop teaching undergraduate subjects then PhD production and novel research grinds to a halt and only a few fields will progress where an AI is capable of self improvement.

2c. I’ve seen that my best students know how to do do something by hand and use a computer to complement/increase their capabilities, not to cover over entire gaps. When you have literally zero skill in an area, you can’t spot when you got a totally bad output from the AI (these days usually because of a lack of context or bad prompting because the student didn’t understand the material, not because the AI wasn’t capable). Somehow I have to incentivize students to learn the material on their own, so that they can be a better user of AI in the future. And a really effective way to do that is a graded test in which AI is not available to help them.

So to try to tie it together, is that there is still some value to a college degree, at least until something better comes along. But that college degree is only useful when it is a signal about the person and not some other tool. And although AI is getting very capable, somehow we have to teach the lower stuff to build up to the higher stuff, so we do need to restrict the use of AI in some educational settings so that we can build a good foundation for future learning.

As to the point about old paper sources being considered more reliable than an internet site, I agree. I am of the generation that wasn’t allowed to cite Wikipedia and it frustrated me. We’ll eventually figure out how to permit proper AI use, much like how many professors now allow you to use Wikipedia to start researching a topic.

Re: Suspecting AI cheating, Ivy League prof ordered in-person final; scores fell 50%

#62

Earlier quoted context omitted.

College is a box to check off because we have lost the ability to support a workforce without college degrees (at least in the US, UK, AU, where I have some experience). Trades are critical but looked down on. Manufacturing is gone (which isn't in itself a bad thing). The service industry doesn't pay a living wage (thankfully it's reasonable here in AU). Apprenticeships don't pay enough. And pretty much all knowledge…

Sounds like you're in AU, I will point out that trades are not looked down on at all in this country. In fact culturally we value chippies and sparkies higher than desk jockeys.

Yeah this was a generalisation across the US/UK/AU. I come from the UK but now live in AU, and know the US a bit.

I think the looking down on trades is not quite as simple as I summarised it as, and I think AU does a lot better at all of the above than the US/UK, but I still see aspects of it. Tradies make good money in all 3 countries due to lack of supply, and yet there are still stereotypes of jobs like lawyers, doctors, (software engineers?) being better in some way.

It's a nuanced problem, but I don't get the impression that trades here are culturally valued higher than a lot of "white collar" work. Compared to ambiguous "desk jockeys" yes, but that's due to negative stereotypes about bullshit jobs, if you actually named a specific job I think you'd find different attitudes. Lawyers, accountants, sales/marketing, various engineering disciplines, IT, I think these are widely considered "better" jobs than trades, even though in most ways that's far from true.

Re: Suspecting AI cheating, Ivy League prof ordered in-person final; scores fell 50%

#63

Earlier quoted context omitted.

College is a box to check off because we have lost the ability to support a workforce without college degrees (at least in the US, UK, AU, where I have some experience). Trades are critical but looked down on. Manufacturing is gone (which isn't in itself a bad thing). The service industry doesn't pay a living wage (thankfully it's reasonable here in AU). Apprenticeships don't pay enough. And pretty much all knowledge…

Sounds like you're in AU, I will point out that trades are not looked down on at all in this country. In fact culturally we value chippies and sparkies higher than desk jockeys.

As a non-Australian I just have to know what “chippies” and “sparkies” are!

Re: Suspecting AI cheating, Ivy League prof ordered in-person final; scores fell 50%

#64
post #3

> Ivy League college students are, by definition, intelligent. I stopped reading after the first sentence.

Why jump on the opportunity to prune reading by rejecting the lot as soon an unrelated premise you disagree with is presented? Perhaps at that point if you stop reading after the first sentence, you could churn the entire article through AI to summarize it into a single sentence, and see if the invalid premise is core to the message?

> Why jump on the opportunity to prune reading by rejecting the lot as soon an unrelated premise you disagree with is presented?

Ars Technica has already trashed its reputation with the infamous controversy over publishing AI hallucinated quotations.

I couldn't decide whether the opening sentence of the submitted article was so dumb that it had to be written by AI or so dumb that it had to be written by a human, and coming to a conclusion on that question didn't seem worth it.

> you could churn the entire article through AI to summarize it

I don't do AI summaries, ever.

Re: Suspecting AI cheating, Ivy League prof ordered in-person final; scores fell 50%

#65
post #63

Earlier quoted context omitted.

Sounds like you're in AU, I will point out that trades are not looked down on at all in this country. In fact culturally we value chippies and sparkies higher than desk jockeys.

As a non-Australian I just have to know what “chippies” and “sparkies” are!

Chippy = carpenter (from wood chips)

Sparky = electrician

Re: Suspecting AI cheating, Ivy League prof ordered in-person final; scores fell 50%

#67

The article, the teacher, and the general academic community skips the hard question when it comes to AI and that's whether these exams are testing knowledge that is still worth internalizing in the same way? Academia has a long history of lagging behind acceptance of new cognitive tools where they claim to want to defend the students, but instead defend the assignments of the past at the expense of the students. Cal…

I have a few thoughts related to this, and maybe I can get them out and ties them back together at the end. 1a. Yes, college isn’t right for everyone, and testing the traditional way with paper and pencil certainly disadvantages some students who would be star performers in a real world setting but are not a good fit for college classes. Think “Good Will Hunting” type people. They do exist. 1b. However, there are cer…

It's worth noting that even though you say people could just go online and start reading, college may be the last time most people ever read a textbook ever again. There's something tough about knowing that you want to do something but then somehow your daily instincts fight you.

It's also why you can't just throw kids in front of Khan Academy.

Re: Suspecting AI cheating, Ivy League prof ordered in-person final; scores fell 50%

#68

Earlier quoted context omitted.

While I think there are a lot of problems created by capitalism right now, I would argue that nothing about the current AI industry is actually capitalist at all. These are companies that are surviving despite absolutely bleeding cash because the ultra-rich have decided that things would be so much more convenient for them if they didn't need to deal with employees anymore. This isn't "well, people are doing this bec…

Honestly, and this is my "hot take of the decade" but I think we're going to get some sort of AI modulated/managed form of capitalism that will approach socialism in some regards but still allow for people to try to "get rich" or whatever. What that's going to look like I couldn't begin to imagine, but I think you might onto something about the AI industry not being all that capitalistic presently. Not an economist,…

Fascism is the more likely outcome.

Re: Suspecting AI cheating, Ivy League prof ordered in-person final; scores fell 50%

#69

Earlier quoted context omitted.

The thing you're angry at, the thing that you're upset about? It's capitalism, it's the coupling of education with jobs, it's credentialism being overturned by new technology, etc. I think education is incredibly important, but I understand that I'm going to have to retrain myself a little bit. A college degree can no-longer be assumed to be a proxy for having put in the effort to deeply study something. Now what's t…

While I think there are a lot of problems created by capitalism right now, I would argue that nothing about the current AI industry is actually capitalist at all. These are companies that are surviving despite absolutely bleeding cash because the ultra-rich have decided that things would be so much more convenient for them if they didn't need to deal with employees anymore. This isn't "well, people are doing this bec…

Uh, they are doing that because they think one of them is going to build the monopoly that will rule the rest of the future approximately forever.

Re: Suspecting AI cheating, Ivy League prof ordered in-person final; scores fell 50%

#70

Earlier quoted context omitted.

What an odd world view. Of course the priviliged have greater social obligations, because they are greater benefactors of life. We're all equal under the law but that doesn't mean that those with greater means shouldn't have greater impacts on society. Meritocracy is often a buzzword of the elite to avoid paying their fair share.

[flagged]

citation needed on that last claim. do you have universal healthcare free at point of service? do you have a healthy supply of public housing? do you let children starve, through no fault of their own but the circumstances of their birth? do you engage is ears of choice and murder 100+ innocent children in a primary school on the first day of the conflict?

the answer to some of these questions should be yes, and others should be no, if you're going to make the claim that the United States of America is the "Greatest Country on Earth, Ever".

Post reply on HN