Live data from Hacker News

CheatGPT

blog.humphd.org

141–150 of 544 posts

Re: CheatGPT

#141
post #21

Perhaps this is another of the early (?) nails in the coffin for traditional higher education. If it becomes harder to assess if someone learned something (with a grade), the results of that assessment (GPA) become less valuable. Software has traditionally been at the forefront of allowing people with non-traditional backgrounds (bootcamps, other degrees, self-taught) to work in the highest echelon of jobs, because o…

Higher education will become just an optional prep course for your sit down conversational AI interview. Once AI can do a good job vetting candidates, I see no reason for companies not to have an open applicant process where anyone can interview and be evaluated. If you are sharp and know your shit, a degree won't matter and the AI interviewer won't care. But this is an "All else being equal" scenario, my true belief…

The event horizon you describe is always there. Be it 3D printing, AI, moore's law... etc... The things these things enable, are hard to predict.

Think about cloud computing. It changes the game massively for startups and for people who need enterprise class infrastructure as mere mortals.

Another constant tension to show you how unpredictable all this is: Do you use kernel networking, let the kernel use hardware offloads, or goto use DPDK? The choice of what to do is changing as hardware changes, the kernel changes etc....

... Once you understand, that life is ALWAYS at an event horizon.. you understand AI is just another such event.

Prediciting the future is for the native... Making the future is the way to go. Currrently the AI guys are doing that. But another thing will rise up, it always does.

Re: CheatGPT

#142

So most interview questions like "make a binary tree" are dead. The best interview question that will never die: "What's the weirdest bug you debugged? What made it weird?" For posterity: https://www.gamedeveloper.com/programming/my-hardest-bug-eve...

Yes, agreed! We ask a variant of this question (we call it "an analysis sample"); from the materials we ask candidates to submit[0]:

"A significant challenge of engineering is dealing with a system when it doesn't, in fact, work correctly. When systems misbehave, engineers must flip their disposition: instead of a creator of their own heaven and earth, they must become a scientist, attempting to reason about a foreign world. Please provide an analysis sample: a written analysis of system misbehavior from some point in your career. If such an analysis is not readily available (as it might not be if one’s work has been strictly proprietary), please recount an incident in which you analyzed system misbehavior, including as much technical detail as you can recall."

These samples are very revealing -- and it feels unlikely that generative AI is going to be of much help, even assuming a fabulist candidate. (And of very little assistance on our values-based questions like "when have you been happiest in your professional career and why?").

[0] https://docs.google.com/document/d/1Xtofg-fMQfZoq8Y3oSAKjEgD...

Re: CheatGPT

#143
The professor could just ask the student: how did you come up with this solution? What was your thinking?

To prepare for this question, the student has to think really hard both about the problem and about the solution. While doing so, we learn a lot - maybe more than we would learn otherwise.

Re: CheatGPT

#144
One possible temporary solution is to use ChatGPT to generate the solution to each question, and only include questions where its answer is incorrect.

Then at worst the challenge for the student is debugging ChatGPT code, which still has merit.

Re: CheatGPT

#145

If I were a teacher, I'd give written exams for coding, as we used to have back in the 80's when we didn't have enough dedicated computers for everyone in the class, and most of kids didn't have a home computer to work with. So instead, we'd implement simple algorithms on paper for in-class quiz/tests/homeworks. I remember in high school, we had an exam, and one of the question was to write a for loop counting from 1…

I graduated in 2015, and I had several classes where exams involved writing pseudocode by hand—particularly the algorithms-focused classes. I think universities will continue to do this.

Re: CheatGPT

#146
>It's been breaking my understanding of what is and isn't possible, and forcing me to think about new ways to solve technical problems.

followed up by

>However, I'm not sure how to understand it as part of student learning.

Is super funny

Re: CheatGPT

#147
post #117

I see a ton of people trying to justify their use of Copilot/ChatGPT (or rushing a startup to cash in on LLMs). Maybe that conflict of interest is why there's very little talk of it being based on plagiarizing and license violation of open source code on which the model was trained. We just suffered through a couple decades of almost every company in our field selling out users' privacy, one way or another. And years…

Right, any humans that reads open source code ever should also be forced to submit any code they write to ensure they've not mentally performed copyright violations.

I don't give two shits if whatever current expensive GPT is dumping out code 'very similar' to open source code today. And you'd be chopping off your own nose if you did too. Thinking these models will remain as expensive to run in the future means that at the time you or I could run 'LibreGPT' on our own hardware, we'd be scared as hell to even write the code because any use of it could get you sued into oblivion.

Burn copyright to the ground.

Re: CheatGPT

#148

So most interview questions like "make a binary tree" are dead. The best interview question that will never die: "What's the weirdest bug you debugged? What made it weird?" For posterity: https://www.gamedeveloper.com/programming/my-hardest-bug-eve...

That's interesting! What did you learn from debugging that weird bug?

Re: CheatGPT

#149
post #126

Maybe take the approach many physics classes embrace (in germany) : All tools are allowed in tests: calculators, books, as long as the correct answer is derived with "correct derivation path". If chatgpt makes some problems too easy maybe it's a good thing because we can raise the bar. Find problems that require true understanding beyond auto complete / copy pasting.

This isn’t too different from requiring students to write essays. No one actually needs another essay about Shakespeare or whatever, and ChatGPT could do it for you, but the point of the exercise is to learn to write. It’s the process, not the product that matters.

Everyone’s ChatGPT gangsta until their ChatGPT-trained doctors start giving fatally wrong diagnosis. 100% agree with whole comment. I interview cheaters time to time, their excuse of not learning stuff is exactly that. However, in the end they’ve cheated themselves more than the system.

Re: CheatGPT

#150
post #53

The failure mode of things like ChatGPT is it can make wrong answers confidently, subtly, and if you don't have the skill to audit what is wrong with the answer, then it can be fairly catastrophic. So instead of making questions generative, you make them audits / debugging type ones. Use ChatGPT to generate a result after several iterations that is wrong and then ask them what is wrong with the result. Since ChatGPT…

the other week i gave chatgpt a simple multiplication problem that it got wrong. very simple problem like 86 * 0.0007 or something. but ive been working with chatgpt for 4-5 weeks now and that wrong answer doesnt make up for all the "good answers" that are usually not perfect. like one day i needed to COALESCE in mysql. i didnt know that, but chatgpt did. theres a few times i would have written a function the complic…

Why would you use chatGPT as a calculator? Use a calculator for that.
Post reply on HN