Live data from Hacker News

CheatGPT

blog.humphd.org

271–280 of 544 posts

Re: CheatGPT

#271
Short term, this is a problem.

Long term, the bar will simply be raised. Smart/motivated kids armed with AI will still outperform the ones who don’t study, and will be graded accordingly.

This will only be a problem so long as AI help is considered cheating, and until teachers are able to recalibrate their standards.

Re: CheatGPT

#272

So most interview questions like "make a binary tree" are dead. The best interview question that will never die: "What's the weirdest bug you debugged? What made it weird?" For posterity: https://www.gamedeveloper.com/programming/my-hardest-bug-eve...

One of the smartest I ever felt was hearing some teammates a cube wall over saying "this one customer from poland says he's crashing a lot" (in our ui). I was like "is he in the polish locale/language? They asked.... yes... "I bet polish has different seperators and our split("/")[1] is crashing"... they checked, had him change to english, crash went away. I felt real yoda like. We had customer on chat so much easier…

Debugging is easy when you know you are coding incorrectly.

But why would Polish have a different path separator? That's a filesystem choice not a language choice.

Re: CheatGPT

#273

I don't understand how most of the comments here seem to be along the lines of "these interview questions are useless now" or "we need to rethink education, it hasn't kept up". These all seem absurdly myopic to me. What we're seeing is the first instance, still very limited and imperfect, of AGI. This is not going to make some interview questions obsolete, or give students more tools to cheat with their homework. It…

This all seems very unlikely to me. Better tools have always made humans more useful, just doing different things. I do think it's interesting to consider whether there is some singularity where that trend dramatically reverses (Vonnegut's Player Piano is one of my favorite books...), but I think the better prior is that this is just another step up the abstraction ladder for humanity.

Re: CheatGPT

#274

I don't understand how most of the comments here seem to be along the lines of "these interview questions are useless now" or "we need to rethink education, it hasn't kept up". These all seem absurdly myopic to me. What we're seeing is the first instance, still very limited and imperfect, of AGI. This is not going to make some interview questions obsolete, or give students more tools to cheat with their homework. It…

Software development isn't really about just acquiring and memorising knowledge to later spit out. You need reasoning, logic and creativity. That AGI can develop complex new software (not just some code that has been scraped of the internet years ago) in 10 years is something I doubt.

Re: CheatGPT

#275
post #21

Perhaps this is another of the early (?) nails in the coffin for traditional higher education. If it becomes harder to assess if someone learned something (with a grade), the results of that assessment (GPA) become less valuable. Software has traditionally been at the forefront of allowing people with non-traditional backgrounds (bootcamps, other degrees, self-taught) to work in the highest echelon of jobs, because o…

Higher education will become just an optional prep course for your sit down conversational AI interview. Once AI can do a good job vetting candidates, I see no reason for companies not to have an open applicant process where anyone can interview and be evaluated. If you are sharp and know your shit, a degree won't matter and the AI interviewer won't care. But this is an "All else being equal" scenario, my true belief…

Why would you even need jobs in this case? Also why does everyone think we live in some magical fairy in which the majority of people with advanced knowledge of a subject are doing so without higher education. Do these people really think everyone is an autodidact? If that was the case, why would they need to cheat on their courses?

Re: CheatGPT

#276
post #191

So most interview questions like "make a binary tree" are dead. The best interview question that will never die: "What's the weirdest bug you debugged? What made it weird?" For posterity: https://www.gamedeveloper.com/programming/my-hardest-bug-eve...

is a hard bug the best interview question? When I do my job right 98% of the time is not debugging. This ratio changed dramatically over the course of my career.

I find the ratio doesn't change, but the trickiness of the bugs gets more complicated as you solve harder problems and avoid previously unknown problems.

Re: CheatGPT

#277

Earlier quoted context omitted.

> The best interview question that will never die: "What's the weirdest bug you debugged? What made it weird?" @ChatGPT: Give me three sample answers to the following questions related to programming: "What`s the weirdest bug you debugged? What made it weird?"

"One time, I was working on a project where the UI kept freezing when a certain button was pressed. I spent hours debugging the code, but couldn't figure out what was causing the problem. Eventually, I discovered that the button's event handler was accidentally triggering an infinite loop that was consuming all the CPU resources, causing the UI to freeze. It was a weird bug because the symptoms were not immediately o…

I feel like I could have written that. Did ChatGPT copy old blog posts of mine? ;-)

Re: CheatGPT

#278
post #251

Earlier quoted context omitted.

I would 100% accept that as an answer since it tells me the candidate is not good with subtext, and that's really needed to solve a lot of interesting problems. After all, if I ask "what's your favorite food you've ever eaten", there's an unspoken implication that it's a food you remember eating. I am not in fact asking you to recall every single food you've ever eaten and choose one... - From my comment below since…

> I would 100% accept that as an answer since it tells me the candidate is not good with subtext, [...] if I ask "what's your favorite food you've ever eaten", there's an unspoken implication that it's a food you remember eating. Will this filter cut many of the best engineers? Our field is full of people who pull 'engineering' out of our behinds, to various degrees. I'd assert that the engineer who doesn't assume an…

> Will this filter cut many of the best engineers?

It won't cut out a single good engineer, let alone the best.

> I'd assert that the engineer who doesn't assume an unspoken implication, but instead qualifies their answer, or tells you when they cannot answer, or asks for clarification... is more likely to be the one who can make a system that works, and tell you when a system will not work.

You grouped the one option that a bad engineer would take, with several that a good engineer would take. "Tells you when they cannot answer" is not what a good engineer does.

They may say "I cannot answer question as-is" as a jump off for clarification.

In fact in my response to your sibling comment I explain that even if there were no interesting bugs you can give an answer that isn't lying, or pulling engineering out of your ass, or dishonest.

-

But flat out refusing or immediately jumping to "well but I can't remember everything!!!!" is still you interpreting subtext... except you've now interpreted the most negative possible subtext. You've assumed your interviewer is asking you to recall things you can't recall and that there is no further room for discussion.

A poor engineer is one that shuts completely down at the first hint of a broken invariant, rather than trying to surface that there is an invalid invariant, or learn more about the broken invariant.

That kind of curiosity to go further than shutting down is what the question is meant to tease out, so you're not beating the system by deciding not to engage, instead you're sending the exact signal being looked for as something to avoid bringing into your organization.

Re: CheatGPT

#279

I don't understand how most of the comments here seem to be along the lines of "these interview questions are useless now" or "we need to rethink education, it hasn't kept up". These all seem absurdly myopic to me. What we're seeing is the first instance, still very limited and imperfect, of AGI. This is not going to make some interview questions obsolete, or give students more tools to cheat with their homework. It…

I think what an LLM is best at is fooling people into thinking it’s intelligent. It is really good at saying things in a natural sounding way, and statistically often getting it right, because certain strings of tokens are encoded. But it’s clear when you start poking that it just as easily tells you that 5/2=3, or that 2+2 != 4. it doesn’t model math or any sort of knowledge at all.

Re: CheatGPT

#280

So most interview questions like "make a binary tree" are dead. The best interview question that will never die: "What's the weirdest bug you debugged? What made it weird?" For posterity: https://www.gamedeveloper.com/programming/my-hardest-bug-eve...

"it involved a program that ran perfectly fine during testing but crashed whenever it was deployed to the production environment. After much investigation, it was found that the bug was caused by a combination of the production environment's security settings and the program's use of random number generation. The program used a random number generator to generate a unique ID for each user that logged in. However, the…

Back in my first job we had a database which worked fine in testing, worked fine on the customer’s server… then started crashing a week later.

Turns out their MS SQL Server install was configured for MM-DD-YYYY output which only crashed when we reached the 13th of the month. Important lessons were learned that day!

Post reply on HN