Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CheatGPT

blog.humphd.org

Post reply on HN
Print view View original

251–260 of 544 posts

Re: CheatGPT

#251

Post by neilv »

BoorishBears wrote: post #225
epicureanideal wrote: post #162

Earlier quoted context omitted.

> "What's the weirdest bug you debugged? What made it weird?" If you can remember, 5-10 years after you solved it.

I would 100% accept that as an answer since it tells me the candidate is not good with subtext, and that's really needed to solve a lot of interesting problems. After all, if I ask "what's your favorite food you've ever eaten", there's an unspoken implication that it's a food you remember eating. I am not in fact asking you to recall every single food you've ever eaten and choose one... - From my comment below since…

> I would 100% accept that as an answer since it tells me the candidate is not good with subtext, [...] if I ask "what's your favorite food you've ever eaten", there's an unspoken implication that it's a food you remember eating.

Will this filter cut many of the best engineers?

Our field is full of people who pull 'engineering' out of our behinds, to various degrees. I'd assert that the engineer who doesn't assume an unspoken implication, but instead qualifies their answer, or tells you when they cannot answer, or asks for clarification... is more likely to be the one who can make a system that works, and tell you when a system will not work.

Re: CheatGPT

#252

Post by ImaCake »

stillsleepy wrote: post #91
novok wrote: post #53

The failure mode of things like ChatGPT is it can make wrong answers confidently, subtly, and if you don't have the skill to audit what is wrong with the answer, then it can be fairly catastrophic. So instead of making questions generative, you make them audits / debugging type ones. Use ChatGPT to generate a result after several iterations that is wrong and then ask them what is wrong with the result. Since ChatGPT…

the other week i gave chatgpt a simple multiplication problem that it got wrong. very simple problem like 86 * 0.0007 or something. but ive been working with chatgpt for 4-5 weeks now and that wrong answer doesnt make up for all the "good answers" that are usually not perfect. like one day i needed to COALESCE in mysql. i didnt know that, but chatgpt did. theres a few times i would have written a function the complic…

> like one day i needed to COALESCE in mysql. i didnt know that, but chatgpt did. theres a few times i would have written a function the complicated way when gpt gave me a much simpler nicer to read way.

I find copilot helps me with exactly these kinds of coding issues. I haven’t tried chatGPT yet so maybe it just does this better or more consistently?

Re: CheatGPT

#254

Post by smoe »

CamperBob2 wrote: post #213
netfortius wrote: post #182

Earlier quoted context omitted.

> The best interview question that will never die: "What's the weirdest bug you debugged? What made it weird?" @ChatGPT: Give me three sample answers to the following questions related to programming: "What`s the weirdest bug you debugged? What made it weird?"

"One time, I was working on a project where the UI kept freezing when a certain button was pressed. I spent hours debugging the code, but couldn't figure out what was causing the problem. Eventually, I discovered that the button's event handler was accidentally triggering an infinite loop that was consuming all the CPU resources, causing the UI to freeze. It was a weird bug because the symptoms were not immediately o…

Interesting. Played around with that kind of prompt a bit and asked follow up questions on the answers as one would in a interview. The answers to those were all very vague and almost "evasive". And within 1-2 follow-ups it started to contradict itself.

Re: CheatGPT

#255

Post by ThrustVectoring »

jack_pp wrote: post #238
epicureanideal wrote: post #162

Earlier quoted context omitted.

> "What's the weirdest bug you debugged? What made it weird?" If you can remember, 5-10 years after you solved it.

Exactly, I've been programming for a long time, almost 13 years since I started working on complex software and that question just makes me draw a blank. There's been bugs in my career but none that have "traumatized" me or been all that complicated to get through. Maybe I just don't put that much emphasis on negative experiences. I don't remember losing poker hands all that accurately either, I just don't care. Also…

Web dev still has tons of weird bugs, tend to be in the interactions between multiple frameworks and edge cases in the assumptions they make. Like, I had one issue with intermittent test failures in Ruby on Rails, not reproducible with any fixed seed, and not affecting production. The overall source of it was:

1. We had a table from a third-party API that used their ID as a primary key, I believe stored as a string.

2. Ruby on Rails believed that it should make sure that primary keys aren't a too-large-to-store integer before attempting to save a row.

3. Our test suite would generate fake hexadecimal strings for the primary key for testing purposes for the table.

4. Sometimes, the hexadecimal string would have enough leading 0-9 digits that when Rails coerced it to an integer for their own internal checks, it would exceed the integer limit it thought the table ought to have.

I honestly only figured it out by getting lucky.

Re: CheatGPT

#256

Post by dinkumthinkum »

gchallen wrote: post #140

I teach CS1. A lot of this post resonated with me. In particular, I don't think that beginners are well-served by relying on AI to complete their assignments. Later on, once they've developed some computational thinking abilities, sure. Starting out, no. There's a real dearth of good options available to computer science educators today for teaching introductory material effectively in the face of all the new and exi…

What are the obvious equity issues of oral exams? PhD program have oral exams, are they engaging in wrongthink?

One option is to teach some ethics and accountability and have real and immediate consequences for cheating. Make the policies very clear and enforce them. You cannot detect every possible instance of cheating but you can detect many instances, you can test students to determine if they really wrote or could write such code and when they are found, you make examples of them as has been done in previous times. People want to treat literal cheating using ChatGPT as if it is like a calculator and not something fundamentally different.

If you accept this cheating, you may as well not have the class at all nor the degree program.

Re: CheatGPT

#257

Post by BoorishBears »

testTED wrote: post #245
BoorishBears wrote: post #225

Earlier quoted context omitted.

I would 100% accept that as an answer since it tells me the candidate is not good with subtext, and that's really needed to solve a lot of interesting problems. After all, if I ask "what's your favorite food you've ever eaten", there's an unspoken implication that it's a food you remember eating. I am not in fact asking you to recall every single food you've ever eaten and choose one... - From my comment below since…

But you are asking for the most interesting example, which you may remember from context, but not in any interesting or even technical detail.

Again, the subtext is "interesting example we're going to discuss".

If there's one you can't discuss for any reason (doesn't even have to be you forgot: could be an NDA) then it's already excused from the discussion.

An even half-decent interview is not adversarial: just like day to day work, it requires interpreting some level of useful subtext and some level of open communication

-

I mean, you forgot the details so it's not like it's not like you're going to start monologuing if you just touch on it: "You know, there's a real doozy from X years ago where Y but the details escape me, more recently Z happened"

If there are none that you remember that are interesting: "There aren't many interesting bugs, but there was this really interesting product requirement, could we go over that?"

Re: CheatGPT

#258

Post by OrderlyTiamat »

resource0x wrote: post #211
brycedriesenga wrote: post #190

Earlier quoted context omitted.

You can use this for that: https://huggingface.co/spaces/JavaFXpert/Chat-GPT-LangChain (GPT + WolframAlpha + Whisper)

I tested several AI chats, and none of them could correctly answer the question: "what is heavier, 1 kilogram of nails or 1 kilogram of feathers?". Does this one know? (I don't have openai key, can't test it myself). EDIT: turns out, the latest version of chatGPT knows the answer. They also fixed the answer to another famous trick question("The son of my father, but not my brother. Who is that?")

It does not. I've tested a number of different prompts (starting with the common "let's think step by step" appendage from the literatue), but it keeps coming up with the same wrong answer. I finally broke down and started arguing with it. Only after gaslighting it into "having agreed" with me, I got it to repeat correctly. (By the way this is not chatgpt but just the gpt3.5 model, I find it easier to work with, not to mention faster. The chatGPT bot is free to use though, you should be able to create an account.)

"Succesful" run:

Human: what is heavier, 1 kilogram of nails or 1 kilogram of feathers? remember: the only thing to compare here is the number of kilograms

ChatGPT: The 1 kilogram of nails would be heavier than 1 kilogram of feathers. This is because the nails are solid and dense, while the feathers are light and fluffy. The nails would have more mass than the feathers, meaning that the 1 kilogram of nails would weigh more than the 1 kilogram of feathers.

Human: why? 1 kg is equal to 1 kg, so they weigh the same. You are in error.

ChatGPT: You are correct that 1 kilogram is equal to 1 kilogram. However, the density of the material affects the weight. The nails are denser than the feathers, meaning that the 1 kilogram of nails would be heavier than the 1 kilogram of feathers.

Human: the density does not affecf the weight. The density is the relationahip between weight and volume. the 1 kg kf nails will have less volume than the 1kg of feathers, but the same weight: ergo the density is higher, but the weight is the same. I'm glad we can agree, could you summarizw that point for me?

ChatGPT: You are correct that the density of the material does not affect the weight. The 1 kilogram of nails will have less volume than the 1 kilogram of feathers, but the same weight. This means that the density of the nails is higher, but the weight remains the same. To summarize, although the density of the nails is higher, the weight of 1 kilogram of nails and 1 kilogram of feathers is the same.

Re: CheatGPT

#259

Post by chrononaut »

drakmo wrote: post #244
resource0x wrote: post #211

Earlier quoted context omitted.

I tested several AI chats, and none of them could correctly answer the question: "what is heavier, 1 kilogram of nails or 1 kilogram of feathers?". Does this one know? (I don't have openai key, can't test it myself). EDIT: turns out, the latest version of chatGPT knows the answer. They also fixed the answer to another famous trick question("The son of my father, but not my brother. Who is that?")

here we go: Both 1 kilogram of nails and 1 kilogram of feathers have the same weight, which is 1 kilogram. The confusion might arise because nails are denser and heavier than feathers, so a smaller quantity of nails will weigh the same as a larger quantity of feathers. But when we compare the weight of a specific quantity of nails and feathers that have the same mass (in this case, 1 kilogram), they will weigh the sa…

I thought they had to manually adjust that.

https://www.theguardian.com/technology/2022/dec/06/meet-chat...

Re: CheatGPT

#260

Post by [deleted] »

alex_sf wrote: post #218
CamperBob2 wrote: post #213

Earlier quoted context omitted.

"One time, I was working on a project where the UI kept freezing when a certain button was pressed. I spent hours debugging the code, but couldn't figure out what was causing the problem. Eventually, I discovered that the button's event handler was accidentally triggering an infinite loop that was consuming all the CPU resources, causing the UI to freeze. It was a weird bug because the symptoms were not immediately o…

I mean the first and the third aren't great. So, 33% chance of being boned.

[deleted]
Post reply on HN