Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT: The Slickest Con Artist of All Time

tedgioia.substack.com

Post reply on HN
Print view View original

321–330 of 444 posts

Re: ChatGPT: The Slickest Con Artist of All Time

#321

Post by gunshai »

fortituded0002 wrote: post #306
gunshai wrote: post #49

My stance is pretty simple. The folks that adapt their own language centers and domain reasoning around using chatGPT (or these types of models) will stand to gain the most out of using them. This article is an eye roll to me, a calculator gives you confidence as well, doesn't mean you used it correctly. It is very hard for me to not outright dismiss articles like this that don't consider the usefulness of the tool.…

I have the same overall reaction. But suspect that your analogy with the calculator to distract some people reading this. Difference being that one is user input error and the other is either ChatGPT misunderstanding what's being asked or just lacking training data and presenting an answer that's incorrect. By yes, an eye roll from me as well. A few months back I heard the horror stories about how a bot answers with…

> misunderstanding what's being asked or just lacking training data and presenting an answer that's incorrect.

I suppose I just don't think we humans are so different. In fact we often lack training data and certainly lack the ability to iterate quickly. In the case of the modern calculator we have the benefit of all the training data necessary to design the system properly, but at its initial inception not so much. As more "training data" or experience with circuit design and applied mathematics the returned output of the calculator improved.

Maybe my expansion of the analogy is off or too esoteric.

Re: ChatGPT: The Slickest Con Artist of All Time

#322

Post by psyklic »

cribbles wrote: post #58

I would greatly appreciate a moratorium on this genre of article until there is compelling accompanying evidence that a meaningful portion of ChatGPT's users are unaware of these shortcomings. I have yet to encounter or even hear of a non-technical person playing around with ChatGPT without stumbling into the type of confidently-stated absurdities and half-truths displayed in this article, and embracing that as a lim…

I showed ChatGPT to some non-technical people, and they immediately asked it political-related questions, such as about carbon emissions. (I assume hoping it would affirm their belief.) These things are very nuanced -- even if the response is technically accurate, it can still leave out important items or falsely suggest importance via the specific wording.

Re: ChatGPT: The Slickest Con Artist of All Time

#323

Post by fnordpiglet »

randomdigitz wrote: post #294
qayxc wrote: post #275

Earlier quoted context omitted.

> Took 20-30 follow-up messages, telling it to add things, remove things, fix things, make things compatible with the versions of software it was running etc. So in other words you basically spent just as much time and effort as if you did it yourself?

This is why I stopped using it as much. Realized it was taking more time asking it questions than just doing the work.

I don’t use it to do work I can do already, I use it to bootstrap in areas of confusion.

Re: ChatGPT: The Slickest Con Artist of All Time

#324

Post by fnordpiglet »

jwie wrote: post #67
fnordpiglet wrote: post #17

I had a detailed conversion with chatGPT about how to gracefully handle terminating conditions of a rust program. It summarized cogently to register at_exit()’s for each thread, panic handlers, and register signal handlers. It advised and explain in detail on my query about the thread handling for each of these variants, gave really helpful advice on collecting join handles in a closure on the main thread and waiting…

It’s only as good as the information it harvested. In a space where less true information is available, the results would be quite poor.

Same would be true for searching for the results or trying to find a human who is familiar. In a space where less true information is available, the results would be quite poor for literally everything.

Re: ChatGPT: The Slickest Con Artist of All Time

#325

Post by fnordpiglet »

worik wrote: post #298
MagicMoonlight wrote: post #121

Earlier quoted context omitted.

Okay so it generated a response which was “reasonable” How do you know it was correct? Because you checked it’s entire output manually and determined it probably wasn’t too wrong? So what happens if you now trust it to write firmware for some difficult old timey hardware that nobody understands anymore. It seems correct. But then it actually was just making it up and the coolant system of the power plant breaks and k…

> So what happens if you now trust it to write firmware for some difficult old timey hardware that nobody understands anymore. How well would anyone do? Would you trust me?

You should see who they hire to write firmware. I wouldn’t trust them with my cable box.

Re: ChatGPT: The Slickest Con Artist of All Time

#326

Post by fnordpiglet »

MagicMoonlight wrote: post #121
rqtwteye wrote: post #111

Earlier quoted context omitted.

Same for me. I fed it a few requirements and test objectives and its comments were pretty reasonable. With a little specialized training it will probably do better than most systems engineers or testers I know.

Okay so it generated a response which was “reasonable” How do you know it was correct? Because you checked it’s entire output manually and determined it probably wasn’t too wrong? So what happens if you now trust it to write firmware for some difficult old timey hardware that nobody understands anymore. It seems correct. But then it actually was just making it up and the coolant system of the power plant breaks and k…

By trying to run it usually. It is sometimes wrong, and I amend things. But I’ve had more occasions where I thought I was right and it was wrong and after a long debugging I realized I had failed to grok some edge in the language and it was indeed correct and I learned something new.

But I would suggest not using a LLM to make nuclear reactor control system code, just like Java.

Re: ChatGPT: The Slickest Con Artist of All Time

#327

Post by hesk »

hooande wrote: post #169
hospadar wrote: post #112

I wonder if the biggest shortcoming of GPT right now is not that it sometimes gets things wrong, or can't cite its sources, or whatever - maybe it needs to learn when to say "I don't know the answer to that question". That's a pretty hard thing for most humans (and myself) to learn to say, and I suspect GPT's training data (tha internet) doesn't include a lot of "I'm not sure" language and probably does include a lot…

You're flat out incorrect about this. GPT loves to say that it doesn't know things. Have you actually used it? Don't take my word for it. Ask it: "How many people are wearing tennis shoes in the United States right now?" The response I got: "I'm sorry, I don't have the current data on the number of people in the United States who are wearing tennis shoes right now. It's not possible to determine this number as it wou…

No, it's bullshit. Notice that the answer is not internally consistent. It first says that it does not have current data on that, which implies that it might have old data and that it would in principle be possible to generate that data. It then states that it is not possible to answer this question. Those two parts of the answer are not consistent with each other.

Re: ChatGPT: The Slickest Con Artist of All Time

#328

Post by fnordpiglet »

freejazz wrote: post #268
fnordpiglet wrote: post #17

I had a detailed conversion with chatGPT about how to gracefully handle terminating conditions of a rust program. It summarized cogently to register at_exit()’s for each thread, panic handlers, and register signal handlers. It advised and explain in detail on my query about the thread handling for each of these variants, gave really helpful advice on collecting join handles in a closure on the main thread and waiting…

> "If this is a con, then consider me a mark." I already did before I read this part.

Stop PM’ing me about transferring funds. I’m not interested in the prince’s inheritance.

Re: ChatGPT: The Slickest Con Artist of All Time

#329

Post by dicknuckle »

moffkalast wrote: post #299
wvenable wrote: post #287

Earlier quoted context omitted.

I asked ChatGPT to create some 6502 assembly to multiply two numbers. It generated an iterative approach (add A to itself B times for A*B) but didn't include the actual addition. I told it that "hey, you forgot to do the addition in that code" and it promptly corrected itself. Personally, I don't consider that terrible given what I was asking it.

Yeah it feels like it performs really well if you know what needs to be done and how. Spare no detail from the description of how exactly it should do it and it'll usually get most of it right and save some of the time spent messing around with initial setup and boilerplate.

That's were I've had my best luck with it.

Re: ChatGPT: The Slickest Con Artist of All Time

#330

Post by Moissanite »

skybrian wrote: post #134
fnordpiglet wrote: post #17

I had a detailed conversion with chatGPT about how to gracefully handle terminating conditions of a rust program. It summarized cogently to register at_exit()’s for each thread, panic handlers, and register signal handlers. It advised and explain in detail on my query about the thread handling for each of these variants, gave really helpful advice on collecting join handles in a closure on the main thread and waiting…

When I got out of school I went to work for Oracle and decided to read the database manuals, even though I wasn't working on database stuff. When I left I read other database manuals and was able to help people code database stuff, not because I was a real database expert, but because I had read the manuals and they hadn't. I think something similar is going on here? For some subjects and some questions, ChatGPT acts…

Yes - from the examples I've seen, it appears that we're heading towards the Star Trek bridge computer. Even back in the days when TNG aired I don't think anyone really considered the computer "intelligent", just extremely capable and with a magical ability to fill in the blanks while also having a perfect memory for facts. Even without true intelligence (whatever that really is), there is no denying that having your own bridge computer would be great.

Of course, we're going to find some way to fuck it up with advertising, but that's because we live in the Terran Empire timeline.

Post reply on HN