Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT: The Slickest Con Artist of All Time

tedgioia.substack.com

Post reply on HN
Print view View original

171–180 of 444 posts

Re: ChatGPT: The Slickest Con Artist of All Time

#171

Post by godelski »

weinzierl wrote: post #166
_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

> "It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. Does this "person" make mistakes? " Does this person stubbornly insist on being right, even if they are wrong? Likely, that's just how humans usually behave - and ChatGPT does the same. At least that was my thinking when people complain about ChatGPT being a know-…

> Does this person stubbornly insist on being right, even if they are wrong?

Actually ChatGPT will almost always change tune if you tell it that it is wrong. That's because it is a stochastic parrot and that's an "unlikely" event if you actually got the answer right. But we'll see if this gets updated/broken too.

Re: ChatGPT: The Slickest Con Artist of All Time

#172

Post by godelski »

Karunamon wrote: post #167
weinzierl wrote: post #166

Earlier quoted context omitted.

> "It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. Does this "person" make mistakes? " Does this person stubbornly insist on being right, even if they are wrong? Likely, that's just how humans usually behave - and ChatGPT does the same. At least that was my thinking when people complain about ChatGPT being a know-…

ChatGPT is pretty amenable to being corrected if it says something you know is crap. And then, in my experience, you usually get better results. That is not something I can say for most humans.

Only if you correct a wrong answer. If you "correct" a right answer ("I don't think that's right") then it gives an even worse answer. So be careful when correcting.

Re: ChatGPT: The Slickest Con Artist of All Time

#173

Post by 2devnull »

Retric wrote: post #130
darkwater wrote: post #97

Earlier quoted context omitted.

GP stated that they asked ChatGPT about a topic they already were familiar with, from other sources. And that the answer made sense.

ChatGPT is optimized to seem to make sense, it’s like talking to a world class con artist you won’t notice it’s talking nonsense. Thus is subtly dangerous in ways that aren’t obvious in the beginning. Sure it seems great at writing boilerplate code, unfortunately it’s really good and writing the kind of bugs that you don’t notice. This is maddening in a completely novel way, like working with an actively malicious au…

You can verify it’s accuracy. The metaphor of “con artist” is a poor one. It has no adversarial interest. A con artist does. Furthermore, con stands for confidence. A con artist manipulates confidence. Gpt and other language models are “what it says on the tin.”

Not understanding the fallibility of LLM is not the fault of the technology. It’s the user’s fault.

Not at all like a con artist. More like a tool that can be dangerous in the hands of the uneducated. That is a very large set of human tools, from nuclear reaction to pharmaceuticals to the automobile. Perhaps we just need “drivers ed” for large language models.

Re: ChatGPT: The Slickest Con Artist of All Time

#174

Post by bagacrap »

cribbles wrote: post #58

I would greatly appreciate a moratorium on this genre of article until there is compelling accompanying evidence that a meaningful portion of ChatGPT's users are unaware of these shortcomings. I have yet to encounter or even hear of a non-technical person playing around with ChatGPT without stumbling into the type of confidently-stated absurdities and half-truths displayed in this article, and embracing that as a lim…

we still use Wikipedia because of convenience and not reliability, so I'm not sure what your point is. Humans will choose convenience over basically any other quality. See: kcups. Doesn't mean kcups are a net win for the world

Re: ChatGPT: The Slickest Con Artist of All Time

#175

Post by flir »

I have a simple canary for ChatGPT correctness that I ask every time it's updated: "What can you tell me about Ice Cold In Alex?" / "Who did Sylvia Syms play?"

I'm not expecting it to get the answer right (I don't think it has that information) but I'm hoping it'll eventually just admit it doesn't know instead of making up something plausible ("Sister Margaret Parker" last time I tried).

As long as it doesn't know what it doesn't know, I'm inclined to think of it as a super-advanced Markov chain. Useful, impressive, but still basically a statistical trick.

Re: ChatGPT: The Slickest Con Artist of All Time

#176

Post by checkyoursudo »

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field.

I have yet to see a single instance of this with any query that requires even a modicum of expertise. I have asked it a lot of questions about my field. I have asked it to summarize and paraphrase and ELI5 all kinds of shit. Everything it produces is either inane or a regurgitation of the input just with verbs and prepositions changed.

I would like to see a nice example, input and output, of this. I really would.

Re: ChatGPT: The Slickest Con Artist of All Time

#178

Post by ronjobber »

rednerrus wrote: post #157
ineptech wrote: post #154

The more I work with LLMs, the more I think of them as plagiarization engines. They do to text what a bitcoin tumbler does to bitcoins: slice them up and recombine them so that it's difficult to trace any specific part of the output to any specific part of the input. It's not a perfect analogy, but it's useful in that it produces correct answers about what LLMs are and aren't good for. For example, the reason they ma…

We're all kind of plagiariztion engines. We're all just slicing up and combining ideas we picked up somewhere else.

I think the difference is that when we communicate those ideas, in scientific/academic literature at least, we reference the source of those original ideas, which permits verification and also plays an important social role.

Re: ChatGPT: The Slickest Con Artist of All Time

#179

Post by shawabawa3 »

checkyoursudo wrote: post #176
_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. I have yet to see a single instance of this with any query that requires even a modicum of expertise. I have asked it a lot of questions about my field. I have asked it to summarize and paraphrase and ELI5 all kinds of shit. Everything it produces is either inane o…

First thing that came to mind:

> Explain how git works in very simple english

Git is a version control system for software development that helps track changes made to the source code over time. It allows multiple developers to collaborate and make changes to the same codebase without interfering with each other.

When you start a project in Git, you create a "repository" that stores all the files and their changes. You can then "commit" changes to the repository, which is like saving a snapshot of the code at a certain point in time. You can also "push" the changes to a remote repository on a server, so that other people can access and work on the code.

If someone else makes changes to the code, you can "pull" the updated version of the repository to your local machine and "merge" the changes with your own code. Git also allows you to create "branches" of the code, so that you can work on different features or fixes without affecting the main code.

By using Git, you can easily revert back to previous versions of the code, compare changes between different versions, and track who made which changes

Re: ChatGPT: The Slickest Con Artist of All Time

#180

Post by braingenious »

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc

It is a copy/paste machine that strings together words to make sentences that look like sentences it’s read.

So it’s like an incredibly smart engineer/scientist/philosopher that does not and fundamentally cannot understand any of those concepts (or any concepts at all.)

A parrot that can recite Macbeth’s monologue is not a s Shakespeare scholar.

Post reply on HN