Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT: The Slickest Con Artist of All Time

tedgioia.substack.com

Post reply on HN
Print view View original

251–260 of 444 posts

Re: ChatGPT: The Slickest Con Artist of All Time

#251

Post by bo0O0od »

rayval wrote: post #199
pdonis wrote: post #78

Earlier quoted context omitted.

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No. You've fallen for exactly the con the article is describing. ChatGPT knows nothing . It is very good at generating text that seems plausible if you know nothing about the subject matter, but is not reliable as an actual source of information . If you happened t…

I was surprised to find ChatGPT does a better job of translating than Google translate, at least for one difficult task I needed to get done. I have been a Google Translate user for more than a decade. I use it a dozen or more times a day for multiple languages. It is a very useful tool. I recently had to translate a passage that included this phrase: 六六大顺 Google Translate gave me: Liuliu Dashun (basically no transla…

Seems far more likely to me that ChatGPT knows the meaning of specific idioms rather than knowing the cultural context of that number and using it in the translation.

Re: ChatGPT: The Slickest Con Artist of All Time

#252

Post by raldi »

cj wrote: post #226
raldi wrote: post #43

Earlier quoted context omitted.

I saw three autonomous cars driving around my neighborhood in just the past week, all with nobody at the wheel.

(Totally unrelated) I wonder how an autonomous car would navigate my narrow dead end road. The road isn't wide enough for 2 cars to pass, so whenever 2 cars pass each other on the road, 1 car needs to pull off onto the shoulder to make room. I have a feeling an autonomous car would take up the whole lane, and not be smart enough to pull off onto the shoulder. Meaning both cars would be stuck there unable to pass each…

After the first one got stuck, don’t you think a programmer would teach it to do exactly what you suggest? Usually before deploying them somewhere, they have safety drivers take them up and down every road at least once, and sometimes dozens of times.

Re: ChatGPT: The Slickest Con Artist of All Time

#253

Post by Retric »

je42 wrote: post #225
Retric wrote: post #198

Earlier quoted context omitted.

The problem is needing to verify accuracy makes it a really bad search engine replacement. A good con artist is generally truthful, they just know how to seem truthful even when they lie which exactly the issue with trusting ChatGPT etc. You can’t tell if the model is telling bullshit unless you already know the truth. This doesn’t make them useless in general, just a poor fit if you can’t easily fact check the resul…

Easy, you use a second source and double check. In google you would go to another search result. For ChatGPT you actually need to verify via Google or actually try it when it is code.

What’s the point of using ChatGPT when searching if you still need to verify the information through another search?

Re: ChatGPT: The Slickest Con Artist of All Time

#254

Post by HarHarVeryFunny »

joenot443 wrote: post #242
karpierz wrote: post #9

Earlier quoted context omitted.

I don't know that it's an "incredibly smart" anything; I think it'd be better described as incredibly well-read, with poor discretion. But there's definitely uses for a tool that produces a lot of boilerplate content that a human can then proofread (ex: writing cover letters, filling out unstructured paperwork, etc.) Edit: I personally underestimated the use-case, because the "more optimal" thing to do when auto-fill…

If you can think of a single individual who can demonstrate knowledge on the fly with the same accuracy and breadth as ChatGPT, I think most of us would agree in calling them “incredibly smart.”

If ChatGPT was a person, you'd immediately realize something was very wrong with them. Some kind of idiot savant perhaps who'd memorized an encyclopedia but was confused by how it all fit together, or maybe someone hallucinating and delusional, only able to hold coherent thoughts together for brief stretches of time before babbling gibberish that made no sense.

Re: ChatGPT: The Slickest Con Artist of All Time

#255

Post by westoncb »

pdonis wrote: post #78
_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No. You've fallen for exactly the con the article is describing. ChatGPT knows nothing . It is very good at generating text that seems plausible if you know nothing about the subject matter, but is not reliable as an actual source of information . If you happened t…

> If you happened to get information from it that turned out to be ok, you were lucky.

This is just saying that sometimes ChatGPT is right and sometimes it's wrong, with an implicit stance on the relative frequencies.

In my own experience it would be more accurate to say you're unlucky if it gets it wrong—but then much of this hinges on having a good sense of what it's going to be good at / useful for.

(The other thing I see people miss all the time when assessing its correctness: it is somewhat "on demand"—it may include bits that are vague and only correct-ish, but if you point out an error or request refinement you can generally get it to clear/tighten things up. It's a bit like using a Google Maps-like zooming UI, but in addition to being variably abstract it's always approximating: the potential for error is ever present.)

Re: ChatGPT: The Slickest Con Artist of All Time

#256

Post by damiankennedy »

renewiltord wrote: post #32

All these articles really sound like “I used an apple to hammer in a screw and it sucked. This has to be the worst plant-based object ever made”. It’s a common junior engineer approach. “I broke our database by running DROP TABLE cows in the console”. Yeah, dude, that’s possible. Just don’t do that. The point of tools isn’t to use them like Homer Simpson. But you know what, it doesn’t matter. Stay behind. Everyone el…

That actually happenned to me once. My manager's manager managed to delete 3 databases before finding me to tell me he cant access the data anymore.

Re: ChatGPT: The Slickest Con Artist of All Time

#257

Post by pdonis »

westoncb wrote: post #255
pdonis wrote: post #78

Earlier quoted context omitted.

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No. You've fallen for exactly the con the article is describing. ChatGPT knows nothing . It is very good at generating text that seems plausible if you know nothing about the subject matter, but is not reliable as an actual source of information . If you happened t…

> If you happened to get information from it that turned out to be ok, you were lucky. This is just saying that sometimes ChatGPT is right and sometimes it's wrong, with an implicit stance on the relative frequencies. In my own experience it would be more accurate to say you're unlucky if it gets it wrong—but then much of this hinges on having a good sense of what it's going to be good at / useful for. (The other thi…

> This is just saying that sometimes ChatGPT is right and sometimes it's wrong

No, it's saying that ChatGPT is not reliable. It's not reliable because getting things right or wrong is not part of its design at all. All it does is generate text that matches a pattern. There is nothing anywhere in it that filters or evaluates text based on any semantic connection with anything else.

Re: ChatGPT: The Slickest Con Artist of All Time

#259

Post by mokus »

soperj wrote: post #66
_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well This is the issue. It's more like the dumbest guy in the room confidently convincing the everyone else that they know what they're talking about.

I think the bigger point is that it’s like a guy in the room. ChatGPT shows that the industry is graduating from Turing test territory to IQ test territory, and a lot of people are really hung up on the fact it’s failing the latter.

Re: ChatGPT: The Slickest Con Artist of All Time

#260

Post by groby_b »

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

In all the fields I've tried (software/music/epidemiology/geosciences/mathematics) ChatGPT can mostly explain things at the lay level, but even there often gets things wrong that without more expert guidance, you will not notice.

Experts in each of these areas were consistently better at giving a correct explanation.

ChatGPT is great when occasionally being wrong doesn't matter. It's still pretty good when you can't be wrong, but you're an expert. But in any "no mistakes, I need to talk to an expert" situation, ChatGPT is setting you up for failure.

That's the key part - the information is unreliable/untrustworthy. As long as that is not a problem, it's awesome.

The problem is that it presents wrong data with absolute confidence in a very convincing voice. (Hence people calling it a con artist). It'll become a huge problem as it makes inroads in mis/disinfo campaigns.

(The usual tech problem. Don't let large scale interests touch it, and you'll be OK)

Post reply on HN