ChatGPT: The Slickest Con Artist of All Time • Hacker News Board

Re: ChatGPT: The Slickest Con Artist of All Time

#191

Post by shagie » Fri, Feb 03, 2023, 6:07 PM UTC

mrstone wrote: post #99

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

I agree. I regularly use ChatGPT to help reword paragraphs to be more succinct. I rarely ask it for actual information that it has to look up. At its core, it's an LLM which models how language is structured, and that's what it's good at.

It has an impressive amount of understanding about language. There was a meme a bit ago about "Everyone on my floor is coding" and the different reaction of a software developer and a doctor. So I asked for jokes about a doctor and a software developer about the different meanings for the words they used.

    Write five jokes about a doctor and a programmer about the different meanings of words they use.

To which GPT responded:

    1. Doctor: I'm sending you a prescription.
    Programmer: Great! I'll take a look at it and see if I can debug it.

    2. Doctor: I need you to take two aspirin.
    Programmer: Is that enough to solve the problem?

    3. Doctor: I'm going to give you some instructions.
    Programmer: Do I need to compile them or can I just run them?

    4. Doctor: I need you to take a break from work.
    Programmer: Does that mean I should pause the program or stop it completely?

    5. Doctor: I need you to stay hydrated.
    Programmer: Does that mean I should store more data or drink more water?

I will certainly admit that those jokes are awful, but the interesting part of it is that... it did it. I think that 3, 4 and 5 are the "best".

I don't think it understands those things with the ability to reason about them, but that demonstrates more than I would expect from a Chinese room.

    Explain the joke "Doctor: I need you to stay hydrated.  Programmer: Does that mean I should store more data or drink more water?"

returns back:

    The joke is a play on the terms "hydrate" and "store data". In programming, "storing data" is the process of saving information to a computer's memory, while "hydrating" means drinking water to stay hydrated. The joke is implying that the programmer is more comfortable with programming than with taking care of their own health, so they interpret the doctor's advice to stay hydrated as a suggestion to store more data.

Re: ChatGPT: The Slickest Con Artist of All Time

#192

Post by pdonis » Fri, Feb 03, 2023, 6:08 PM UTC

Throwaway78389 wrote: post #128

pdonis wrote: post #78

Earlier quoted context omitted.

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No. You've fallen for exactly the con the article is describing. ChatGPT knows nothing . It is very good at generating text that seems plausible if you know nothing about the subject matter, but is not reliable as an actual source of information . If you happened t…

How many hours you spent trying ChatGPT out? I spent at least high tens, maybe even hundreds. You're absolutely wrong. Yes, it hallucinates, yes its wrong about obscure topics - but calling having success with it luck is absolutely wrong. It's very consistently good. Especially about things like programming, physics, math - and now I'm using it as my teaching assistant for my pilot training, it's perfect (and I can v…

> I can very simply verify the answers are good with my FAA Pilot Handbook

Thank you for agreeing with my point.

If you need to check the answers with your FAA Pilot Handbook, wouldn't it be simpler just to read the FAA Pilot Handbook? That handbook, unlike the text generated by ChatGPT, was written by a process that was aware of the semantic relationship between the text being written and the things in the world that the text was referring to. That is what makes the FAA Pilot Handbook a reliable source of information about flying. ChatGPT is not any such process. That's why it is not a reliable source of information--as you agree, since you need to check what it says about flying with the FAA Pilot Handbook.

Re: ChatGPT: The Slickest Con Artist of All Time

#193

Post by Forgeties79 » Fri, Feb 03, 2023, 6:08 PM UTC

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

>but it is still ludicrously useful.

Wrote me a script to quickly convert SRT to VTT files as I edit and prep for uploading online. I haven't done anything akin to "coding" since I took a very basic class in college and messed around with HTML/CSS. Shaves several steps off of every video I do (I'm in a high volume world here).

Re: ChatGPT: The Slickest Con Artist of All Time

#194

Post by beepbooptheory » Fri, Feb 03, 2023, 6:10 PM UTC

tempest_ wrote: post #84

webstrand wrote: post #27

Earlier quoted context omitted.

In my experience from asking it questions about things that I know well, the deeper you get into the weeds with it, the more likely it is to invent some kind of extremely plausible-sounding but incorrect answer. How confident are you that what it told you is actually correct?

That is the one thing I think about when people say "I used it to learn X and it was very good" but when people are learning they are not in a position to to judge if what they learned is correct. A lot of other stuff is just regurgitation of a wiki article. Still it is very interesting and definitely a step forward.

And then we must remember that wiki articles themselves can also be quite wrong.

Re: ChatGPT: The Slickest Con Artist of All Time

#195

Post by rhdunn » Fri, Feb 03, 2023, 6:11 PM UTC

Karunamon wrote: post #163

kibwen wrote: post #110

Earlier quoted context omitted.

In other words, everyone should assume that it's lying at all times. Nobody should be using a product that both lies to you and then lies to you about the fact that it's lying to you.

Stating a falsehood without intent to deceive is just incorrectness, not a "lie". Let's dial the hyperbole back a few notches.

That would depend on a) how OpenAI are marketing it, and b) what disclaimers/etc. they put in the UI.

Re: ChatGPT: The Slickest Con Artist of All Time

#196

Post by Obscurity4340 » Fri, Feb 03, 2023, 6:12 PM UTC

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

Its honestly astounding how coherent and dare, I say, eloquent it can be. From factual "Google"-type inquiries to jokes to whatever. I particularly enjoy the poetry it generates in defined formats like limericks, sonnets, etc. Yet to encounter any actual "bullshit" that it is frequently accused of (although I tend to ask closed-type questions or provide very specific parameters with which to constrain its output)

Re: ChatGPT: The Slickest Con Artist of All Time

#197

Post by redox99 » Fri, Feb 03, 2023, 6:21 PM UTC

hooande wrote: post #169

hospadar wrote: post #112

I wonder if the biggest shortcoming of GPT right now is not that it sometimes gets things wrong, or can't cite its sources, or whatever - maybe it needs to learn when to say "I don't know the answer to that question". That's a pretty hard thing for most humans (and myself) to learn to say, and I suspect GPT's training data (tha internet) doesn't include a lot of "I'm not sure" language and probably does include a lot…

You're flat out incorrect about this. GPT loves to say that it doesn't know things. Have you actually used it? Don't take my word for it. Ask it: "How many people are wearing tennis shoes in the United States right now?" The response I got: "I'm sorry, I don't have the current data on the number of people in the United States who are wearing tennis shoes right now. It's not possible to determine this number as it wou…

I've actually used it quite a bit and he's not "flat out incorrect". I'd say you're the one wrong.

It might reply that when you ask it something that it can deduce it can't know, but ask it something reasonable and it will almost certainly hallucinate it instead of telling you it doesn't know about it.

> What function in Unreal Engine makes an actor transparent?

> The function to make an actor transparent in Unreal Engine is called "SetActorOpacity".

(There is no function called SetActorOpacity)

Re: ChatGPT: The Slickest Con Artist of All Time

#198

Post by Retric » Fri, Feb 03, 2023, 6:22 PM UTC

2devnull wrote: post #173

Retric wrote: post #130

Earlier quoted context omitted.

ChatGPT is optimized to seem to make sense, it’s like talking to a world class con artist you won’t notice it’s talking nonsense. Thus is subtly dangerous in ways that aren’t obvious in the beginning. Sure it seems great at writing boilerplate code, unfortunately it’s really good and writing the kind of bugs that you don’t notice. This is maddening in a completely novel way, like working with an actively malicious au…

You can verify it’s accuracy. The metaphor of “con artist” is a poor one. It has no adversarial interest. A con artist does. Furthermore, con stands for confidence. A con artist manipulates confidence. Gpt and other language models are “what it says on the tin.” Not understanding the fallibility of LLM is not the fault of the technology. It’s the user’s fault. Not at all like a con artist. More like a tool that can b…

The problem is needing to verify accuracy makes it a really bad search engine replacement. A good con artist is generally truthful, they just know how to seem truthful even when they lie which exactly the issue with trusting ChatGPT etc. You can’t tell if the model is telling bullshit unless you already know the truth.

This doesn’t make them useless in general, just a poor fit if you can’t easily fact check the results.

Re: ChatGPT: The Slickest Con Artist of All Time

#199

Post by rayval » Fri, Feb 03, 2023, 6:23 PM UTC

pdonis wrote: post #78

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No. You've fallen for exactly the con the article is describing. ChatGPT knows nothing . It is very good at generating text that seems plausible if you know nothing about the subject matter, but is not reliable as an actual source of information . If you happened t…

I was surprised to find ChatGPT does a better job of translating than Google translate, at least for one difficult task I needed to get done.

I have been a Google Translate user for more than a decade. I use it a dozen or more times a day for multiple languages. It is a very useful tool.

I recently had to translate a passage that included this phrase: 六六大顺

Google Translate gave me: Liuliu Dashun (basically no translation, just a transliteration or romanization)

Bing Translator: Six Six Dashun (a little bit better because 六 does mean six in some contexts.

Baidu: Liu Liu (basically no translation)

Naver: Six or Six Dashun (partial translation, plus added an incorrect particle)

Only ChatGPT gave the correct answer: "Everything going smoothly"

There is a cultural context that is essential to the translation, which apparently only ChatGPT has. It turns out that the number 6 is a lucky number in Chinese. It has the meaning of being “auspicious”. In Chinese culture, there is an idiom called “六六大顺 (liù liù dà shùn)”, which means “Everything comes in a smooth way” or more naturally "Everything going smoothly".

BTW, after seeing the ChatGTP results, I found this contextual info using Google search, in a website http://www.ichineselearning.com

It is ironic that Google search "knows" (helps you find) the cultural context but Google Translate apparently does not leverage this info.

Re: ChatGPT: The Slickest Con Artist of All Time

#200

Post by bjhartin » Fri, Feb 03, 2023, 6:23 PM UTC

msm_ wrote: post #102

pdonis wrote: post #78

Earlier quoted context omitted.

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No. You've fallen for exactly the con the article is describing. ChatGPT knows nothing . It is very good at generating text that seems plausible if you know nothing about the subject matter, but is not reliable as an actual source of information . If you happened t…

I think that's unnecessarily pessimistic view. I have recently started using ChatGPT for mid-hard questions (when I can't get an answer immediately using Google), mostly "how do I" questions. It lies of dreams up fake answers about half of the time, but this still means that 50% of time I get my solution without digging through obscure manuals. > It is basically like having an incredibly smart engineer/scientists/phi…

It seems to me the utility of the 'good' 50% depends entirely upon your ability to recognize it.

How do you know when to dig through those obscure manuals?

It seems to me that it'd be more useful when:

a) I don't care whether the text says things which are not true

Write me a poem about ...

b) I care, and will have to verify it's results, but that cost is worth it

Generate some test data ...

I think we're at a very dangerous intersection between an apparent decline in American's ability to detect nonsense and an automated way to create convincing nonsense.