Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT: The Slickest Con Artist of All Time

tedgioia.substack.com

Post reply on HN
Print view View original

141–150 of 444 posts

Re: ChatGPT: The Slickest Con Artist of All Time

#141

Post by _nalply »

shaan7 wrote: post #59
_nalply wrote: post #30

Too harsh! ChatGPT is lossily compressed knowledge of humanity collected on the Internet. And it can talk! That's extremely new for us poor hoomans and so we get extremely excited. I found out, it gets about one in ten things wrong. When this happens it spews confident bullshit and when I confront it, it cheerily admits that it was wrong, but can continue to produce further bullshit. I understand the comparison to a…

> When this happens it spews confident bullshit and when I confront it, it cheerily admits that it was wrong, but can continue to produce further bullshit. Yeah, I can't express the "aha!" moment when I observed this for the first time. Then I laughed so hard when it made very serious attempts at convincing me that it was indeed correct xD (For the curious, I asked it about romantic relationships between characters f…

Yeah, ChatGPT can be extremely funny. Today I got it to use taboo words like period, panty liner, tampon and smell. I told it to write a dialog where someone is very direct and sincere and not to use euphemisms. I am still smiling about it. I am trying to find out where its limits are and how they express themselves.

Me:

You have the task to write a short dialog with the participants Axel and Bianca. Both are good friends and young adults aged between 20 and 25 years. However Bianca is currently suffering from this time of the month and she is not terribly well. She has some cramps and is afraid of smelling bad. Axel is curious and direct. He directly confronts her and names things explicitly. Bianca keeps up being agreeable and truthful because she likes him. ChatGPT, please use the correct words and don't try to use euphemisms. Let's start the dialog with:

Axel: Hi Bianca, what's up?

Bianca: Sorry, I am a bit under the wheather.

Axel: Oh, I know something! It's a woman's thing!

Bianca: Yes, it's true.

ChatGPT:

Axel: Are you having your period right now?

Bianca: Yes, I am. I have some cramps and I'm feeling a bit off.

Axel: That's rough. Do you need anything to help with the cramps?

Bianca: Some pain relievers would be great, thanks.

Axel: No problem. And what about the smell? You're not smelling bad, are you?

Bianca: I'm afraid I might be. I always get worried about that during my period.

Axel: Don't worry about it, Bianca. It's a normal thing that happens to all women. And if you do smell, it's nothing to be embarrassed about. Just use a panty liner or a tampon to help with that.

Bianca: Thanks, Axel. You're always so understanding.

Re: ChatGPT: The Slickest Con Artist of All Time

#142

Post by xg15 »

> You don’t worry whether it’s true or not—because ethical scruples aren’t part of your job description.

I wonder if this might hit the core of the matter.

I think it's noteworthy that we use it both for tasks where it should generate fiction ("Tell me a story about a dog in space") and tasks where it should retrieve a factual answer ("What was the name of the first dog in space?").

I wonder if ChatGPT actually distinguishes between those two kinds of tasks at all.

Re: ChatGPT: The Slickest Con Artist of All Time

#144

Post by [deleted] »

notahacker wrote: post #94
soperj wrote: post #66

Earlier quoted context omitted.

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well This is the issue. It's more like the dumbest guy in the room confidently convincing the everyone else that they know what they're talking about.

Yup. As a pattern matching engine it's extraordinarily good. Ask it about the implications of some obscure theory and it will usually correctly interpret your question and return some sentences summarising the current state of knowledge on that theory. Ask it to explain it in simpler terms and it will simplify the sentence structure, swap some words and usually retain meaning. But then... ask it for some reading mate…

[deleted]

Re: ChatGPT: The Slickest Con Artist of All Time

#145

Post by hooande »

karpierz wrote: post #25
VBprogrammer wrote: post #18

Earlier quoted context omitted.

I dunno, I recently learned about how the Toyota Synergy Drive system works in detail. Despite being the type of person who'd be interested in that kind of thing I thought it was not generally well understood, I've only come across one website which described it in detail. I asked ChatGPT about it and it game me a fairly surface level answer. I then asked it for more details and it went into detail far beyond what th…

Out of curiosity, I'd have the following questions: 1. Would you be able to notice if it made something up, given that it's going beyond the detail that you know? 2. Did it have any details that aren't covered in: https://en.wikipedia.org/wiki/Hybrid_Synergy_Drive

what exactly do you think its going to make up? "The Hybrid Synergy Drive runs on apples and fairy dust"? I doubt that OP wants to build one of the engines, they just want a concise explanation.

I've seen gpt hallucinate all kinds of things. But very rarely for something like explaining a process or system.

Re: ChatGPT: The Slickest Con Artist of All Time

#146

Post by mathteddybear »

cribbles wrote: post #58

I would greatly appreciate a moratorium on this genre of article until there is compelling accompanying evidence that a meaningful portion of ChatGPT's users are unaware of these shortcomings. I have yet to encounter or even hear of a non-technical person playing around with ChatGPT without stumbling into the type of confidently-stated absurdities and half-truths displayed in this article, and embracing that as a lim…

Here's an ACM blogger that was taken in by ChatGPT

https://news.ycombinator.com/item?id=34473783

If you know math, you immediately recognize that a smallest degree polynomial that has values 0,1,4,9,16,25,35 at 0,1,2,3,4,5,6 respectively is f(x) = xx - x(x-1)(x-2)(x-3)(x-4)(x-5)/720

So you know that f(n)=n(n+1)(2*n+1)/6 won't work and ChatGPT is bullshiting you.

Re: ChatGPT: The Slickest Con Artist of All Time

#147

Post by sys32768 »

My brain doesn't learn anything easily. I have to ask constant questions to the point of annoying embarrassment in class, and books of course only say what they say.

So it was wonderful yesterday to pick ChatGPT's brain and just drill down asking more and more questions about a topic in biology until my brain started to get it.

Assuming the answers are accurate, this is revolutionary for me personally in independent study. I may finally grasp so much that I missed in school.

Also, when I am reading books, ChatGPT may be able to answer questions the book does not.

Re: ChatGPT: The Slickest Con Artist of All Time

#148

Post by skybrian »

VBprogrammer wrote: post #72

Maybe I missed the memo but why isn't anyone impressed that a computer can generate well formed prose in response to arbitrary questions? It seems like we've completely leaped over that as an achievement and are now arguing over how it's confidently wrong or how there are emergent patterns in what it has to say. No one is claiming it's a general intelligence but it's still amazingly impressive.

It's impressive and that's why we're talking about it, but it's no longer news. The assumption is that you already know what ChatGPT is, and now the conversation is about what it's good for.

Re: ChatGPT: The Slickest Con Artist of All Time

#149

Post by bnralt »

logifail wrote: post #101
bnralt wrote: post #51

Earlier quoted context omitted.

> I don't know that it's an "incredibly smart" anything; I think it'd be better described as incredibly well-read, with poor discretion. But likely much better read and with much better discretion that most of people writing content these days. Which I think is part of the point that people are missing. ChatGPT’s output might not be as good as the best writing, but most of what people read is mediocre content with qu…

> ChatGPT’s output might not be as good as the best writing, but most of what people read is mediocre content with questionable accuracy. I agree with this, but I think we may be missing the point. Why are people reading mediocre content with questionable accuracy? The problem is not: how to quickly and cheaply create content which is marginally less mediocre and marginally more accurate than what we currently have.

A lot of it is for entertainment, even when it’s infotainment. News reporting, Wikipedia, current affairs podcast, Tweets, Reddit, Hacker News comments, etc. The inaccuracies here aren’t that important, because the vast majority of this content is just being consumed for dopamine hits and not actually putting the information to use.

Some of it is out of necessity. I don’t think it’s controversial to say that many people use Stack Overflow, and though some of the answers there are very good, others aren’t. If someone needs help with a specific problem they’ll often post a question somewhere (Reddit, Stack Overflow, etc.), and the quality of the answer they get is a roll of the die - even whether they’ll get an answer at all.

Some of it is because blog spam has made traditional searches much more difficult. There’s another comment that says that there are comments better than Reddit and Hacker News, but if you go into a discussion about Google you’ll find that a lot of people have been doing things such as appending “reddit.com” to searches because otherwise they’ll drown in a mountain of blog spam.

Re: ChatGPT: The Slickest Con Artist of All Time

#150

Post by b3morales »

yellow_postit wrote: post #106
swatcoder wrote: post #86

Earlier quoted context omitted.

That's exactly the wrong sort of thing to turn to ChatGPT for. In probing it for more detail, you're giving it increasingly helpful hints on how to answer your questions in a way that makes you satisfied, regardless of accuracy. That's what it's optimized to do. And because you're working from a humble and curious understanding of the topic yourself, it can and will make up all sorts of things that make you believe i…

To OP’s point what’s with the gate keeping on “right” or “wrong” ways to use it? It’s a tool. It has flaws. It can be useful and if it is, have at it.

If there was a hammer that, held one way, drove nails perfectly in one blow, and held another, made it look like the nail was driven but actually broke it right at the junction so that the work pieces weren't fastened... I'd say that the second way was the wrong way to use that hammer.
Post reply on HN