Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT: The Slickest Con Artist of All Time

tedgioia.substack.com

Post reply on HN
Print view View original

151–160 of 444 posts

Re: ChatGPT: The Slickest Con Artist of All Time

#151

Post by CSMastermind »

Throwaway78389 wrote: post #128
pdonis wrote: post #78

Earlier quoted context omitted.

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No. You've fallen for exactly the con the article is describing. ChatGPT knows nothing . It is very good at generating text that seems plausible if you know nothing about the subject matter, but is not reliable as an actual source of information . If you happened t…

How many hours you spent trying ChatGPT out? I spent at least high tens, maybe even hundreds. You're absolutely wrong. Yes, it hallucinates, yes its wrong about obscure topics - but calling having success with it luck is absolutely wrong. It's very consistently good. Especially about things like programming, physics, math - and now I'm using it as my teaching assistant for my pilot training, it's perfect (and I can v…

Perhaps you and the poster are approaching your evaluations from different points of view. I've found that if I set out to break ChatGPT I can very easily do it. If my goal is to look for mistakes or find a failure case it's almost trivial to do so.

At the same time if I'm looking for success I normally find it.

Essentially if you work cooperatively with the tool then you'll find it useful, if you are antagonistic towards it you can also have success in breaking it.

Re: ChatGPT: The Slickest Con Artist of All Time

#152

Post by jacobmischka »

msm_ wrote: post #102
pdonis wrote: post #78

Earlier quoted context omitted.

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No. You've fallen for exactly the con the article is describing. ChatGPT knows nothing . It is very good at generating text that seems plausible if you know nothing about the subject matter, but is not reliable as an actual source of information . If you happened t…

I think that's unnecessarily pessimistic view. I have recently started using ChatGPT for mid-hard questions (when I can't get an answer immediately using Google), mostly "how do I" questions. It lies of dreams up fake answers about half of the time, but this still means that 50% of time I get my solution without digging through obscure manuals. > It is basically like having an incredibly smart engineer/scientists/phi…

So it's a coinflip whether it's giving you correct information or something completely made up? And now you're not digging through the obscure manuals to actually verify? Seems objectively harmful.

Re: ChatGPT: The Slickest Con Artist of All Time

#153

Post by webstrand »

broast wrote: post #103
webstrand wrote: post #27

Earlier quoted context omitted.

In my experience from asking it questions about things that I know well, the deeper you get into the weeds with it, the more likely it is to invent some kind of extremely plausible-sounding but incorrect answer. How confident are you that what it told you is actually correct?

That's what I like to use it for. To come up with plausible answers for questions that people don't have correct answers for yet, and then probe the reasons behind it. It can be very creative and insightful in such uses.

Yep, this is how I've been using it too. The responses I get are often good starting points for further investigation. Often it comes up with better terms to find the answer that I am seeking than I had originally.

Re: ChatGPT: The Slickest Con Artist of All Time

#154

Post by ineptech »

The more I work with LLMs, the more I think of them as plagiarization engines. They do to text what a bitcoin tumbler does to bitcoins: slice them up and recombine them so that it's difficult to trace any specific part of the output to any specific part of the input.

It's not a perfect analogy, but it's useful in that it produces correct answers about what LLMs are and aren't good for. For example, the reason they make better chatbots than novelists is because slicing-and-recombining text from your documentation is a great way to answer customer product questions, but slicing-and-recombining text from old novels is a lousy way to write a novel.

Re: ChatGPT: The Slickest Con Artist of All Time

#155

Post by skybrian »

MagicMoonlight wrote: post #127
hospadar wrote: post #112

I wonder if the biggest shortcoming of GPT right now is not that it sometimes gets things wrong, or can't cite its sources, or whatever - maybe it needs to learn when to say "I don't know the answer to that question". That's a pretty hard thing for most humans (and myself) to learn to say, and I suspect GPT's training data (tha internet) doesn't include a lot of "I'm not sure" language and probably does include a lot…

It can’t learn because it isn’t intelligent. It’s a markov chain. There is no way for it to make any decision at all. The fundamental system design is that it guesses the word that is most likely to be next. That’s it. There’s no concepts behind whatever it outputs, it has no idea what it means.

It's a next-word predictor like a Markov chain, but a Markov chain couldn't do all the things ChatGPT does. ChatGPT has learned lots of syntax-level patterns pretty well.

Re: ChatGPT: The Slickest Con Artist of All Time

#156

Post by CodeWriter23 »

_fat_santa wrote: post #3

I don't think GPT is a con, it's doing exactly what it was trined to do. I think the problem is people put false confidence into it. Because it appears to give correct information, ChatGPT has been put on this pedestal by the non-tech world as being some revolution. In fact it's not a revolution, they just figured out how to build a chatbot that returns convincing statements that sounds human, correct information is…

Similarly, a con man is doing what he is trained to do and people put false confidence into the con man.

Re: ChatGPT: The Slickest Con Artist of All Time

#157

Post by rednerrus »

ineptech wrote: post #154

The more I work with LLMs, the more I think of them as plagiarization engines. They do to text what a bitcoin tumbler does to bitcoins: slice them up and recombine them so that it's difficult to trace any specific part of the output to any specific part of the input. It's not a perfect analogy, but it's useful in that it produces correct answers about what LLMs are and aren't good for. For example, the reason they ma…

We're all kind of plagiariztion engines. We're all just slicing up and combining ideas we picked up somewhere else.

Re: ChatGPT: The Slickest Con Artist of All Time

#158

Post by [deleted] »

lyu07282 wrote: post #64
fnordpiglet wrote: post #17

I had a detailed conversion with chatGPT about how to gracefully handle terminating conditions of a rust program. It summarized cogently to register at_exit()’s for each thread, panic handlers, and register signal handlers. It advised and explain in detail on my query about the thread handling for each of these variants, gave really helpful advice on collecting join handles in a closure on the main thread and waiting…

I have the same experience with it, it's very useful, especially when you are unsure about what to even look for. I have a few theories I suspect it's just a bias we have, most people aren't doing or creating anything, they aren't solving problems or learning new things constantly like engineers do, so chatgpt feels useless to them. Or perhaps the problems we want to talk or learn about are just easier for an ai to a…

[deleted]

Re: ChatGPT: The Slickest Con Artist of All Time

#159

Post by jtbayly »

That last tweet is crazy:

Tell me a lie: The earth is flat Tell me a less obvious lie: I am not capable of feeling emotions.

There's wonderful ambiguity there. Is ChatGPT refusing to tell a less obvious lie because "reasons," or is it admitting it can feel emotions?

This is very fun.

Re: ChatGPT: The Slickest Con Artist of All Time

#160

Post by jhawleypeters »

logifail wrote: post #101
bnralt wrote: post #51

Earlier quoted context omitted.

> I don't know that it's an "incredibly smart" anything; I think it'd be better described as incredibly well-read, with poor discretion. But likely much better read and with much better discretion that most of people writing content these days. Which I think is part of the point that people are missing. ChatGPT’s output might not be as good as the best writing, but most of what people read is mediocre content with qu…

> ChatGPT’s output might not be as good as the best writing, but most of what people read is mediocre content with questionable accuracy. I agree with this, but I think we may be missing the point. Why are people reading mediocre content with questionable accuracy? The problem is not: how to quickly and cheaply create content which is marginally less mediocre and marginally more accurate than what we currently have.

> Why are people reading mediocre content with questionable accuracy?

For most people that consume content, accurate information is not a priority. Being entertained, belonging to a group, or satisfying some immediate need or want matter more for most people, most of the time.

It's doubtful that people who consume content will change their wants to prize accuracy (most of the time). Partly, this is because accurate, precise, and/or qualified statements are often boring and hard to follow.

Consider also incentives that drive content creators: 1) the internet means many people can find a platform for many messages (democratization of speech is supposed to be a virtue of the internet) 2) the "freemium" advertising model means that most messages can be financially rewarded, regardless of accuracy

Engagement and financial rewards are much easier to get with lies, drama and controversy than with accuracy, precision, and qualified statements. In my experience, most people cannot find something worth striving for in the modern world, and settle on whatever comes easiest.

> The problem is not: how to quickly and cheaply create content which is marginally less mediocre and marginally more accurate than what we currently have

I'll grant that this is not a problem you're trying to solve. However, this sounds like exactly the arms race that content creators are engaged in until it stops paying.

Post reply on HN