Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT: The Slickest Con Artist of All Time

tedgioia.substack.com

Post reply on HN
Print view View original

201–210 of 444 posts

Re: ChatGPT: The Slickest Con Artist of All Time

#201

Post by Throwaway78389 »

pdonis wrote: post #192
Throwaway78389 wrote: post #128

Earlier quoted context omitted.

How many hours you spent trying ChatGPT out? I spent at least high tens, maybe even hundreds. You're absolutely wrong. Yes, it hallucinates, yes its wrong about obscure topics - but calling having success with it luck is absolutely wrong. It's very consistently good. Especially about things like programming, physics, math - and now I'm using it as my teaching assistant for my pilot training, it's perfect (and I can v…

> I can very simply verify the answers are good with my FAA Pilot Handbook Thank you for agreeing with my point. If you need to check the answers with your FAA Pilot Handbook, wouldn't it be simpler just to read the FAA Pilot Handbook? That handbook, unlike the text generated by ChatGPT, was written by a process that was aware of the semantic relationship between the text being written and the things in the world tha…

I don't think I agreed with anything you said.

I work with all available material in many different ways (Anki cards, videos, different explanations of the same thing, etc), and ChatGPT is another way to learn and help me generate learning material. For example I have it ask me questions like a tutor would. Or I ask various questions when I'm unsure about the wider context - e.g. it provides much more about the underlying physics than the Pilot Handbook itself. If I don't understand, I can ask for clarification, or an explanation like I am 5.

Reading the Pilot Handbook is a big part of learning but being limited to it would be hard. I'm very happy about having Chatgpt available.

Re: ChatGPT: The Slickest Con Artist of All Time

#202

Post by kibwen »

Karunamon wrote: post #187
kibwen wrote: post #184

Earlier quoted context omitted.

If the people running the platform are advertising a product that produces truths while delivering falsehoods and pathetically refusing to label their falsehoods as falsehoods, that's a lie.

There is a bit right on the chat box above every conversation that says, verbatim, "may occasionally generate incorrect information". Determining truth is not in the wheelhouse of a language learning model and it is not defective for not doing so in the same way a tablesaw is not defective for not driving nails. Adjust your expectations accordingly.

> "may occasionally generate incorrect information"

The points of this thread are 1) this is a dramatic understatement; ChatGPT's output is not just occasionally incorrect, but usually incorrect in some dimension, and 2) in the absence of any fine-grained confidence value for individual statements, you must pessimistically assume that all statements are incorrect until proven otherwise, which dramatically reduces the utility of the tool compared to how its fanboys portray it as the second coming of Christ.

Re: ChatGPT: The Slickest Con Artist of All Time

#203

Post by ChildOfChaos »

logifail wrote: post #101
bnralt wrote: post #51

Earlier quoted context omitted.

> I don't know that it's an "incredibly smart" anything; I think it'd be better described as incredibly well-read, with poor discretion. But likely much better read and with much better discretion that most of people writing content these days. Which I think is part of the point that people are missing. ChatGPT’s output might not be as good as the best writing, but most of what people read is mediocre content with qu…

> ChatGPT’s output might not be as good as the best writing, but most of what people read is mediocre content with questionable accuracy. I agree with this, but I think we may be missing the point. Why are people reading mediocre content with questionable accuracy? The problem is not: how to quickly and cheaply create content which is marginally less mediocre and marginally more accurate than what we currently have.

"Why are people reading mediocre content with questionable accuracy?"

Because it's literally everywhere you look and is very hard to filter due to huge amount of it everywhere. Particularly if you are trying to learn and grow on a certain subject, it's difficult to filter out what you don't know is 'mediocre' or 'questionable' at least not until you've already consumed it.

Re: ChatGPT: The Slickest Con Artist of All Time

#204

Post by saiya-jin »

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

I can see tons of jobs disappearing even with current level. Web these days is filled with mediocre articles, and quite a few bureaucratic jobs can and will be replaced by some next iteration.

Instead of army of mediocre journalists filling web portals, you just need 1 person to proof read it per business, maybe not even that. I can see the same for software dev in less than decade with current pace, 1 senior going over generated code and doing minor tweaks. Which is probably road to hell long term since how will juniors become seniors, but cost savings can be real and massive, drop in quality small if at all and it may be good enough for many.

It has very good language skills, in my obscure and rather difficult native language which only cca 6 million people use it formulated perfect sentences consistently. I can easily imagine faking social, even love chats easily. This can help lonely or elderly people (with good speech recognition/generation, since language skill itself is there already). With image/video generation porn industry will be steamrolled, 24/7 streams of anything you can imagine, looking realistic enough, tailored just to you.

Or just help with learning foreign languages, heck few months ago a guy was posting here on HN just a smart chat that could be used for training and got standing ovation also from me for that, this has it and everything else.

I understand google's reaction, instead of browsing through paid entries it gives me answer directly, no need to click through search and then wikipedia and then parsing texts. At least in some cases. This could help google massively if they could somehow do it on their scale.

People trying to find errors in it are missing the point now - its not about academic perfection, but often good-enough almost for free. Also, this is not final product, only 4th alpha/beta/whatever, previous ones were much worse. Imagine 10 or 20 years down the road...

Re: ChatGPT: The Slickest Con Artist of All Time

#205

Post by bglazer »

pdonis wrote: post #186
kvetching wrote: post #132

Earlier quoted context omitted.

So frustrating to see people complain that it provides wrong information. With this logic it can never know anything. It is not made for querying. It's made to complete the text you provide it, with some minor modifications to help it answer questions. It does know the most common patterns of characters in the internet which implicitly contain knowledge.

> With this logic it can never know anything. That's correct: ChatGPT does not know anything. That's not what it's built to do. > It does know the most common patterns of characters in the internet which implicitly contain knowledge. No, it doesn't. It knows patterns of text in its training data, but it knows nothing about the semantics of the text--its relationship to other things in the world, which is what is invo…

I share your skepticism of LLM’s output but I don’t think it’s fair to say they know nothing about semantics. I think it’s still an open question as to what degree LLM’s encode a coherent world model. Also you can just ask chatgpt about objects and their relationships and it gets the answer right way more often than you’d expect by chance, so it has some understanding of the world. Not good enough for me to trust it though

Re: ChatGPT: The Slickest Con Artist of All Time

#206

Post by rednerrus »

ronjobber wrote: post #178
rednerrus wrote: post #157

Earlier quoted context omitted.

We're all kind of plagiariztion engines. We're all just slicing up and combining ideas we picked up somewhere else.

I think the difference is that when we communicate those ideas, in scientific/academic literature at least, we reference the source of those original ideas, which permits verification and also plays an important social role.

You can ask it how it came up with its answer and it will do it's best to give you an explanation. It may not cite its sources directly but it will try to explain.

Re: ChatGPT: The Slickest Con Artist of All Time

#207

Post by svachalek »

hooande wrote: post #145
karpierz wrote: post #25

Earlier quoted context omitted.

Out of curiosity, I'd have the following questions: 1. Would you be able to notice if it made something up, given that it's going beyond the detail that you know? 2. Did it have any details that aren't covered in: https://en.wikipedia.org/wiki/Hybrid_Synergy_Drive

what exactly do you think its going to make up? "The Hybrid Synergy Drive runs on apples and fairy dust"? I doubt that OP wants to build one of the engines, they just want a concise explanation. I've seen gpt hallucinate all kinds of things. But very rarely for something like explaining a process or system.

It writes things that sound reasonable in context. So it will likely give something that sounds like a Popular Mechanics article about how some kind of machine works, but it may not really be this particular machine that you are asking about, or any actual machine in existence. That's the tricky thing about it, if it actually said "fairy dust" you know you can stop reading, but what it actually writes typically needs some actual understanding of the subject to refute.

Re: ChatGPT: The Slickest Con Artist of All Time

#208

Post by lamontcg »

pdonis wrote: post #78
_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No. You've fallen for exactly the con the article is describing. ChatGPT knows nothing . It is very good at generating text that seems plausible if you know nothing about the subject matter, but is not reliable as an actual source of information . If you happened t…

Yeah, ChatGPT is exploiting how bad humans (or at least Americans) are at detecting sociopathic liars and bullshit artists.

It is consistent with how awful our business management class is along with our politics and relationships.

Re: ChatGPT: The Slickest Con Artist of All Time

#209

Post by Karunamon »

kibwen wrote: post #202
Karunamon wrote: post #187

Earlier quoted context omitted.

There is a bit right on the chat box above every conversation that says, verbatim, "may occasionally generate incorrect information". Determining truth is not in the wheelhouse of a language learning model and it is not defective for not doing so in the same way a tablesaw is not defective for not driving nails. Adjust your expectations accordingly.

> "may occasionally generate incorrect information" The points of this thread are 1) this is a dramatic understatement; ChatGPT's output is not just occasionally incorrect, but usually incorrect in some dimension, and 2) in the absence of any fine-grained confidence value for individual statements, you must pessimistically assume that all statements are incorrect until proven otherwise, which dramatically reduces the…

The first point is a bald-faced assertion with only anecdotal evidence, the second is a reduction to the absurd. It is absurd because if you uncritically accept the words of anything you read online, without, say, validating them with common sense, your own experience and knowledge, and so forth, the problem is with the reader, not the words.

You and the author of this article are making this false dichotomy where there is no middle ground between "usually incorrect" (which is hyperbolic nonsense and trivially falsified by five minutes of using it), and "always correct" (which even your straw "fanboys" have not done to my knowledge), and then using this dichotomy to set up another one to pretend that the only way to act on information read from a computer screen is either uncritical acceptance or presume that it's bullshit.

Neither of these models are accurate and neither of them have any relation to how people in the real world generally interact with information online or with ChatGPT.

Furthermore, your insistence on "labeling falsehoods" is not something we can do accurately anyways, let alone in the context of a language model which has no concept of truth or falsehood in the first place! You are asking for something completely unreasonable and I can't tell if you're doing it out of ignorance or bad faith.

Re: ChatGPT: The Slickest Con Artist of All Time

#210

Post by mhink »

bnralt wrote: post #131
galleywest200 wrote: post #89

Earlier quoted context omitted.

I use it like Wikipedia: a good jumping off point. Often times I will forget an exact thing I wish to find online so I will ask ChatGPT to list multiple things that could be what I just described. I then take this list to my search engine du jour.

Good point about Wikipedia. In just a few years the image people had of Wikipedia changed from an untrustworthy joke (The Office had a joke about Michael Scott trusting Wikipedia) to one of the first places people went to get information. Before that, there was a lot of concern about blogs when they were the hot new thing. Here’s a New York Times article[1] from that period discussing the blogging controversy: > Ther…

> Good point about Wikipedia. In just a few years the image people had of Wikipedia changed from an untrustworthy joke (The Office had a joke about Michael Scott trusting Wikipedia) to one of the first places people went to get information.

> As an aside, it's interesting to see comments here bring up the potential unreliability of ChatGPT - and then say that people should go read Wikipedia instead.

On the other hand, Wikipedia's image changed largely due to the transparency behind its editorial processes- both the edit history and the discussions regarding editorial decisions are available in public. Are there pages where it's important to take the information presented with a grain of salt? Sure, but I have FAR more metadata available to help me discern how reliable the article may be.

And that's kinda the difference- having a way to figure out the source of the information. I was in high school when Wikipedia really took off, and although there were more than a few kids who got caught copying their essays from Wikipedia (and a few more who got chastised for the lesser sin of trying to cite it as a source), the vast majority of folks were perfectly capable of realizing that all you needed to do was scroll down to the list of citations and work from there.

With all that in mind, the contrast with ChatGPT should be pretty clear. You can ask it to explain a topic, and it'll probably do a decent job (or appear to do so) but the process by which it produced that output is extremely opaque. And (as has been noted in other threads) OpenAI hasn't been particularly forthcoming about how they influence its responses, which is pretty ironic considering their name.

Post reply on HN