Live data from Hacker News

ChatGPT: The Slickest Con Artist of All Time

tedgioia.substack.com

131–140 of 444 posts

Re: ChatGPT: The Slickest Con Artist of All Time

#131
post #81

Earlier quoted context omitted.

There's a significant epistemological problem: how do you know that what it's telling you is true? For things that you yourself can verify easily, this isn't too much of a problem. It gives you some boost tips, you see whether the program works or not. But as you come to rely on it, and it matters whether the answers are correct in important ways, what validation/verification/QA steps can you apply to ensure that you…

I use it like Wikipedia: a good jumping off point. Often times I will forget an exact thing I wish to find online so I will ask ChatGPT to list multiple things that could be what I just described. I then take this list to my search engine du jour.

Good point about Wikipedia. In just a few years the image people had of Wikipedia changed from an untrustworthy joke (The Office had a joke about Michael Scott trusting Wikipedia) to one of the first places people went to get information.

Before that, there was a lot of concern about blogs when they were the hot new thing. Here’s a New York Times article[1] from that period discussing the blogging controversy:

> There are two obvious differences between bloggers and the traditional press: unlike bloggers, professional journalists have a) editors and b) the need to maintain a professional reputation so that sources will continue to talk to them. I've been a journalist for more than a decade, and on two occasions I asked acquaintances whether I could print information that they had told me in social situations. Both times, they made clear that if I published they would never speak to me again. Without a reputation for trustworthiness, neither friendship nor journalism can be sustained over time.

The concerns weren’t incorrect - Wikipedia or Blogs are often unreliable. But the concerns overlooked how useful these things could be despite their problems.

As an aside, it's interesting to see comments here bring up the potential unreliability of ChatGPT - and then say that people should go read Wikipedia instead.

[1] https://www.nytimes.com/2004/12/19/magazine/your-blog-or-min...

Re: ChatGPT: The Slickest Con Artist of All Time

#132
post #78
post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No. You've fallen for exactly the con the article is describing. ChatGPT knows nothing . It is very good at generating text that seems plausible if you know nothing about the subject matter, but is not reliable as an actual source of information . If you happened t…

So frustrating to see people complain that it provides wrong information. With this logic it can never know anything. It is not made for querying. It's made to complete the text you provide it, with some minor modifications to help it answer questions. It does know the most common patterns of characters in the internet which implicitly contain knowledge.

Re: ChatGPT: The Slickest Con Artist of All Time

#133
post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

ChatGPT/GPT fails a lot when you probe its knowledge, but I find it works really well when you give it an explanation of the whole domain of the problem, then something to do with it.

For example can give examples of some programming language, or an API, and natural language explaining what you want to do with it (or code in another language), and it will translate this with remarkable accuracy, 0 shot. We've never had anything that's even close to this capability.

Re: ChatGPT: The Slickest Con Artist of All Time

#134

I had a detailed conversion with chatGPT about how to gracefully handle terminating conditions of a rust program. It summarized cogently to register at_exit()’s for each thread, panic handlers, and register signal handlers. It advised and explain in detail on my query about the thread handling for each of these variants, gave really helpful advice on collecting join handles in a closure on the main thread and waiting…

When I got out of school I went to work for Oracle and decided to read the database manuals, even though I wasn't working on database stuff. When I left I read other database manuals and was able to help people code database stuff, not because I was a real database expert, but because I had read the manuals and they hadn't.

I think something similar is going on here? For some subjects and some questions, ChatGPT acts like a co-worker who has read all the manuals, and that's enough.

Re: ChatGPT: The Slickest Con Artist of All Time

#135

After having played it ChatGPT for a bit, mostly asking computer questions, I've had mixed results. Some are amazing, others are gibberish. But what struck me the other day is a couple of quotes from, of all things, Galaxy Quest which seem particularly apt. "May I remind you that this man is wearing a costume, not a uniform." and "You know, with all that makeup and stuff, I actually thought you were SMART for a secon…

It's quite limited, in frequently making even basic mistakes, but when it works well, it displays real intelligence. It synthesizes new knowledge, as opposed to just retrieving it. When it works well, It's not just a magic trick.

That’s not intelligence.

If I generated and printed every possible 300 page book and handed all of those books to you, you’d get a book about the American Revolution where Taylor Swift led the armies of the revolutionaries.

That would not make my system intelligent. I haven’t written anything because I don’t know how to write. I’ve literally just got every possible book and you’ve been given one at random that seems relevant to what you want.

Re: ChatGPT: The Slickest Con Artist of All Time

#136

Earlier quoted context omitted.

That's exactly the wrong sort of thing to turn to ChatGPT for. In probing it for more detail, you're giving it increasingly helpful hints on how to answer your questions in a way that makes you satisfied, regardless of accuracy. That's what it's optimized to do. And because you're working from a humble and curious understanding of the topic yourself, it can and will make up all sorts of things that make you believe i…

To OP’s point what’s with the gate keeping on “right” or “wrong” ways to use it? It’s a tool. It has flaws. It can be useful and if it is, have at it.

The issue is that approach maximizes its flaws. It’s like shooing yourself in the foot with a nail gun, that’s objectively the wrong thing to do.

Re: ChatGPT: The Slickest Con Artist of All Time

#137

Earlier quoted context omitted.

That's exactly the wrong sort of thing to turn to ChatGPT for. In probing it for more detail, you're giving it increasingly helpful hints on how to answer your questions in a way that makes you satisfied, regardless of accuracy. That's what it's optimized to do. And because you're working from a humble and curious understanding of the topic yourself, it can and will make up all sorts of things that make you believe i…

GP stated that they asked ChatGPT about a topic they already were familiar with, from other sources. And that the answer made sense.

Conversely I asked it some simple questions about some songs' harmonic structures, and music theory, and it got things quite wrong.

There's already an example of it in the article where it incorrectly stated the tritone of F was Bb.

Re: ChatGPT: The Slickest Con Artist of All Time

#140
post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

Agree. ChatGPT provides great results while doing quite a bit of dreaming at the same time. The role of the human is to separate dreams from reality.
Post reply on HN