Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT: The Slickest Con Artist of All Time

tedgioia.substack.com

Post reply on HN
Print view View original

221–230 of 444 posts

Re: ChatGPT: The Slickest Con Artist of All Time

#221

Post by 2devnull »

Retric wrote: post #198
2devnull wrote: post #173

Earlier quoted context omitted.

You can verify it’s accuracy. The metaphor of “con artist” is a poor one. It has no adversarial interest. A con artist does. Furthermore, con stands for confidence. A con artist manipulates confidence. Gpt and other language models are “what it says on the tin.” Not understanding the fallibility of LLM is not the fault of the technology. It’s the user’s fault. Not at all like a con artist. More like a tool that can b…

The problem is needing to verify accuracy makes it a really bad search engine replacement. A good con artist is generally truthful, they just know how to seem truthful even when they lie which exactly the issue with trusting ChatGPT etc. You can’t tell if the model is telling bullshit unless you already know the truth. This doesn’t make them useless in general, just a poor fit if you can’t easily fact check the resul…

Good point. I don’t think they will be a good search engine replacement, but maybe a powerful supplement for some users. And honestly with SEO and other issues even search engines can be manipulated in ways that an average user might have difficulty detecting.

Re: ChatGPT: The Slickest Con Artist of All Time

#222

Post by freejazz »

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

>"It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field."

LOL where do you get this notion? I haven't seen one chatGPT bot useful for law. They all have no idea what they are doing and are basically dangerous if you don't know better than the LLM. There is literally nothing chatGPT can do for me as an attorney that isn't already solved by using a template.

Re: ChatGPT: The Slickest Con Artist of All Time

#223

Post by LarryMullins »

bnralt wrote: post #131
galleywest200 wrote: post #89

Earlier quoted context omitted.

I use it like Wikipedia: a good jumping off point. Often times I will forget an exact thing I wish to find online so I will ask ChatGPT to list multiple things that could be what I just described. I then take this list to my search engine du jour.

Good point about Wikipedia. In just a few years the image people had of Wikipedia changed from an untrustworthy joke (The Office had a joke about Michael Scott trusting Wikipedia) to one of the first places people went to get information. Before that, there was a lot of concern about blogs when they were the hot new thing. Here’s a New York Times article[1] from that period discussing the blogging controversy: > Ther…

For nearly as long as Wikipedia has existed, there have been many people who trust it and about as many people who deride those who trust it.

Re: ChatGPT: The Slickest Con Artist of All Time

#224

Post by bitsnbytes »

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

FWIW I see it the same and have had the same experience.

The tech is there now to be utilized as an assistant. However as the end user you better have enough knowledge to understand the solution or answer it spits at you to fact check it.

The biggest danger I see is utilizing the tool for solutions or answers by novices that don't understand enough to grasp the solution and confirm if in fact the answer it gives is correct or appropriate.

Simplistic example: A non techie manager trying to have it create sql queries to get the information they are asking for , is asking for trouble. However in the hands of a sql developer it could provide enough of the query build out to just require minimal tweaks.

Re: ChatGPT: The Slickest Con Artist of All Time

#225

Post by je42 »

Retric wrote: post #198
2devnull wrote: post #173

Earlier quoted context omitted.

You can verify it’s accuracy. The metaphor of “con artist” is a poor one. It has no adversarial interest. A con artist does. Furthermore, con stands for confidence. A con artist manipulates confidence. Gpt and other language models are “what it says on the tin.” Not understanding the fallibility of LLM is not the fault of the technology. It’s the user’s fault. Not at all like a con artist. More like a tool that can b…

The problem is needing to verify accuracy makes it a really bad search engine replacement. A good con artist is generally truthful, they just know how to seem truthful even when they lie which exactly the issue with trusting ChatGPT etc. You can’t tell if the model is telling bullshit unless you already know the truth. This doesn’t make them useless in general, just a poor fit if you can’t easily fact check the resul…

Easy, you use a second source and double check.

In google you would go to another search result.

For ChatGPT you actually need to verify via Google or actually try it when it is code.

Re: ChatGPT: The Slickest Con Artist of All Time

#226

Post by cj »

raldi wrote: post #43
ResearchCode wrote: post #26

Earlier quoted context omitted.

Still waiting on those autonomous cars. Unlike many other software development tools, I've found ChatGPT to be useless so far.

I saw three autonomous cars driving around my neighborhood in just the past week, all with nobody at the wheel.

(Totally unrelated) I wonder how an autonomous car would navigate my narrow dead end road. The road isn't wide enough for 2 cars to pass, so whenever 2 cars pass each other on the road, 1 car needs to pull off onto the shoulder to make room.

I have a feeling an autonomous car would take up the whole lane, and not be smart enough to pull off onto the shoulder. Meaning both cars would be stuck there unable to pass each other.

I really hope these "nobody behind the wheel" cars at least have someone monitoring / controlling them remotely when they get into confusing situations.

Re: ChatGPT: The Slickest Con Artist of All Time

#227

Post by danielbln »

MagicMoonlight wrote: post #121
rqtwteye wrote: post #111

Earlier quoted context omitted.

Same for me. I fed it a few requirements and test objectives and its comments were pretty reasonable. With a little specialized training it will probably do better than most systems engineers or testers I know.

Okay so it generated a response which was “reasonable” How do you know it was correct? Because you checked it’s entire output manually and determined it probably wasn’t too wrong? So what happens if you now trust it to write firmware for some difficult old timey hardware that nobody understands anymore. It seems correct. But then it actually was just making it up and the coolant system of the power plant breaks and k…

Are you assuming occasional non-factual output is going to be an issue in the future?

Re: ChatGPT: The Slickest Con Artist of All Time

#228

Post by danielbln »

jwie wrote: post #67
fnordpiglet wrote: post #17

I had a detailed conversion with chatGPT about how to gracefully handle terminating conditions of a rust program. It summarized cogently to register at_exit()’s for each thread, panic handlers, and register signal handlers. It advised and explain in detail on my query about the thread handling for each of these variants, gave really helpful advice on collecting join handles in a closure on the main thread and waiting…

It’s only as good as the information it harvested. In a space where less true information is available, the results would be quite poor.

There are already ways of dynamically populating the context of the model with information retrieves as text embeddings from a vector store, based on the initial prompt. This way you can feed the model current and correct information which it will weave into its output

Re: ChatGPT: The Slickest Con Artist of All Time

#229

Post by tester457 »

kibwen wrote: post #110
tester457 wrote: post #95

Earlier quoted context omitted.

For the api appending such language is a waste of tokens, therefore a waste of money. People should use it while knowing it hallucinates all the time.

In other words, everyone should assume that it's lying at all times. Nobody should be using a product that both lies to you and then lies to you about the fact that it's lying to you.

It's not lying about the fact that it lies, they have a disclaimer.

It's the user's responsibility to verify truth. This was trained on the internet and everyone knows not to believe everything you see on the internet. This isn't any different.

Post reply on HN