ChatGPT: The Slickest Con Artist of All Time • Hacker News Board

Re: ChatGPT: The Slickest Con Artist of All Time

#121

Post by MagicMoonlight » Fri, Feb 03, 2023, 4:56 PM UTC

rqtwteye wrote: post #111

fnordpiglet wrote: post #17

I had a detailed conversion with chatGPT about how to gracefully handle terminating conditions of a rust program. It summarized cogently to register at_exit()’s for each thread, panic handlers, and register signal handlers. It advised and explain in detail on my query about the thread handling for each of these variants, gave really helpful advice on collecting join handles in a closure on the main thread and waiting…

Same for me. I fed it a few requirements and test objectives and its comments were pretty reasonable. With a little specialized training it will probably do better than most systems engineers or testers I know.

Okay so it generated a response which was “reasonable”

How do you know it was correct? Because you checked it’s entire output manually and determined it probably wasn’t too wrong?

So what happens if you now trust it to write firmware for some difficult old timey hardware that nobody understands anymore. It seems correct. But then it actually was just making it up and the coolant system of the power plant breaks and kills 20,000 people.

Re: ChatGPT: The Slickest Con Artist of All Time

#122

Post by koprulusector » Fri, Feb 03, 2023, 4:56 PM UTC

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

What people also seem to forget (if they knew in the first place) is how bad GPT3 was compared to GPT4/ChatGPT. I subscribe to SubSimulatorGPT3 on Reddit and it generates tons of text that looks and feels like English, for example, but is devoid of any real meaning. Every sentence, every paragraph I read from GPT3 has me questioning whether or not I’ve had a stroke.

Now, compare that to your comment, and it’s INSANE how much the capability (and the conversation) has shifted.

GPT3 was what, three or four years ago? What will we be debating, what will the capabilities be, in another three or four years?

Re: ChatGPT: The Slickest Con Artist of All Time

#123

Post by logifail » Fri, Feb 03, 2023, 4:56 PM UTC

VBprogrammer wrote: post #60

enragedcacti wrote: post #50

Earlier quoted context omitted.

> Despite being the type of person who'd be interested in that kind of thing I thought it was not generally well understood Its an extremely well understood piece of technology and has been for decades outside of Toyota. Here is a 2005 paper that accurately models the fundamental behavior of the Prius HSD [1]. There are dozens of pages that explain it in various detail, including ones with live simulators of the powe…

I think you and I have different definitions of not generally well understood. I would be impressed if more than 1 in 1000 people could describe it to the level of "it's a planetary gearbox with 2 electric motors used as inputs and outputs to control the effective gear ratio".

> I would be impressed if more than 1 in 1000 people could describe it

Q: What use is it attempting to measure ChatGPT's capabilities on something vs the average person on the street, or vs the average user with access to google.com? What does that actually tell us?

Maybe I'm showing my age, but if one wanted to learn or discover something, wouldn't one first typically try and find a reliable source/expert in that field or subject?

Re: ChatGPT: The Slickest Con Artist of All Time

#124

Post by lukebitts » Fri, Feb 03, 2023, 4:56 PM UTC

ResearchCode wrote: post #26

ajuc wrote: post #7

"Cars won't replace horses, because they require roads and horses don't."

Still waiting on those autonomous cars. Unlike many other software development tools, I've found ChatGPT to be useless so far.

What have you tried? I personally have used it for writing tests (and ideas for tests), to explain some old convoluted legacy code, to generate easy-concept functions like rect/rect collisions, rect/curve collisions, turning an in memory tree into html text, etc. All things I could do in 5 to 10 minutes, but ChatGPT does it in 5 seconds.

I've also used it to generate the floorplan of several commercial buildings, the jobs needed to keep those buildings operating, people to fill those jobs with varying ages/names/appearances and the relationships between all these people (only here things started breaking down). All this as background for a game I'm developing, this task would have taken significantly more time for me to do manually.

Re: ChatGPT: The Slickest Con Artist of All Time

#125

Post by ilaksh » Fri, Feb 03, 2023, 4:56 PM UTC

lostoldsession wrote: post #14

Part of me thinks one of the big reasons Google has held back so much is because of ethical concerns and/or just general fear of not having complete knowledge of how AI (incomplete to boot) will impact the world. We know that Google has some extremely powerful AI, but they never let it out of the lab. Just the most heavily neutered and clamped versions to help accentuate their existing products. Now it seems that Ope…

It's because Google has this exact same problem with their AI models. Also they would probably have to double their compute capacity if a billion of their customers started using it. (My made up numbers). It uses hundreds of GB of GPU RAM during the inference. I am guessing they don't have enough GPUs to do that and still have a Google Cloud.

It's different from OpenAI because of the existing user base is like a billion users or something.

Re: ChatGPT: The Slickest Con Artist of All Time

#126

Post by zdw » Fri, Feb 03, 2023, 4:58 PM UTC

soperj wrote: post #66

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well This is the issue. It's more like the dumbest guy in the room confidently convincing the everyone else that they know what they're talking about.

We've entered the Dunning–Kruger effect phase of the AI revolution.

Re: ChatGPT: The Slickest Con Artist of All Time

#127

Post by MagicMoonlight » Fri, Feb 03, 2023, 4:58 PM UTC

hospadar wrote: post #112

I wonder if the biggest shortcoming of GPT right now is not that it sometimes gets things wrong, or can't cite its sources, or whatever - maybe it needs to learn when to say "I don't know the answer to that question". That's a pretty hard thing for most humans (and myself) to learn to say, and I suspect GPT's training data (tha internet) doesn't include a lot of "I'm not sure" language and probably does include a lot…

It can’t learn because it isn’t intelligent. It’s a markov chain.

There is no way for it to make any decision at all. The fundamental system design is that it guesses the word that is most likely to be next. That’s it. There’s no concepts behind whatever it outputs, it has no idea what it means.

Re: ChatGPT: The Slickest Con Artist of All Time

#128

Post by Throwaway78389 » Fri, Feb 03, 2023, 4:58 PM UTC

pdonis wrote: post #78

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No. You've fallen for exactly the con the article is describing. ChatGPT knows nothing . It is very good at generating text that seems plausible if you know nothing about the subject matter, but is not reliable as an actual source of information . If you happened t…

How many hours you spent trying ChatGPT out? I spent at least high tens, maybe even hundreds. You're absolutely wrong. Yes, it hallucinates, yes its wrong about obscure topics - but calling having success with it luck is absolutely wrong. It's very consistently good. Especially about things like programming, physics, math - and now I'm using it as my teaching assistant for my pilot training, it's perfect (and I can very simply verify the answers are good with my FAA Pilot Handbook, don't worry).

Re: ChatGPT: The Slickest Con Artist of All Time

#129

Post by enragedcacti » Fri, Feb 03, 2023, 4:58 PM UTC

VBprogrammer wrote: post #60

enragedcacti wrote: post #50

Earlier quoted context omitted.

> Despite being the type of person who'd be interested in that kind of thing I thought it was not generally well understood Its an extremely well understood piece of technology and has been for decades outside of Toyota. Here is a 2005 paper that accurately models the fundamental behavior of the Prius HSD [1]. There are dozens of pages that explain it in various detail, including ones with live simulators of the powe…

I think you and I have different definitions of not generally well understood. I would be impressed if more than 1 in 1000 people could describe it to the level of "it's a planetary gearbox with 2 electric motors used as inputs and outputs to control the effective gear ratio".

ahh, I misunderstood what you meant. In that case can you explain to me why its meaningful that chatGPT can sometimes explain things for which there are tons of resources online to learn about them but that a lot of people aren't familiar with? I legitimately just don't understand why that is an interesting bar to clear, especially given that it can also unpredictably lie about some aspects of the thing its explaining.

It would perhaps be interesting if it were such a hard topic that only 1 in 1000 could be capable of understanding it, but I would hazard a guess that most people familiar with the prius who don't understand its drivetrain probably don't understand it because they don't care, not because its too complicated to understand.

Re: ChatGPT: The Slickest Con Artist of All Time

#130

Post by Retric » Fri, Feb 03, 2023, 4:59 PM UTC

darkwater wrote: post #97

swatcoder wrote: post #86

Earlier quoted context omitted.

That's exactly the wrong sort of thing to turn to ChatGPT for. In probing it for more detail, you're giving it increasingly helpful hints on how to answer your questions in a way that makes you satisfied, regardless of accuracy. That's what it's optimized to do. And because you're working from a humble and curious understanding of the topic yourself, it can and will make up all sorts of things that make you believe i…

GP stated that they asked ChatGPT about a topic they already were familiar with, from other sources. And that the answer made sense.

ChatGPT is optimized to seem to make sense, it’s like talking to a world class con artist you won’t notice it’s talking nonsense. Thus is subtly dangerous in ways that aren’t obvious in the beginning.

Sure it seems great at writing boilerplate code, unfortunately it’s really good and writing the kind of bugs that you don’t notice. This is maddening in a completely novel way, like working with an actively malicious autocomplete.