Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT: The Slickest Con Artist of All Time

tedgioia.substack.com

Post reply on HN
Print view View original

231–240 of 444 posts

Re: ChatGPT: The Slickest Con Artist of All Time

#231

Post by damiankennedy »

CodeWriter23 wrote: post #162

This my personal opinion and may be entirely worthless. The quality of answers I read in all of the examples posted in that article read like the questions were routed to an offshore boiler room where the answers were crafted by humans. Like some modern day Mechanical Turk. Especially in the 6 eggs example, there is a complete discontinuity of thought across the answers, isn't this within a single session with the AI…

I think you have 4 eggs left.

You broke 2 which means you cracked the shell and got the part you cook out. You fried those eggs and ate them. You still have 4 eggs.

The trick is the word "broke". Normally if you break something you can not use it and discard it but in egg folklore breaking an egg is step 1 in a subset of egg cooking procedures.

Re: ChatGPT: The Slickest Con Artist of All Time

#232

Post by jacobjjacob »

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

The latter group exists to counter the former. If folks weren’t making some of these over-the-top claims, then there wouldn’t be the claims to debunk.

Re: ChatGPT: The Slickest Con Artist of All Time

#233

Post by damiankennedy »

_nalply wrote: post #141
shaan7 wrote: post #59

Earlier quoted context omitted.

> When this happens it spews confident bullshit and when I confront it, it cheerily admits that it was wrong, but can continue to produce further bullshit. Yeah, I can't express the "aha!" moment when I observed this for the first time. Then I laughed so hard when it made very serious attempts at convincing me that it was indeed correct xD (For the curious, I asked it about romantic relationships between characters f…

Yeah, ChatGPT can be extremely funny. Today I got it to use taboo words like period, panty liner, tampon and smell. I told it to write a dialog where someone is very direct and sincere and not to use euphemisms. I am still smiling about it. I am trying to find out where its limits are and how they express themselves. Me: You have the task to write a short dialog with the participants Axel and Bianca. Both are good fr…

launching chatHR.exe

Re: ChatGPT: The Slickest Con Artist of All Time

#234

Post by rqtwteye »

MagicMoonlight wrote: post #121
rqtwteye wrote: post #111

Earlier quoted context omitted.

Same for me. I fed it a few requirements and test objectives and its comments were pretty reasonable. With a little specialized training it will probably do better than most systems engineers or testers I know.

Okay so it generated a response which was “reasonable” How do you know it was correct? Because you checked it’s entire output manually and determined it probably wasn’t too wrong? So what happens if you now trust it to write firmware for some difficult old timey hardware that nobody understands anymore. It seems correct. But then it actually was just making it up and the coolant system of the power plant breaks and k…

You certainly have to validate the output but I am pretty sure not too far in the future AI will be able to do a better job than humans.

Re: ChatGPT: The Slickest Con Artist of All Time

#235

Post by choppaface »

_huayra_ wrote: post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

What’s worse, too much hype, or the non-believers?

Well the OpenAI hype squad is racking up personal fortunes from their work in a finding deal that is pretty clearly over-valued. Especially versus what Deepmind sold for [1].

While perhaps both sides are not being honest, YC’s own sama is clearly profiting excessively over the hubub. He’s clearly proving to be a symbol of the faction of YC that rakes in money through arbitrage not prolonged value.

[1] https://techcrunch.com/2014/01/26/google-deepmind/

Re: ChatGPT: The Slickest Con Artist of All Time

#236

Post by naniwaduni »

yellow_postit wrote: post #106
swatcoder wrote: post #86

Earlier quoted context omitted.

That's exactly the wrong sort of thing to turn to ChatGPT for. In probing it for more detail, you're giving it increasingly helpful hints on how to answer your questions in a way that makes you satisfied, regardless of accuracy. That's what it's optimized to do. And because you're working from a humble and curious understanding of the topic yourself, it can and will make up all sorts of things that make you believe i…

To OP’s point what’s with the gate keeping on “right” or “wrong” ways to use it? It’s a tool. It has flaws. It can be useful and if it is, have at it.

It's a tool whose flaw is that sometimes it secretly does the exact opposite of what you're trying to do with it in a way that's only detectable if you didn't already need it.

It's like Babelfish, but for monoglots.

Re: ChatGPT: The Slickest Con Artist of All Time

#237

Post by CodeWriter23 »

damiankennedy wrote: post #231
CodeWriter23 wrote: post #162

This my personal opinion and may be entirely worthless. The quality of answers I read in all of the examples posted in that article read like the questions were routed to an offshore boiler room where the answers were crafted by humans. Like some modern day Mechanical Turk. Especially in the 6 eggs example, there is a complete discontinuity of thought across the answers, isn't this within a single session with the AI…

I think you have 4 eggs left. You broke 2 which means you cracked the shell and got the part you cook out. You fried those eggs and ate them. You still have 4 eggs. The trick is the word "broke". Normally if you break something you can not use it and discard it but in egg folklore breaking an egg is step 1 in a subset of egg cooking procedures.

Touché There is a range to “break”. You may be correct.

Re: ChatGPT: The Slickest Con Artist of All Time

#238

Post by fnordpiglet »

skybrian wrote: post #134
fnordpiglet wrote: post #17

I had a detailed conversion with chatGPT about how to gracefully handle terminating conditions of a rust program. It summarized cogently to register at_exit()’s for each thread, panic handlers, and register signal handlers. It advised and explain in detail on my query about the thread handling for each of these variants, gave really helpful advice on collecting join handles in a closure on the main thread and waiting…

When I got out of school I went to work for Oracle and decided to read the database manuals, even though I wasn't working on database stuff. When I left I read other database manuals and was able to help people code database stuff, not because I was a real database expert, but because I had read the manuals and they hadn't. I think something similar is going on here? For some subjects and some questions, ChatGPT acts…

I was actually literally thinking about that as I worked on my solo project. This is like a coworker I can ask questions of and bounce ideas off of. It actually gives some insightful ideas. I was working on a file rotation scheme and it suggested naming the files uuid__ with the intent that resolution conflicts won’t need to be walked very often if ever optimizing the common case of file.0, file.1, etc. This isn’t like earth shattering or something I wouldn’t have thought of on my own, but it was something I would have a convo about and someone would suggest it and I would pause and consider and find value in it. This is beyond a reference material discussion, this is bouncing architectural ideas off of someone who knows their shit but occasionally hallucinates. That’s generally what I look for in coworkers and employees anyways.

Note: I’m aware UUIDs can contain the time but the intent was the UUID was invariant in the process and it would move to another file once it got to a certain size.

Re: ChatGPT: The Slickest Con Artist of All Time

#239

Post by naniwaduni »

hansvm wrote: post #188
swatcoder wrote: post #86

Earlier quoted context omitted.

That's exactly the wrong sort of thing to turn to ChatGPT for. In probing it for more detail, you're giving it increasingly helpful hints on how to answer your questions in a way that makes you satisfied, regardless of accuracy. That's what it's optimized to do. And because you're working from a humble and curious understanding of the topic yourself, it can and will make up all sorts of things that make you believe i…

If you're able to validate the results though (so they logically fit together, or with that newfound vocabulary and pseudounderstanding can you search for that information elsewhere), is that really such a bad thing? I've been doing exactly that to quickly ramp up on some new numerical algorithms I previously didn't know existed.

That's a really big if.

Re: ChatGPT: The Slickest Con Artist of All Time

#240

Post by hn_throwaway_99 »

pdonis wrote: post #192
Throwaway78389 wrote: post #128

Earlier quoted context omitted.

How many hours you spent trying ChatGPT out? I spent at least high tens, maybe even hundreds. You're absolutely wrong. Yes, it hallucinates, yes its wrong about obscure topics - but calling having success with it luck is absolutely wrong. It's very consistently good. Especially about things like programming, physics, math - and now I'm using it as my teaching assistant for my pilot training, it's perfect (and I can v…

> I can very simply verify the answers are good with my FAA Pilot Handbook Thank you for agreeing with my point. If you need to check the answers with your FAA Pilot Handbook, wouldn't it be simpler just to read the FAA Pilot Handbook? That handbook, unlike the text generated by ChatGPT, was written by a process that was aware of the semantic relationship between the text being written and the things in the world tha…

> If you need to check the answers with your FAA Pilot Handbook, wouldn't it be simpler just to read the FAA Pilot Handbook?

No, absolutely not. It's much easier to verify a couple specific pieces of information that you're unsure of then go hunting through a large corpus of information trying to pick out the bits that are specific to what you want.

I've used ChatGPT across lots of different areas and I find it incredibly useful. I'm not blindly trusting what it spits out, but it's pretty simple to verify what it's saying. While I definitely do have concerns about the impacts of ChatGPT on a societal level, and what will happen when so much computer-generated content can flood the Internet, but, at a personal level, the complaint that ChatGPT "bullshits with confidence" is not really that much of a problem for what I use it for.

Edit: To give a specific, real-world example, there was a post recently about using ChatGPT to replace a SQL Analyst. Now, ChatGPT definitely will and did churn out wrong answers, but it was incredibly useful to use as a starting point for some complex queries. When it fails, it tended to fail in pretty obvious ways, and to the complaint that it can fail in more subtle ways that look correct, I've certainly dealt with tons of human-generated queries that had the same issues. Are those all useless?

Post reply on HN