Live data from Hacker News

ChatGPT: The Slickest Con Artist of All Time

tedgioia.substack.com

51–60 of 444 posts

Re: ChatGPT: The Slickest Con Artist of All Time

#51
post #9
post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

I don't know that it's an "incredibly smart" anything; I think it'd be better described as incredibly well-read, with poor discretion. But there's definitely uses for a tool that produces a lot of boilerplate content that a human can then proofread (ex: writing cover letters, filling out unstructured paperwork, etc.) Edit: I personally underestimated the use-case, because the "more optimal" thing to do when auto-fill…

> I don't know that it's an "incredibly smart" anything; I think it'd be better described as incredibly well-read, with poor discretion.

But likely much better read and with much better discretion that most of people writing content these days.

Which I think is part of the point that people are missing. ChatGPT’s output might not be as good as the best writing, but most of what people read is mediocre content with questionable accuracy. Being better than the majority of the writing being consumed is still a pretty big deal, even if it’s not better than all of the writing that’s being consumed. Especially since we can expect ChatGPT to get better.

The same is true for AI art. Could you hire a trained artist and work with them to do a better job? Probably, but most people don’t have the time and money to pursue such a project.

There are still trained artisans producing high quality physical goods that you can place an order with. But the masses have greatly benefited from the factories that have produced lower quality products that are much more widely available, and available at a moments notice. It’s not wrong to point out when the mass produced stuff might not have the same quality as the products produced. But if you only focus on this aspect you miss the reason why mass production is important.

Re: ChatGPT: The Slickest Con Artist of All Time

#52
post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. Does this "person" make mistakes? Can't cite their sources? Yeah this definitely happens (especially the sources thing), but when you're trying to understand something new and complex and you can't get the "gist" of it, ChatGPT does surprisingly well. if you don't…

By having a conversation with it. Ask probing questions. Verify later with wikipedia.

For instance I asked it about space time metrics, and I learned about the varying types and in what circumstances each are used. Ask it about their mathematical properties and it gives a succinct explanation contrasting the different approaches. Certainly you can see the value now of conversational style pedagogy.

Think of it like learning about things from mates at the bar. In the thick of it there may be bits of inaccuracy, but you'll certainly know more about a topic than before. Then you can verify bits of things later in papers or books, etc.

Re: ChatGPT: The Slickest Con Artist of All Time

#53
post #23
post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

> Can't cite their sources? Has there been research on neural nets who could cite their sources? In common cases, thousands or millions of items in the training set have contributed to the weights in the pathways that activate for a certain input. But maybe for some inputs, only a small number of items in the training set contributed strongly to the weights on the pathways the are decisive to the output? Could you de…

I would imagine that citing sources would look more like finding sources that have a minimum distance to the output in a latent space representation? I don’t think it’ll make sense to track how each connection weight changed over time due to various inputs.

Re: ChatGPT: The Slickest Con Artist of All Time

#54
post #19

Earlier quoted context omitted.

That goes for pretty much any resource; there's a lot of botched or sub-optimal explanations of complex things out on the internet. But being able to have a conversation with someone who has a decent conceptual grasp on something is better than a perfect encyclopedic article alone, and ChatGPT is able to roughly do that for a lot of subjects. As for how you know it's accurate, you don't really have to know it's accur…

> That goes for pretty much any resource; there's a lot of botched or sub-optimal explanations of complex things out on the internet. But being able to have a conversation with someone who has a decent conceptual grasp on something is better than a perfect encyclopedic article alone, and ChatGPT is able to roughly do that for a lot of subjects. people keep saying this, but it isn't true. if you ignore sites like this…

Clearly you are of the opinion that ChatGPT is useless, there are superior resources already available for any topic, and it's all just hype.

Well, then we don't need to argue this since the problem will elegantly solve itself if that's true.

I disagree though. For a lot of things it feels like I can get much better answers than Google, especially when it comes to somewhat conceptual questions.

(Also, I don't use news aggregator comments to learn things unless it's the only possible source. But if you think every blog post or YouTube video that got things wrong has a detailed errata, you'd be sorely wrong. It's so uncommon on YouTube that the practice is usually commended when noticed.)

Re: ChatGPT: The Slickest Con Artist of All Time

#55
post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

[deleted]

Re: ChatGPT: The Slickest Con Artist of All Time

#56
post #47

Usefulness is the correct measure. ChatGPT is limited, but immediately very useful in a surprising number of ways. Compare that to the Bitcoin hype, where, even though it has had years, is still mainly useful for drug transactions and other illegal transfers.

We are completely off topic at this point but it is worth mentioning that due to the pseudoanonymous nature of bitcoin and ethereum, they are actually quite bad for those things and tokens such an Monero, ZCash, Grin, Beam, Mina and many others are much better suited to illegal transfers.

Also worth mentioning, what is illegal strongly depends on jurisdiction and can sometimes be moral despite their legal status.

Re: ChatGPT: The Slickest Con Artist of All Time

#57
post #4

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philos…

I use ChatGPT not because it is so great at solving tricky technical question, but because Google has become so incredibly shitty at it. Google's response these days is ten YouTube videos and a random mix of web posts dating between 2005 and 2023 for a piece of software that I'm using the latest version of. I honestly don't even bother trying anymore.

ChatGPT is a mixed bag. Sometimes it nails it, but I've also seen it hallucinate complete nonsense solutions, for instance in Davinci Fusion using a node type that doesn't even exist in the program.

As for "incredibly intelligent philosopher", I don't even find it an interesting "person" to talk to. Its responses are pedestrian, pedantic and boring. A question like "if God is good then why evil in the world?" is met with a summary of the wikipedia article on the problem of evil. I can read a Wikipedia article, thank you.

Re: ChatGPT: The Slickest Con Artist of All Time

#58
I would greatly appreciate a moratorium on this genre of article until there is compelling accompanying evidence that a meaningful portion of ChatGPT's users are unaware of these shortcomings. I have yet to encounter or even hear of a non-technical person playing around with ChatGPT without stumbling into the type of confidently-stated absurdities and half-truths displayed in this article, and embracing that as a limitation of the tool.

It seems to me that the overwhelming majority of people working with ChatGPT are aware of the "con" described in this article -- even if they view it as a black box, like Google, and lack a top-level understanding of how an LLM works. Far greater misperceptions around ChatGPT prevail than the idea that it is an infallible source of knowledge.

I'm in my 30s, so I remember the very early days of Wikipedia and the crisis of epistemology it seemed to present. Can you really trust an encyclopedia anyone can edit? Well, yes and no -- it's a bit like a traditional encyclopedia in that way. The key point to observe is that two decades on, we're still using it, a lot, and the trite observation that it "could be wrong" has had next to no bearing on its social utility. Nor have repeated observations to that effect tended to generate much intellectually stimulating conversation.

So yeah, ChatGPT gets stuff wrong. That's the least interesting part of the story.

Re: ChatGPT: The Slickest Con Artist of All Time

#59
post #30

Too harsh! ChatGPT is lossily compressed knowledge of humanity collected on the Internet. And it can talk! That's extremely new for us poor hoomans and so we get extremely excited. I found out, it gets about one in ten things wrong. When this happens it spews confident bullshit and when I confront it, it cheerily admits that it was wrong, but can continue to produce further bullshit. I understand the comparison to a…

> When this happens it spews confident bullshit and when I confront it, it cheerily admits that it was wrong, but can continue to produce further bullshit.

Yeah, I can't express the "aha!" moment when I observed this for the first time. Then I laughed so hard when it made very serious attempts at convincing me that it was indeed correct xD (For the curious, I asked it about romantic relationships between characters from two unrelated video games and it said they had one)

Re: ChatGPT: The Slickest Con Artist of All Time

#60

Earlier quoted context omitted.

I dunno, I recently learned about how the Toyota Synergy Drive system works in detail. Despite being the type of person who'd be interested in that kind of thing I thought it was not generally well understood, I've only come across one website which described it in detail. I asked ChatGPT about it and it game me a fairly surface level answer. I then asked it for more details and it went into detail far beyond what th…

> Despite being the type of person who'd be interested in that kind of thing I thought it was not generally well understood Its an extremely well understood piece of technology and has been for decades outside of Toyota. Here is a 2005 paper that accurately models the fundamental behavior of the Prius HSD [1]. There are dozens of pages that explain it in various detail, including ones with live simulators of the powe…

I think you and I have different definitions of not generally well understood. I would be impressed if more than 1 in 1000 people could describe it to the level of "it's a planetary gearbox with 2 electric motors used as inputs and outputs to control the effective gear ratio".
Post reply on HN