Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

news.ycombinator.com

Post reply on HN
Print view View original

91–100 of 118 posts

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#91

Post by throw310822 »

qwytw wrote: post #66
throw310822 wrote: post #36

Have any of you had experience interacting with a person with Down's syndrome? They might have the same troubles giving correct answers even to simple question and need guidance to answer them correctly. Would you say that they don't possess "general intelligence"? Of course not. Being unable to perform some tasks that seem trivial to us does not, per se, prove a lack of general intelligence.

Are you implying that ChatGPT possesses "general intelligence"? It's very advanced statistical model/search engine. It can be pretty good at imitating humans and provided answers bases on a huge corpus of content it was trained on. However that's not really that close to human intelligence. I mean it's both several magnitudes more 'intelligent' than humans in some ways and also more stupid in different ways.

> Are you implying that ChatGPT possesses "general intelligence"?

I am saying that it well could. We should probably agree on a definition of "general intelligence" though.

> It's very advanced statistical model/search engine.

That is just how the sausage is made. Of course there is no magic or soul or life spark in it- if that's needed to make AGI, then AGI is impossible.

> I mean it's both several magnitudes more 'intelligent' than humans in some ways and also more stupid in different ways.

Yes, definitely. But again, do you define AGI as average human equivalent intelligence? Shouldn't that deserve a different label (Average-Human intelligence maybe)?

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#92

Post by BiteCode_dev »

qwytw wrote: post #62
BiteCode_dev wrote: post #58

People are starting to entropomorphise gpt3, stating it's bullshitting or lying, instead of realizing this is just a tool that is useful for some things and not for others. The fact it talks makes people treats it differently than Excel. But when Excel turns your date into a nonsensical value, people understand it's just a program with limitations

I guess the main concern that it's often confidently unreliable. i.e. it's answers sort of make sense most of the time, yet when they are not it's hard to spot. IMHO in most cases where it's difficult to validate its answers or exactness is required this makes more than useless, maybe even harmful.

My issue here is with the word "confident". It's a program. It's as confident as Excel is with date parsing, meaning it's completely neutral about it. It just does its thing.

We attribute this "confident" adjective to the result, not because it outputs a confidence interval, but because it uses language and we associate that with human traits.

Even technical people get tricked, I read and heard some very emotional responses to what it outputs. Sometime people got angry that it was "not honest" or "trying to weasel out of the answer". Personally, I caught myself several times trying to prove it wrong.

That's just missing the point of what it is.

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#93

Post by Fnoord »

qwytw wrote: post #62
BiteCode_dev wrote: post #58

People are starting to entropomorphise gpt3, stating it's bullshitting or lying, instead of realizing this is just a tool that is useful for some things and not for others. The fact it talks makes people treats it differently than Excel. But when Excel turns your date into a nonsensical value, people understand it's just a program with limitations

I guess the main concern that it's often confidently unreliable. i.e. it's answers sort of make sense most of the time, yet when they are not it's hard to spot. IMHO in most cases where it's difficult to validate its answers or exactness is required this makes more than useless, maybe even harmful.

I suppose it would be akin to a difficult to spot false positive where the person who decided (say, customs) doubles down and you end up having to argue your case.

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#94

Post by fulafel »

gadtfly wrote: post #20

This is an artifact of an implementation-specific trick that trades performance at character-level tasks for performance at everything else. It does not reflect anything inherent about this type of model's capabilities: https://www.gwern.net/GPT-3#bpes GPT-3 does not see individual characters. It sees "djsjcnnrjfkalcr" chunked as [d, js, jc, nn, r, j, f, k, al, cr]. You can see for yourself here: https://beta.openai.…

Nitpick: Gwern proposes this as a "plausible explanation" and does not make a definitive claim.

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#95

Post by lmc »

dns_snek wrote: post #79
lmc wrote: post #38

Me: How many characters are in the string "sdfyskkgagag"? ChatGPT: The string "sdfyskkgagag" has 14 characters. Me: That is incorrect, please try again ChatGPT: I apologize for the mistake. The string "sdfyskkgagag" has 12 characters. To me, that sounds like it has some understanding. Fallible and weird, but it's there.

I tried this for myself and the results were mildly amusing. Me: How many characters are in the string "sdfyskkgagag"? ChatGPT: There are 13 characters in the string "sdfyskkgagag". Me: That is incorrect, please try again ChatGPT: I apologize if my previous answer was incorrect. There are indeed 13 characters in the string "sdfyskkgagag". Me: That is incorrect, please try again ChatGPT: I apologize for the mistake. T…

Haha, okay, yes :-)

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#96

Post by ErrantX »

benj111 wrote: post #26
ErrantX wrote: post #14

Yeh so this and other examples for me demonstrate that ChatGPT has no logic per se. These examples demonstrate that it is not analysing your question and calculating the answer, instead it is approximating and answer based on a corpus of questions. So in essence it doesn't actually yet understand what you are asking it. I'd be intrigued to see some kind of amalgamation of an AI tool like ChatGPT with a logic tool (sa…

>would combat a lot of the ways folks have found to break this. Isn't it more that chatgpt is broken, and users are pointing it out I suppose it depends what chatgpt is for, but I assume the end goal isn't just 'chat' (god I hope not) so it does actually need to reliably know correct answers or say when it doesn't. BSing is the worst option.

I mean arguably we BS all the time; how many pub quizzes do people confidently assert an answer? So I don't think that's totally the issue.

It's slightly different to the point I was talking to; but the thing about ChatGPT I find "uncanny" right now is how formal and expert it sounds about everything. A person would vary in certainty/surety and would sometimes show their working out etc.

So again it's a demonstration of how AI currently works; a best fit to the form and words of the question rather than genuine determination.

(I am not at all an AI expert but my understanding is that essentially all of this is not an engineering problem but a compute scale problem to train a large enough corpus - but someone expert please correct me!)

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#97

Post by BiteCode_dev »

bambax wrote: post #7

ChatGPT is assertive, authoritative, and even goes on to lie to justify its answers and "be right". It's a gasslighting machine. Here's an example I posted here a month ago: https://news.ycombinator.com/item?id=33852236

Those are human traits.

ChatGPT is not human, and cannot be more assertive, authoritative or lying than Python.

If you give python "0.1 + 0.3", it will instantly, and without a doubt, output "0.30000000000000004". Is Python arrogantly thinking it's right and misleading you?

Of course not. This last sentence makes no sense, Python is just a program that does what it's programmed to do.

But because ChatGPT is using natural language as a UI, it triggers a very irrational response in humans interacting with it.

We are starting to anthropomorphize it, use adjective like "assertive", "authoritative", and "lie" to label its output, which is like calling Covid cruel and vicious because it kills old people, or a NPC lazy because it does not work to pay for its food.

There is no intent of ChatGPT to deceive or justify its supposed lies. There is only a program that does what's its programmed to do, and shows its limitations.

What do you expect, that is says "I think it's x but I can be wrong with a confidence interval of z" with every answer? Do you expect google to tell that when it outputs the results?

GPT is not talking to you. It's an output. It's not confident. It's a program printing text matching statistical analysis.

It's worrying that such a simple program is already close enough to the real thing that people, even on HN, are already attributing it character.

It reminds me of this scene from community with the pen named Steve: https://www.youtube.com/watch?v=z906aLyP5fg

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#99

Post by deely3 »

zelphirkalt wrote: post #46
jacquesm wrote: post #43

The only thing missing now is a feedback loop that causes it to make such mistakes only once. That can't be that hard to add.

How many times does it need to be reported to gain sufficient weight to make it switch its answer?

How can we be sure that such thing is really fixed? I mean, how can we be sure that it calculates lenght of string correctly every time?

Re: ChatGPT: How many letters has the string “djsjcnnrjfkalcr”?

#100

Post by qwytw »

throw310822 wrote: post #91
qwytw wrote: post #66

Earlier quoted context omitted.

Are you implying that ChatGPT possesses "general intelligence"? It's very advanced statistical model/search engine. It can be pretty good at imitating humans and provided answers bases on a huge corpus of content it was trained on. However that's not really that close to human intelligence. I mean it's both several magnitudes more 'intelligent' than humans in some ways and also more stupid in different ways.

> Are you implying that ChatGPT possesses "general intelligence"? I am saying that it well could. We should probably agree on a definition of "general intelligence" though. > It's very advanced statistical model/search engine. That is just how the sausage is made. Of course there is no magic or soul or life spark in it- if that's needed to make AGI, then AGI is impossible. > I mean it's both several magnitudes more '…

I Sure, I can agree that it depending on how you defined intelligence it can be considered intelligent in a way.

> Of course there is no magic or soul or life spark in it- if that's needed to make AGI

I don't know. I mean maybe AGI is possible, but if the goal is to imitate humans rather than some other more practical purpose it would probably have to be very different from ChatGPT.

Trying to compare it to average human intelligence is not even apples to oranges in my opinion. It's something completely different yet superficially similar to the way how humans think (due to obvious reasons)

Post reply on HN