Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Don Knuth Plays with ChatGPT” but with ChatGPT-4

gist.github.com

Post reply on HN
Print view View original

71–80 of 141 posts

Re: “Don Knuth Plays with ChatGPT” but with ChatGPT-4

#71

Post by jameshart »

Worth noting also that, while asking Bing chat to "Tell me what Donald Knuth says to Stephen Wolfram about chatGPT" doesn't (yet) produce exactly the right result, it produced the following answer when asked what Donald Knuth says about chatGPT:

> Donald Knuth, a computer scientist and mathematician known for his contributions to the field of computer programming, particularly in the area of algorithms and data structures, has expressed some skepticism about the potential of artificial intelligence to achieve true human-level intelligence and creativity[1]. He once conducted an experiment with chatGPT where he posed 20 questions to it and analyzed its responses[1]. Is there anything specific you would like to know about his views on GPT?

With [1] being a citation link to https://cs.stanford.edu/~knuth/chatGPT20.txt

Re: “Don Knuth Plays with ChatGPT” but with ChatGPT-4

#72

Post by coldtea »

thfuran wrote: post #52
nearbuy wrote: post #36

Earlier quoted context omitted.

Still fairly impressive. Probably better than most people could do if given 60 seconds, but probably worse than most people if given 10 minutes.

I would rate a person who provides no sentence at all as performing significantly better, and I suspect most people could pretty quickly come up with something.

>I would rate a person who provides no sentence at all as performing significantly better

The logic failure in the above statement is probably worse than the logic failure of not being able to spontaneously compose a phrase with just 5-letter words - and slipping in one or two with a higher word-count.

>I suspect most people could pretty quickly come up with something

You'd be very surprised then. Most people fail at even more basic tasks.

Heck, most candidate programmers fail at fizz-buzz (not that more difficult than the above)

Re: “Don Knuth Plays with ChatGPT” but with ChatGPT-4

#73

Post by fnordpiglet »

What I find amazing about the original exchange was the profound lack of curiosity Knuth demonstrated. Because the model wasn’t flawless in performance he pinned it as a curiosity that was good at grammar and vacuous otherwise and wasn’t interested to hear how it improves. This reminds me of an awful lot of the computing field in this drama as it plays out. People that literally know how implausible any of these feats have been using traditional approaches immediately discount the entire thing the moment it hallucinates - and it feels like the more deterministic the bent of the person the more absolutely dismissive they are of what’s transpiring in front of us.

These models are doing feats that are stupendous and impossible before their advent. Not just a little bit, but the capability differences are so vast that it’s perhaps not even recognizable by people as being as vast as it is. I am impressed that Wolfram seems to have immediately grasped its significance and is running with it.

The fact this gist demonstrates essentially every single flaw was addressed. But that Knuth apparently doesn’t know / care months after GPT4’s introduction is demonstrative of a different type of personality.

I know which I aspire to be.

Re: “Don Knuth Plays with ChatGPT” but with ChatGPT-4

#74

Post by 11101010001100 »

qup wrote: post #53
bacon_waffle wrote: post #43

Earlier quoted context omitted.

> the only bad thing is that it's hallucinating an answer instead of admitting it doesn't know. Isn't this a fundamental issue?

Only if it can't be corrected. How do you rate the likelihood of this problem being unsolvable?

Ask ChatGPT.

Re: “Don Knuth Plays with ChatGPT” but with ChatGPT-4

#75

Post by harshreality »

nttl wrote: post #40
underdeserver wrote: post #3

Interesting that it didn't get the 5-letter word sentence right.

ChatGPT: You didn't say 5-non-repeat-letters, human, jez

Both the first and last words have repeating letters, so they fail under that interpretation too. There would have to be a bizarre interpretation that consecutive-repeating letters are counted as one, but non-consecutive are counted separately, for its response to be considered correct.

An AI aware of how to optimally answer questions put to it would find the least objectionable interpretation when one is a subset of the other. It also failed by not constructing a simpler sentence, like subject-verb-object or subject-verb-adjective-object, since its limitations related to letters and tokens, and its failure to double check its answers before output, mean it can make errors. The more it writes, the more chance it has of making an error.

Re: “Don Knuth Plays with ChatGPT” but with ChatGPT-4

#77

Post by [deleted] »

fnordpiglet wrote: post #73

What I find amazing about the original exchange was the profound lack of curiosity Knuth demonstrated. Because the model wasn’t flawless in performance he pinned it as a curiosity that was good at grammar and vacuous otherwise and wasn’t interested to hear how it improves. This reminds me of an awful lot of the computing field in this drama as it plays out. People that literally know how implausible any of these feat…

[deleted]

Re: “Don Knuth Plays with ChatGPT” but with ChatGPT-4

#78

Post by dkersten »

Rebelgecko wrote: post #46
bee_rider wrote: post #10

Earlier quoted context omitted.

That is a weird way of spelling mergesort.

Sleepsort is the most elegant & efficient sorting algorithm

Sleepsort just pushes the sorting task to the task scheduler, which uses sone other algorithm to do the sorting.

Re: “Don Knuth Plays with ChatGPT” but with ChatGPT-4

#80

Post by thfuran »

nearbuy wrote: post #62
thfuran wrote: post #52

Earlier quoted context omitted.

I would rate a person who provides no sentence at all as performing significantly better, and I suspect most people could pretty quickly come up with something.

> I would rate a person who provides no sentence at all as performing significantly better Why? > I suspect most people could pretty quickly come up with something It only takes 60 seconds to test that on yourself. It's not that easy to come up with something of similar length to ChatGPT's answer that also sounds somewhat natural/sensible.

>Why?

For the same reason that "I don't know" is generally a better response than bullshitting.

>It's not that easy to come up with something of similar length to ChatGPT's answer that also sounds somewhat natural/sensible

Those weren't requirements.

Post reply on HN