Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I quiz ChatGPT about math

billwadge.com

Post reply on HN
Print view View original

91–96 of 96 posts

Re: I quiz ChatGPT about math

#91

Post by jrm4 »

aimor wrote: post #82
jrm4 wrote: post #79

It feels like I've stumped it nearly completely with many variations on the following question: "In binary, do most people have 10 legs?" Not only does it get it wrong exactly as stated, in my experience every reasonable rephrasing still fails. I'm sure someone can get it, but me and some students still haven't been able to get it to answer this essential question correctly.

Had a good laugh with this: "Do most people have 0b10 legs? No, most people do not have 0b10 legs. 0b10 is a binary number that represents the decimal number 2. Humans typically have two legs. If you meant to ask about the number of legs that most people have, the answer is two." It basically said, "No. Yes." So then I tried to get it to say Yes to every question. It took some convincing, but I got there, and this is…

This is exactly my experience. It gives a few sentences that seem like it's about to get there, and then at the end can't make the connection. Strongly supports my ideas about not being too impressed with AI.

Re: I quiz ChatGPT about math

#92

Post by pmarreck »

CrazyStat wrote: post #3

I quizzed ChatGPT about math a few days back. It correctly answers that 1023 is odd, correctly answers that the square of an odd number is also odd, and is about 50/50 on whether it decides 1023^2 is even or odd. The author of the linked post repeatedly refers to ChatGPT's "reasoning". But ChatGPT doesn't reason, it just generates text. Sometimes that text might look like reasoning to a human--it's the sort of text t…

ChatGPT is a liberal arts major.

One day it will "dual major" in math and then develop a determination for when the situation demands rigor (hard calculation) or when it demands "correct word estimation" (its current interaction mode).

Re: I quiz ChatGPT about math

#93

Post by hackerman123469 »

dalmo3 wrote: post #83
SamBam wrote: post #15

I was actually very impressed with some math questions it was able to answer for me: S: If flabb means three, snabb means four, and whabb mean five, what is flabb plus whabb? If flabb means three, snabb means four, and whabb means five, then flabb plus whabb would be equal to three plus five, or eight. In this made-up numbering system, the words “flabb,” “snabb,” and “whabb” represent the numbers three, four, and fiv…

> bbbbbbb It's impressive that gpt was able to abstract the number system, but... There are 7 Bs there.

I caught that too!

Re: I quiz ChatGPT about math

#94

Post by CrazyStat »

pmarreck wrote: post #92
CrazyStat wrote: post #3

I quizzed ChatGPT about math a few days back. It correctly answers that 1023 is odd, correctly answers that the square of an odd number is also odd, and is about 50/50 on whether it decides 1023^2 is even or odd. The author of the linked post repeatedly refers to ChatGPT's "reasoning". But ChatGPT doesn't reason, it just generates text. Sometimes that text might look like reasoning to a human--it's the sort of text t…

ChatGPT is a liberal arts major. One day it will "dual major" in math and then develop a determination for when the situation demands rigor (hard calculation) or when it demands "correct word estimation" (its current interaction mode).

A liberal arts major should also be able to draw conclusions of the form

  All A are B  
  C is an A  
  Therefore C is a B.

  A = square of odd number  
  B = odd number  
  C = 1023^2
My 5-year-old is not a math major, but he manages it.

Re: I quiz ChatGPT about math

#95

Post by pmarreck »

CrazyStat wrote: post #94
pmarreck wrote: post #92

Earlier quoted context omitted.

ChatGPT is a liberal arts major. One day it will "dual major" in math and then develop a determination for when the situation demands rigor (hard calculation) or when it demands "correct word estimation" (its current interaction mode).

A liberal arts major should also be able to draw conclusions of the form All A are B C is an A Therefore C is a B. A = square of odd number B = odd number C = 1023^2 My 5-year-old is not a math major, but he manages it.

The irony of ChatGPT not being logical enough is not lost on me

Re: I quiz ChatGPT about math

#96

Post by Lockal »

joshuahedlund wrote: post #73
otabdeveloper4 wrote: post #60

Earlier quoted context omitted.

8 / 2(2+2) was an internet meme, so not that astonishing. It should be in the training set somewhere.

It would be interesting to repeat the experiment with different integers.

Replace "2+2" with "2-2" and the answer is still the same. It does not read expressions, it does not see divisions by zero. It just outputs the training data, associated with parenthesis.

It will still hallucinate "division by zero" fallacies when asked questions like:

   Given that a - b > 0, can we simplify?
   a / (a - b) = 1 / (a - b)
   a = 1
Post reply on HN