Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I quiz ChatGPT about math

billwadge.com

Post reply on HN
Print view View original

71–80 of 96 posts

Re: I quiz ChatGPT about math

#71

Post by gernb »

HDThoreaun wrote: post #36
dxbydt wrote: post #32

Earlier quoted context omitted.

If I post that same question on Wikipedia and confidently assert the answer is 1, some good Samaritan will edit it within a day & say no dumbass its 6, and then link me to the Talk page & we'll duke it out over there. On ChatGPT, you have no recourse other than the thumb down button. When you give it the right answer it simply refuses to accept! It's sure amusing as an adult, but be aware that several schools (atleas…

Well my point is there aren't pages for specific math problems on wikipedia, because that's not the goal of wikipedia. ChatGPT can tell you the rules of algebra, but it's bad at applying them. If you want to solve math problems you should go to wolframalpha. My personal opinion is that having an encyclopedia that is sometimes wrong is actually incredibly valuable for educating people in the social media age. Distingu…

The problem is when it's not objectively wrong but it has an opinion of what is right and refuses to consider alternatives. Just had an experience with it when I asked for advice on xyz it effectively said "xyz is bad, um-kay" when "xyz" was not objectively bad and there are debates about it all over. I tried 5 ~ 6 different ways to get it to acknowledge there were different points of view but it just kept parroting the same paragraph that effectively said "xyz is bad, um-kay"

It made me really scared for the day there is the one "true think" and that's the only answer you get from Siri, Google, etc...

Re: I quiz ChatGPT about math

#72

Post by clownshoez »

bawolff wrote: post #54
teawrecks wrote: post #26

Earlier quoted context omitted.

The point is that Wikipedia is infinitely more fallible than a mathematical equation.

That doesn't make sense. Math equations are expressions of ideas. They can be fallible as much as anything else. Just because 1+1=3 is an equation doesn't make it true.

Yes of course and Euclid's parallel postulate is a matter of fact in your opinion too?

We use abstractions friend, some are useful often and others are not often used.

Re: I quiz ChatGPT about math

#73

Post by joshuahedlund »

otabdeveloper4 wrote: post #60
wyldfire wrote: post #34

Earlier quoted context omitted.

> There are a few reasons why people may argue about the answer to a mathematical question like “8 / 2(2+2)”. One reason is that there is more than one wav to interpret the question and arrive at a different answer. For example, some people may interpret the question as “8 / (2 * (2+2))” and arrive at an answer of 1, while others mav interpret it as “(8 / 2) * (2+2)” and arrive at an answer of 16. Wow, this is kinda…

8 / 2(2+2) was an internet meme, so not that astonishing. It should be in the training set somewhere.

It would be interesting to repeat the experiment with different integers.

Re: I quiz ChatGPT about math

#74

Post by unnouinceput »

I gave up having ChatGPT do the math when explaining/doing homework with my kids. I don't have time to proof its mistakes. As such my prompt for it when dealing with math is along the lines of:

"Solve . Use WolframAlpha and give me the link".

Then I just copy the link it's spitting out and I go myself to WolframAlpha for the correct result.

ChatGPT is great at understanding the description of the problem in plain language and writing the WolframAlpha input I need. It shortens greatly my time to verify my kids exercises since WolframAlpha's input box is not that great.

Re: I quiz ChatGPT about math

#75

Post by xg15 »

BoppreH wrote: post #44
kevinventullo wrote: post #23

Earlier quoted context omitted.

That is very cool, and surprising to me given I was also initially underwhelmed. One nitpick though: in the second question, although the explanation itself is clean, there appear to be seven b’s in its representation of the number six.

I've heard it's because GPT's input is whole words/tokens, not characters. So it has little insight into their spelling or letter positions, unless it's explicitly mentioned in the training set (e.g. rhymes are easy, counting letters of rare words is hard). Though this is a specially bad example, surely it had bbbbbb somewhere in it's training set associated with "6".

It's even more interesting, I think. The tokens are a byte pair encoding [1] of the input string. So a short, frequent word might be represented as one token, but an infrequent word (such as "bbbbbbb") might be represented by several tokens, each of which might or might not correspond to a letter.

This might also explain the weird "off-by-one" errors with the ROT13 task.

[1] https://en.m.wikipedia.org/wiki/Byte_pair_encoding

Re: I quiz ChatGPT about math

#76

Post by SamBam »

SketchySeaBeast wrote: post #40
SamBam wrote: post #15

I was actually very impressed with some math questions it was able to answer for me: S: If flabb means three, snabb means four, and whabb mean five, what is flabb plus whabb? If flabb means three, snabb means four, and whabb means five, then flabb plus whabb would be equal to three plus five, or eight. In this made-up numbering system, the words “flabb,” “snabb,” and “whabb” represent the numbers three, four, and fiv…

> If flabb means three, snabb means four, and whabb means five, then flabb plus whabb would be equal to three plus five, or eight. In this made-up numbering system, the words “flabb,” “snabb,” and “whabb” represent the numbers three, four, and five, respectively. When these numbers are added together, the result is the number eight. Was its training model a freshman university student trying to get through a 10 page…

Actually, I cut some of the more verbose parts of all three answers...

Re: I quiz ChatGPT about math

#77

Post by SamBam »

tmp60beb0ed wrote: post #63
wyldfire wrote: post #34

Earlier quoted context omitted.

> There are a few reasons why people may argue about the answer to a mathematical question like “8 / 2(2+2)”. One reason is that there is more than one wav to interpret the question and arrive at a different answer. For example, some people may interpret the question as “8 / (2 * (2+2))” and arrive at an answer of 1, while others mav interpret it as “(8 / 2) * (2+2)” and arrive at an answer of 16. Wow, this is kinda…

Regarding the issue of "mav", it looks like the poster used screen capture followed by some image-to-text OCR routine (possibly tesseract). This feels like an OCR fail.

Exactly what happened. I was capturing some of my earlier conversations as screenshots, and didn't feel like retyping the whole thing. Errors are mine, apologies.

Re: I quiz ChatGPT about math

#78

Post by aaron695 »

SamBam wrote: post #15

I was actually very impressed with some math questions it was able to answer for me: S: If flabb means three, snabb means four, and whabb mean five, what is flabb plus whabb? If flabb means three, snabb means four, and whabb means five, then flabb plus whabb would be equal to three plus five, or eight. In this made-up numbering system, the words “flabb,” “snabb,” and “whabb” represent the numbers three, four, and fiv…

No post body was provided.

Re: I quiz ChatGPT about math

#79

Post by jrm4 »

It feels like I've stumped it nearly completely with many variations on the following question:

"In binary, do most people have 10 legs?"

Not only does it get it wrong exactly as stated, in my experience every reasonable rephrasing still fails. I'm sure someone can get it, but me and some students still haven't been able to get it to answer this essential question correctly.

Re: I quiz ChatGPT about math

#80

Post by bumbledraven »

dxbydt wrote: post #12

Here's a very easy middle school competition problem - https://imgur.com/a/YdFSBVY It repeatedly got the answer wrong. I engaged in a long conversation with it (too long to post) where I tried to convince it that its answer was wrong, but it became quite rude at a certain point & alleged I was trying to trick it or something (?!) so I just gave up. Unfortunately, I was doing the whole exercise in front of my 10 year…

In case anyone wants a solution to this “very easy middle school competition problem”: https://www.quora.com/How-do-you-solve-this-analytically-sum...
Post reply on HN