I quiz ChatGPT about math
billwadge.com
I quiz ChatGPT about math
1–10 of 96 posts
Re: I quiz ChatGPT about math
#2Funny I quiz it about Gnosticism and the Uncreated Father. Much more fascinating to find it's non-science-y edges IMO.
Re: I quiz ChatGPT about math
#3I quizzed ChatGPT about math a few days back. It correctly answers that 1023 is odd, correctly answers that the square of an odd number is also odd, and is about 50/50 on whether it decides 1023^2 is even or odd.
The author of the linked post repeatedly refers to ChatGPT's "reasoning". But ChatGPT doesn't reason, it just generates text. Sometimes that text might look like reasoning to a human--it's the sort of text that a human who reasoned through the question would generate--but there was no reasoning involved. It's important to understand this.
> Based on my teaching experience, I’d say chatGPT scored like a second year math major.
Based on my experience, ChatGPT scored worse than my 5-year-old, who gets the 1023^2 question right every time.
Re: I quiz ChatGPT about math
#4> Substituting our values into this formula, we get: x1 = (3 + (10 / 3)) / 2 = 5
This is really wrong, why didn't the poster notice this? They just said
> 3.4 is not a very good approximation but other than that the response is correct.
No, Newtons method produces a really good approximation for square root very quickly, ChatGPT didn't do the method correctly which is why the result is so bad. People who can't instantly spot those mistakes shouldn't use ChatGPT, and people who can spot them instantly doesn't need to use ChatGPT.
Re: I quiz ChatGPT about math
#5If you read this carefully...
One way to prove that there are infinitely many prime numbers is by using the concept of contradiction. Suppose that there are only a finite number of prime numbers, say p_1, p_2, ..., p_n. Then consider the number N defined as follows:
N = p_1 * p_2 * ... * p_n + 1
If N is not prime, then it must have a prime divisor p that is not one of the primes p_1, p_2, ..., p_n. However, this contradicts the assumption that there are only finitely many prime numbers, since p is a prime that is not on the list. Therefore, N must be a prime number, which means that there are at least one more prime number beyond the list p_1, p_2, ..., p_n. This contradiction shows that our assumption that there are only finitely many prime numbers must be false, so there must be infinitely many prime numbers.
Re: I quiz ChatGPT about math
#6I quizzed ChatGPT about math a few days back. It correctly answers that 1023 is odd, correctly answers that the square of an odd number is also odd, and is about 50/50 on whether it decides 1023^2 is even or odd. The author of the linked post repeatedly refers to ChatGPT's "reasoning". But ChatGPT doesn't reason, it just generates text. Sometimes that text might look like reasoning to a human--it's the sort of text t…
Prompt:
> is 1023^2 even or odd?
Response:
> 1023 squared is even because it is equal to 1,046,529, which is an even number. To determine if a number is even or odd, you can check whether it is divisible by 2. If it is divisible by 2, it is even. If it is not divisible by 2, it is odd.
> In general, any number that is a perfect square will be even if the number being squared is odd, and it will be odd if the number being squared is even. This is because odd numbers, when squared, always result in an odd number, and even numbers, when squared, always result in an even number.
Re: I quiz ChatGPT about math
#7I quizzed ChatGPT about math a few days back. It correctly answers that 1023 is odd, correctly answers that the square of an odd number is also odd, and is about 50/50 on whether it decides 1023^2 is even or odd. The author of the linked post repeatedly refers to ChatGPT's "reasoning". But ChatGPT doesn't reason, it just generates text. Sometimes that text might look like reasoning to a human--it's the sort of text t…
I reminds me of study methods that involve "memorizing" as opposed to thinking about the problem, and applying various methods to solve it. It can solve some algorithm problems at a certain level, but struggles at a detailed answer in some scenarios. I guess it's not surprise, given ML is essentially exploring the local solution space of some abstract problem; it reminds me of a Taylor series expansion or a breadth-first search of an abstract problem space.
Re: I quiz ChatGPT about math
#8[deleted]
Re: I quiz ChatGPT about math
#9If you read this carefully... One way to prove that there are infinitely many prime numbers is by using the concept of contradiction. Suppose that there are only a finite number of prime numbers, say p_1, p_2, ..., p_n. Then consider the number N defined as follows: N = p_1 * p_2 * ... * p_n + 1 If N is not prime, then it must have a prime divisor p that is not one of the primes p_1, p_2, ..., p_n. However, this cont…
Good catch. The proof could simply state that, like all integers > 1, N must have a prime divisor, and that divisor cannot be any of the listed primes, since dividing N by any of them leaves remainder 1.
Re: I quiz ChatGPT about math
#10People love pointing out how bad GPT3 is at math - it’s just not remotely surprising, though. We all learned a long time ago that just patternmatching isn’t enough to solve math questions and we need to actually activate our brains and run algorithms to get valid answers.
Even mathy people fall into this - we all know that some numbers ‘look prime’. And up to 100 that instinct can work really well (except for 91).
But we all apply a heuristic for when to stop guessing and start mathing.
But guess what! GPT3 can do that too!
https://twitter.com/goodside/status/1568448128495534081
This thread predates chatgpt - but it really shows that GPT can pretty reliably spot when winging it won’t work. And in this case, it could effectively be given the ability to whip out a calculator to figure it out.
(Edited to add)
Found the follow up thread where Riley went into more detail and enhanced the results:
https://twitter.com/goodside/status/1581805503897735168?s=20...