Can AI do maths yet? Thoughts from a mathematician • Hacker News Board

Re: Can AI do maths yet? Thoughts from a mathematician

#121

Post by lupire » Mon, Dec 23, 2024, 3:21 PM UTC

wongarsu wrote: post #56

lazide wrote: post #22

Earlier quoted context omitted.

If constrained by existing human knowledge to come up with an answer, won’t it fundamentally be unable to push human knowledge forward?

Reasoning is essentially the creation of new knowledge from existing knowledge. The better the model can reason the less constrained it is to existing knowledge. The challenge is how to figure out if a model is genuinely reasoning

Reasoning is a very minor (but essential) part of knowledge creation.

Knowledge creation comes from collecting data from the real world, and cleaning it up somehow, and brainstorming creative models to explain it.

NN/LLM's version of model building is frustrating because it is quite good, but not highly "explainable". Human models have higher explainability, while machine models have high predictive value on test examples due to an impenetrable mountain of algebra.

Re: Can AI do maths yet? Thoughts from a mathematician

#122

Post by alphan0n » Mon, Dec 23, 2024, 3:25 PM UTC

lcnPylGDnU4H9OF wrote: post #96

alphan0n wrote: post #64

As far as ChatGPT goes, you may as well be asking: Can AI use a calculator? The answer is yes, it can utilize a stateful python environment and solve complex mathematical equations with ease.

There is a difference between correctly stating that 2 + 2 = 4 within a set of logical rules and proving that 2 + 2 = 4 must be true given the rules.

I think you misunderstood, ChatGPT can utilize Python to solve a mathematical equation and provide proof.

https://chatgpt.com/share/676980cb-d77c-8011-b469-4853647f98...

Re: Can AI do maths yet? Thoughts from a mathematician

#123

Post by optimalsolver » Mon, Dec 23, 2024, 3:46 PM UTC

upghost wrote: post #119

optimalsolver wrote: post #115

Earlier quoted context omitted.

This was my first thought when I saw the results: https://news.ycombinator.com/item?id=42473470

Insightful comment. The thing that's extremely frustrating is look at all the energy poured into this conversation around benchmarks. There is a fundamental assumption of honesty and integrity in the benchmarking process by at least some people. But when the dataset is compromised and generation N+1 has miraculous performance gains, how can we see this as anything other than a ploy to pump up valuations? Some people…

It's sadly inevitable that when billions in funding and industry hype are tied to performance on a handful of benchmarks, scores will somehow, magically, continue to go up.

Needless to say, it doesn't bring us any closer to AGI.

The only solution I see here is people crafting their own, private benchmarks that the big players don't care about enough to train on. That, at least, gives you a clearer view of the field.

Re: Can AI do maths yet? Thoughts from a mathematician

#124

Post by yodsanklai » Mon, Dec 23, 2024, 3:53 PM UTC

I understand the appeal of having a machine helping us with maths and expanding the frontier of knowledge. They can assist researchers and make them more productive. Just like they can make already programmers more productive.

But maths are also fun and fulfilling activity. Very often, when we learn a math theory, it's because we want to understand and gain intuition on the concepts, or we want to solve a puzzle (for which we can already look up the solution). Maybe it's similar to chess. We didn't develop search engines to replace human players and make them play together, but they helped us become better chess players or understanding the game better.

So the recent progress is impressive, but I still don't see how we'll use this tech practically and what impacts it can have and in which fields.

Re: Can AI do maths yet? Thoughts from a mathematician

#125

Post by amelius » Mon, Dec 23, 2024, 4:04 PM UTC

jampekka wrote: post #19

I just spent a few days trying to figure out some linear algebra with the help of ChatGPT. It's very useful for finding conceptual information from literature (which for a not-professional-mathematician at least can be really hard to find and decipher). But in the actual math it constantly makes very silly errors. E.g. indexing a vector beyond its dimension, trying to do matrix decomposition for scalars and insisting…

Don't most mathematical papers contain at least one such error?

Re: Can AI do maths yet? Thoughts from a mathematician

#126

Post by amelius » Mon, Dec 23, 2024, 4:05 PM UTC

lupire wrote: post #114

thomashop wrote: post #110

Earlier quoted context omitted.

Ask it to plot the graph with python plotting utilities. Not using its image generator. I think you need a ChatGPT subscription though for it to be able to run python code.

You seem to get 2(?) free Python program runs per week(?) as part of the 01 preview. When you visit chatgpt on the free account it automatically gives you the best model and then disables it after some amount of work and says to come back later or upgrade.

Just install Python locally, and copy paste the code.

Re: Can AI do maths yet? Thoughts from a mathematician

#127

Post by amelius » Mon, Dec 23, 2024, 4:07 PM UTC

Filligree wrote: post #44

glimshe wrote: post #27

Earlier quoted context omitted.

Isn't Wolfram Alpha a better "ChatGPT of Math"?

Wolfram Alpha is better at actually doing math, but far worse at explaining what it’s doing, and why.

I wish there was a way to tell Chatgpt where it has made a mistake, with a single mouse click.

Re: Can AI do maths yet? Thoughts from a mathematician

#128

Post by lupire » Mon, Dec 23, 2024, 4:08 PM UTC

actionfromafar wrote: post #60

AnerealDew wrote: post #35

Earlier quoted context omitted.

Only if you think current "AI" is on the same level as human creativity and intelligence, which it clearly is not.

I think current "AI" (i.e. LLMs) is unable to push human knowledge forward, but not because it's constrained by existing human knowledge. It's more like peeking into a very large magic-8 ball, new answers everytime you shake it. Some useful.

That's not a strong reason. Yes, that means ChatGPT isn't good at wholly independently pushing knowledge forward, but a good brainstormer that is right even 10% of the time is an incredible found of knowledge.

Re: Can AI do maths yet? Thoughts from a mathematician

#129

Post by TeamDman » Mon, Dec 23, 2024, 4:10 PM UTC

Flenkno wrote: post #25

noFaceDiscoG668 wrote: post #2

"once" the training data can do it, LLMs will be able to do it. and AI will be able to do math once it comes to check out the lights of our day and night. until then it'll probably wonder continuously and contiguously: "wtf! permanence! why?! how?! by my guts, it actually fucking works! why?! how?!"

AWS announced 2 or 3 weeks a way of formulating rules into a formal language. AI doesn't need to learn everything, our LLM Models already contain EVERYTHING. Including ways of how to find a solution step by step. Which means, you can tell an LLM to translate whatever you want, into a logical language and use an external logic verifier. The only thing a LLM or AI needs to 'understand' at this point is to make sure tha…

Is this the AWS thing you referenced? https://aws.amazon.com/what-is/automated-reasoning/

Re: Can AI do maths yet? Thoughts from a mathematician

#130

Post by vouaobrasil » Mon, Dec 23, 2024, 4:12 PM UTC

My favourite moments of being a graduate student in math was showing my friends (and sometimes professors) proofs of propositions and theorems that we discussed together. To be the first to put together a coherent piece of reasoning that would convince them of the truth was immensely exciting. Those were great bonding moments amongst colleagues. The very fact that we needed each other to figure out the basics of the subject was part of what made the journey so great.

Now, all of that will be done by AI.

Reminds of the time when I finally enabled invincibility in Goldeneye 007. Rather boring.

I think we've stopped to appreciate the human struggle and experience and have placed all the value on the end product, and that's we're developing AI so much.

Yeah, there is the possibility of working with an AI but at that point, what is the point? Seems rather pointless to me in an art like mathematics.