Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can AI do maths yet? Thoughts from a mathematician

xenaproject.wordpress.com

Post reply on HN
Print view View original

21–30 of 364 posts

Re: Can AI do maths yet? Thoughts from a mathematician

#21

Post by LittleTimothy »

It's fascinating that this has run into the exact same problem as the Quantum research. Ie, in the quantum research to demonstrate any valuable forward progress you must compute something that is impossible to do with a traditional computer. If you can't do it with a traditional computer, it suddenly becomes difficult to verify correctness (ie, you can't just check it was matching the traditional computer's answer.

In the same way ChatGPT scores 25% on this and the question is "How close were those 25% to questions in the training set". Or to put it another way we want to answer the question "Is ChatGPT getting better at applying it's reasoning to out-of-set problems or is it pulling more data into it's training set". Or "Is the test leaking into the training".

Maybe the whole question is academic and it doesn't matter, we solve the entire problem by pulling all human knowledge into the training set and that's a massive benefit. But maybe it implies a limit to how far it can push human knowledge forward.

Re: Can AI do maths yet? Thoughts from a mathematician

#22

Post by lazide »

LittleTimothy wrote: post #21

It's fascinating that this has run into the exact same problem as the Quantum research. Ie, in the quantum research to demonstrate any valuable forward progress you must compute something that is impossible to do with a traditional computer. If you can't do it with a traditional computer, it suddenly becomes difficult to verify correctness (ie, you can't just check it was matching the traditional computer's answer. I…

If constrained by existing human knowledge to come up with an answer, won’t it fundamentally be unable to push human knowledge forward?

Re: Can AI do maths yet? Thoughts from a mathematician

#23

Post by actionfromafar »

lazide wrote: post #22
LittleTimothy wrote: post #21

It's fascinating that this has run into the exact same problem as the Quantum research. Ie, in the quantum research to demonstrate any valuable forward progress you must compute something that is impossible to do with a traditional computer. If you can't do it with a traditional computer, it suddenly becomes difficult to verify correctness (ie, you can't just check it was matching the traditional computer's answer. I…

If constrained by existing human knowledge to come up with an answer, won’t it fundamentally be unable to push human knowledge forward?

Then much of human research and development is also fundamentally impossible.

Re: Can AI do maths yet? Thoughts from a mathematician

#24

Post by actionfromafar »

fifilura wrote: post #18
asddubs wrote: post #14

Earlier quoted context omitted.

it can take my math and point out a step I missed and then show me the correct procedure but still get the wrong result because it can't reliably multiply 2-digit numbers

Better than an average human then.

Different than an average human.

Re: Can AI do maths yet? Thoughts from a mathematician

#25

Post by Flenkno »

noFaceDiscoG668 wrote: post #2

"once" the training data can do it, LLMs will be able to do it. and AI will be able to do math once it comes to check out the lights of our day and night. until then it'll probably wonder continuously and contiguously: "wtf! permanence! why?! how?! by my guts, it actually fucking works! why?! how?!"

AWS announced 2 or 3 weeks a way of formulating rules into a formal language.

AI doesn't need to learn everything, our LLM Models already contain EVERYTHING. Including ways of how to find a solution step by step.

Which means, you can tell an LLM to translate whatever you want, into a logical language and use an external logic verifier. The only thing a LLM or AI needs to 'understand' at this point is to make sure that the statistical translation from left to right is high enough.

Your brain doesn't just do logic out of the box, You conclude things and formulate them.

And plenty of companies work on this. Its the same with programming, if you are able to write code and execute it, you execute it until the compiler errors are gone. Now your LLM can write valid code out of the box. Let the LLM write unit tests, now it can verify itself.

Claude for example offers you, out of the box, to write a validation script. You can give claude back the output of the script claude suggested to you.

Don't underestimate LLMs

Re: Can AI do maths yet? Thoughts from a mathematician

#26

Post by Havoc »

lazide wrote: post #22
LittleTimothy wrote: post #21

It's fascinating that this has run into the exact same problem as the Quantum research. Ie, in the quantum research to demonstrate any valuable forward progress you must compute something that is impossible to do with a traditional computer. If you can't do it with a traditional computer, it suddenly becomes difficult to verify correctness (ie, you can't just check it was matching the traditional computer's answer. I…

If constrained by existing human knowledge to come up with an answer, won’t it fundamentally be unable to push human knowledge forward?

I don't think many expect AI to push knowledge forward? A thing that basically just regurgitates consensus historic knowledge seems badly suited to that

Re: Can AI do maths yet? Thoughts from a mathematician

#27

Post by glimshe »

jampekka wrote: post #19

I just spent a few days trying to figure out some linear algebra with the help of ChatGPT. It's very useful for finding conceptual information from literature (which for a not-professional-mathematician at least can be really hard to find and decipher). But in the actual math it constantly makes very silly errors. E.g. indexing a vector beyond its dimension, trying to do matrix decomposition for scalars and insisting…

Isn't Wolfram Alpha a better "ChatGPT of Math"?

Re: Can AI do maths yet? Thoughts from a mathematician

#28

Post by Someone »

est wrote: post #13

At this stage I assume everything having a sequencial pattern can and will be automated by LLM AIs.

I think that’s provably incorrect for the current approach to LLMs. They all have a horizon over which they correlate tokens in the input stream.

So, for any LLM, if you intersperse more than that number of ‘X’ tokens between each useful token, they won’t be able to do anything resembling intelligence.

The current LLMs are a bit like n-gram databases that do not use letters, but larger units.

Post reply on HN