Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can AI do maths yet? Thoughts from a mathematician

xenaproject.wordpress.com

Post reply on HN
Print view View original

111–120 of 364 posts

Re: Can AI do maths yet? Thoughts from a mathematician

#111

Post by ajmurmann »

derangedHorse wrote: post #106
newpavlov wrote: post #38

Earlier quoted context omitted.

>in the quantum research to demonstrate any valuable forward progress you must compute something that is impossible to do with a traditional computer This is factually wrong. The most interesting problems motivating the quantum computing research are hard to solve, but easy to verify on classical computers. The factorization problem is the most classical example. The problem is that existing quantum computers are not…

> This is factually wrong. What's factually wrong about it? OP said "you must compute something that is impossible to do with a traditional computer" which is true, regardless of the output produced. Verifying an output is very different from verifying the proper execution of a program. The difference between testing a program and seeing its code. What is being computed is fundamentally different from classical compu…

They left out the key part which was incorrect and the sentence right after "If you can't do it with a traditional computer, it suddenly becomes difficult to verify correctness"

The point stands that for actually interesting problems verifying correctness of the results is trivial. I don't know if "adherence to instructions" transudates at all to quantum computing.

Re: Can AI do maths yet? Thoughts from a mathematician

#112

Post by ludwik »

asadotzler wrote: post #88
ludwik wrote: post #81

Earlier quoted context omitted.

> But if all models were truly open, then we could simply verify what they had been trained on How do you verify what a particular open model was trained on if you haven’t trained it yourself? Typically, for open models, you only get the architecture and the trained weights. How can you reliably verify what the model was trained on from this? Even if they provide the training set (which is not typically the case), yo…

The OP said "truly open" not "open model" or any of the other BS out there. If you are truly open you share the training corpora as well or at least a comprehensive description of what it is and where to get it.

It seems like you skipped the second paragraph of my comment?

Re: Can AI do maths yet? Thoughts from a mathematician

#113

Post by semolinapudding »

Xcelerate wrote: post #49

So here's what I'm perplexed about. There are statements in Presburger arithmetic that take time doubly exponential (or worse) in the size of the statement to reach via any path of the formal system whatsoever. These are arithmetic truths about the natural numbers. Can these statements be reached faster in ZFC? Possibly—it's well-known that there exist shorter proofs of true statements in more powerful consistent sys…

ZFC is way worse than Presburger arithmetic -- since it is undecidable, we know that the length of the minimal proof of a statement cannot be bounded by a computable function of the length of the statement.

This has little to do with the usefulness of LLMs for research-level mathematics though. I do not think that anyone is hoping to get a decision procedure out of it, but rather something that would imitate human reasoning, which is heavily based on analogies ("we want to solve this problem, which shares some similarities with that other solved problem, can we apply the same proof strategy? if not, can we generalise the strategy so that it becomes applicable?").

Re: Can AI do maths yet? Thoughts from a mathematician

#114

Post by lupire »

thomashop wrote: post #110
lordnacho wrote: post #93

Earlier quoted context omitted.

The only thing I've consistently had issues with while using AI is graphs. If I ask it to put some simple function, it produces a really weird image that has nothing to do with the graph I want. It will be a weird swirl of lines and words, and it never corrects itself no matter what I say to it. Has anyone had any luck with this? It seems like the only thing that it just can't do.

Ask it to plot the graph with python plotting utilities. Not using its image generator. I think you need a ChatGPT subscription though for it to be able to run python code.

You seem to get 2(?) free Python program runs per week(?) as part of the 01 preview.

When you visit chatgpt on the free account it automatically gives you the best model and then disables it after some amount of work and says to come back later or upgrade.

Re: Can AI do maths yet? Thoughts from a mathematician

#115

Post by optimalsolver »

upghost wrote: post #65

I didn't see anyone else ask this but.. isn't the FrontierMath dataset compromised now? At the very least OpenAI now knows the questions if not the answers. I would expect that the next iteration will "magically" get over 80% on the FrontierMath test. I imagine that experiment was pretty closely monitored.

This was my first thought when I saw the results:

https://news.ycombinator.com/item?id=42473470

Re: Can AI do maths yet? Thoughts from a mathematician

#116

Post by SJC_Hacker »

actionfromafar wrote: post #60
AnerealDew wrote: post #35

Earlier quoted context omitted.

Only if you think current "AI" is on the same level as human creativity and intelligence, which it clearly is not.

I think current "AI" (i.e. LLMs) is unable to push human knowledge forward, but not because it's constrained by existing human knowledge. It's more like peeking into a very large magic-8 ball, new answers everytime you shake it. Some useful.

It may be able to push human knowledge forward to an extent.

In the past, there was quite a bit of low hanging fruit such that you could have polymaths able to contribute to a wide variety of fields, such as Newton.

But in the past 100 years or so, the problem is there is so much known, it is impossible for any single person to have deep knowledge of everything. e.g. its rare to find a really good mathematician who also has a deep knowledge (beyond intro courses) about say, chemistry.

Would a sufficiently powerful AI / ML model be able to come up with this synthesis across fields?

Re: Can AI do maths yet? Thoughts from a mathematician

#117

Post by lupire »

glimshe wrote: post #27
jampekka wrote: post #19

I just spent a few days trying to figure out some linear algebra with the help of ChatGPT. It's very useful for finding conceptual information from literature (which for a not-professional-mathematician at least can be really hard to find and decipher). But in the actual math it constantly makes very silly errors. E.g. indexing a vector beyond its dimension, trying to do matrix decomposition for scalars and insisting…

Isn't Wolfram Alpha a better "ChatGPT of Math"?

No. Wolfram Alpha can't solve anything that isn't a function evaluation or equation. And it can't do modular arithmetic to save its unlife.

WolframOne/Mathematica is better, but that requires the user (or ChatGPT!)to write complicated code, not natural language queries.

Re: Can AI do maths yet? Thoughts from a mathematician

#118

Post by jebarker »

> I am dreading the inevitable onslaught in a year or two of language model “proofs” of the Riemann hypothesis which will just contain claims which are vague or inaccurate in the middle of 10 pages of correct mathematics which the human will have to wade through to find the line which doesn’t hold up.

I wonder what the response of working mathematicians will be to this. If the proofs look credible it might be too tempting to try and validate them, but if there’s a deluge that could be a hug time sync. Imagine if Wiles or Perelman had produced a thousand different proofs for their respective problems.

Re: Can AI do maths yet? Thoughts from a mathematician

#119

Post by upghost »

optimalsolver wrote: post #115
upghost wrote: post #65

I didn't see anyone else ask this but.. isn't the FrontierMath dataset compromised now? At the very least OpenAI now knows the questions if not the answers. I would expect that the next iteration will "magically" get over 80% on the FrontierMath test. I imagine that experiment was pretty closely monitored.

This was my first thought when I saw the results: https://news.ycombinator.com/item?id=42473470

Insightful comment. The thing that's extremely frustrating is look at all the energy poured into this conversation around benchmarks. There is a fundamental assumption of honesty and integrity in the benchmarking process by at least some people. But when the dataset is compromised and generation N+1 has miraculous performance gains, how can we see this as anything other than a ploy to pump up valuations? Some people have millions of dollars at stake here and they don't care about the naysayers in the peanut gallery like us.

Re: Can AI do maths yet? Thoughts from a mathematician

#120

Post by bqmjjx0kac »

jebarker wrote: post #118

> I am dreading the inevitable onslaught in a year or two of language model “proofs” of the Riemann hypothesis which will just contain claims which are vague or inaccurate in the middle of 10 pages of correct mathematics which the human will have to wade through to find the line which doesn’t hold up. I wonder what the response of working mathematicians will be to this. If the proofs look credible it might be too tem…

Maybe the coming onslaught of AI slop "proofs" will give a little bump to proof assistants like Coq. Of course, it would still take a human mathematician some time to verify theorem definitions.
Post reply on HN