Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can AI do maths yet? Thoughts from a mathematician

xenaproject.wordpress.com

Post reply on HN
Print view View original

221–230 of 364 posts

Re: Can AI do maths yet? Thoughts from a mathematician

#221

Post by [deleted] »

obastani wrote: post #176
joe_the_user wrote: post #173

Earlier quoted context omitted.

The reddit thread is ... interesting (direct link[1]). It seems to be a debate among mathematicians some of whom do have access to the secret set. But they're debating publicly and so naturally avoiding any concrete examples that would give the set away so wind-up with fuzzy-fiddly language for the qualities of the problem tiers. The "reality" of keeping this stuff secret 'cause someone would train on it is itself bi…

It's not about training directly on the test set, it's about people discussing questions in the test set online (e.g., in forums), and then this data is swept up into the training set. That's what makes test set contamination so difficult to avoid.

[deleted]

Re: Can AI do maths yet? Thoughts from a mathematician

#222

Post by monktastic1 »

amelius wrote: post #175
aiono wrote: post #135

Earlier quoted context omitted.

Where is this data from?

It's a question, and to be fair to AI it should actually refer to papers before review.

Yes, it's a question, but you haven't answered what you read that makes you suspect so.

Re: Can AI do maths yet? Thoughts from a mathematician

#223

Post by staunton »

dinosaurdynasty wrote: post #164
pixl97 wrote: post #139

Earlier quoted context omitted.

I mean, that is why they mention super-Turning processes like quantum based computing.

Quantum computing actually isn't super-Turing, it "just" computes some things faster. (Strictly speaking it's somewhere between a standard Turing machine and a nondeterministic Turing machine in speed, and the first can emulate the second.)

If we're nitpicking: quantum computing algorithms could (if implemented) compute certain things faster than the best classical algorithms we know. We don't know any quantum algorithms that are provably faster than all possible classical algorithms.

Re: Can AI do maths yet? Thoughts from a mathematician

#224

Post by edflsafoiewq »

bongodongobob wrote: post #206
CamperBob2 wrote: post #184

Earlier quoted context omitted.

You know, "You're using it wrong" is usually meant to carry an ironic or sarcastic tone, right? It dates back to Steve Jobs blaming an iPhone 4 user for "holding it wrong" rather than acknowledging a flawed antenna design that was causing dropped calls. The closest Apple ever came to admitting that it was their problem was when they subsequently ran an employment ad to hire a new antenna engineering lead. Maybe it's…

It's not an LLM. You're simply asking too much of it. It doesn't work the way you want it to, sorry.

Correct, so it isn't a "ChatGPT of Math", which was the point.

Re: Can AI do maths yet? Thoughts from a mathematician

#225

Post by kevinventullo »

jebarker wrote: post #118

> I am dreading the inevitable onslaught in a year or two of language model “proofs” of the Riemann hypothesis which will just contain claims which are vague or inaccurate in the middle of 10 pages of correct mathematics which the human will have to wade through to find the line which doesn’t hold up. I wonder what the response of working mathematicians will be to this. If the proofs look credible it might be too tem…

Honestly I think it won’t be that different from today, where there is no shortage of cranks producing “proofs” of the Riemann Hypothesis and submitting them to prestigious journals.

Re: Can AI do maths yet? Thoughts from a mathematician

#226

Post by bongodongobob »

lern_too_spel wrote: post #172
bongodongobob wrote: post #156

Earlier quoted context omitted.

Eugenics and vitamin C as a cure all.

If Pauling's eugenics policies were bad, then the laws against incest that are currently on the books in many states (which are also eugenics policies that use the same mechanism) are also bad. There are different forms of eugenics policies, and Pauling's proposal to restrict the mating choices of people carrying certain recessive genes so their children don't suffer is ethically different from Hitler exterminating p…

From what I remember, he wanted to mark people with tattoos or something.

Re: Can AI do maths yet? Thoughts from a mathematician

#227

Post by SpaceManNabs »

ludwik wrote: post #112
asadotzler wrote: post #88

Earlier quoted context omitted.

The OP said "truly open" not "open model" or any of the other BS out there. If you are truly open you share the training corpora as well or at least a comprehensive description of what it is and where to get it.

It seems like you skipped the second paragraph of my comment?

Because it is mostly hogwash.

Lots of ai researchers have shown that you can both give credit and discredit "open models" when you are given a dataset and training steps.

Many lauded papers fell into reddit Ml or twitter ire when people couldnt reproduce the model or results.

If you are given the training set, the weights, the steps required, and enough compute, you can do it.

Having enough compute and people releasing the steps is the main impediment.

For my research I always release all of my code, and the order of execution steps, and of course the training set. I also give confidence intervals based on my runs so people can reproduce and see if we get similar intervals.

Re: Can AI do maths yet? Thoughts from a mathematician

#228

Post by bwfan123 »

sincerecook wrote: post #75

No it can't, and there's no such thing as AI. How is a thing that predicts the next-most-likely word going to do novel math? It can't even do existing math reliably because logical operations and statistical approximation are fundamentally different. It is fun watching grifters put lipstick on this thing and shop it around as a magic pig though.

openai and epochai (frontier math) are startups with a strong incentive to push such narratives. the real test will be in actual adoption in real world use cases.

the management class has a strong incentive to believe in this narrative, since it helps them reduce labor cost. so they are investing in it.

eventually, the emperor will be seen to have no clothes at least in some usecases for which it is being peddled right now.

Re: Can AI do maths yet? Thoughts from a mathematician

#229

Post by cowl »

bondarchuk wrote: post #84
newpavlov wrote: post #38

Earlier quoted context omitted.

>in the quantum research to demonstrate any valuable forward progress you must compute something that is impossible to do with a traditional computer This is factually wrong. The most interesting problems motivating the quantum computing research are hard to solve, but easy to verify on classical computers. The factorization problem is the most classical example. The problem is that existing quantum computers are not…

No, it is factually right, at least if Scott Aaronson is to be believed: > Having said that, the biggest caveat to the “10^25 years” result is one to which I fear Google drew insufficient attention. Namely, for the exact same reason why (as far as anyone knows) this quantum computation would take ~10^25 years for a classical computer to simulate, it would also take ~10^25 years for a classical computer to directly ve…

that applies specifically to this artificial problem google created to be hard for classical computers and in fact in the end it turned out it was not so much. IBM came up with a method to do what google said it would take 10.000 years on a classical computers in just 2 days. I would not be surprised if a similar reduction happened also to their second attempt if anyone was motivated enough to look at it.

In general we have thousands of optimisations problems that are hard to solve but immediate to verify.

Re: Can AI do maths yet? Thoughts from a mathematician

#230

Post by ls612 »

upghost wrote: post #73
jvanderbot wrote: post #69

Earlier quoted context omitted.

I figured their model was independently evaluated against the questions/answers. That's not to say it's not compromised by "Here's a bag of money" type methods, but I don't even think it'd be a reasonable test if they just handed over the dataset.

I'm sure it was independently evaluated, but I'm sure the folks running the test were not given an on-prem installation of ChatGPT to mess with. It was still done via API calls, presumably through the chat interface UI. That means the questions went over the fence to OpenAI. I'm quite certain they are aware of that, and it would be pretty foolish not to take advantage of at least knowing what the questions are.

Depending on the plan the researchers used they may have contractual protections against OpenAI training on their inputs.
Post reply on HN