Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can AI do maths yet? Thoughts from a mathematician

xenaproject.wordpress.com

Post reply on HN
Print view View original

241–250 of 364 posts

Re: Can AI do maths yet? Thoughts from a mathematician

#241

Post by sourcepluck »

vouaobrasil wrote: post #130

My favourite moments of being a graduate student in math was showing my friends (and sometimes professors) proofs of propositions and theorems that we discussed together. To be the first to put together a coherent piece of reasoning that would convince them of the truth was immensely exciting. Those were great bonding moments amongst colleagues. The very fact that we needed each other to figure out the basics of the…

> Now, all of that will be done by AI.

No "AI" of any description is doing novel proofs at the moment. Not o3, or anything else.

LLMs are good for chatting about basic intuition with, up to and including complex subjects, if and only if there are publically available data on the topic which have been fed to the LLM during its training. They're good at doing summaries and overviews of specific things (if you push them around and insist they don't waffle and ignore garbage carefully and keep your critical thinking hat on, etc etc).

It's like having a magnifying glass that focuses in on the small little maths question you might have, without you having to sift through ten blogs or videos or whatever.

That's hardly going to replace graduate students doing proofs with professors, though, at least not with the methods being employed thus far!

Re: Can AI do maths yet? Thoughts from a mathematician

#242

Post by upghost »

ls612 wrote: post #230
upghost wrote: post #73

Earlier quoted context omitted.

I'm sure it was independently evaluated, but I'm sure the folks running the test were not given an on-prem installation of ChatGPT to mess with. It was still done via API calls, presumably through the chat interface UI. That means the questions went over the fence to OpenAI. I'm quite certain they are aware of that, and it would be pretty foolish not to take advantage of at least knowing what the questions are.

Depending on the plan the researchers used they may have contractual protections against OpenAI training on their inputs.

Sure, but given the resourcing at OpenAI, it would not be hard to clean[1] the inputs. I'm just trying to be realistic here, there are plenty of ways around contractual obligations and a significant incentive to do so.

[1]: https://en.wikipedia.org/wiki/Clean-room_design

Re: Can AI do maths yet? Thoughts from a mathematician

#243

Post by voltaireodactyl »

lern_too_spel wrote: post #172
bongodongobob wrote: post #156

Earlier quoted context omitted.

Eugenics and vitamin C as a cure all.

If Pauling's eugenics policies were bad, then the laws against incest that are currently on the books in many states (which are also eugenics policies that use the same mechanism) are also bad. There are different forms of eugenics policies, and Pauling's proposal to restrict the mating choices of people carrying certain recessive genes so their children don't suffer is ethically different from Hitler exterminating p…

FWIW my understanding is that the policies against incest you mention actually have much less to do with controlling genetic reproduction and are more directed at combating familial rape/grooming/etc.

Not a fun thing to discuss, but apparently a significant issue, which I guess should be unsurprising given some of the laws allowing underage marriage if the family signs off.

Mentioning only to draw attention to the fact that theoretical policy is often undeniable in a vacuum, but runs aground when faced with real world conditions.

Re: Can AI do maths yet? Thoughts from a mathematician

#244

Post by raincole »

vouaobrasil wrote: post #146
busyant wrote: post #138

As someone who has a 18 yo son who wants to study math, this has me (and him) ... worried ... about becoming obsolete? But I'm wondering what other people think of this analogy. I used to be a bench scientist (molecular genetics). There were world class researchers who were more creative than I was. I even had a Nobel Laureate once tell me that my research was simply "dotting 'i's and crossing 't's". Nevertheless, I…

I was just thinking about this. I already posted a comment here, but I will say that as a mathematician (PhD in number theory), that for me, AI signficantly takes away the beauty of doing mathematics within a realm in which AI is used. The best part of math (again, just for me) is that it was a journey that was done by hand with only the human intellect that computers didn't understand. The beauty of the subject was…

> Now most will just use some AI.

Do people with PhD in math really ask AI to explain math concepts to them?

Re: Can AI do maths yet? Thoughts from a mathematician

#245

Post by xmprt »

phkahler wrote: post #217
obastani wrote: post #176

Earlier quoted context omitted.

It's not about training directly on the test set, it's about people discussing questions in the test set online (e.g., in forums), and then this data is swept up into the training set. That's what makes test set contamination so difficult to avoid.

>> It's not about training directly on the test set, it's about people discussing questions in the test set online Don't kid yourself. There are 10's of billions of dollars going into AI. Some of the humans involved would happily cheat on comparative tests to boost investment.

The incentives are definitely there, but even CEOs and VCs know that if they cheat the tests just to get more investment, they're only cheating themselves. No one is liquidating within the next 5 years so either they end up getting caught and lose everything or they spent all this energy trying to cheat while having a subpar model which results in them losing to competitors who actually invested in good technology.

Having a higher valuation could help with attracting better talent or more funding to invest in GPUs and actual model improvements but I don't think that outweighs the risks unless you're a tiny startup with nothing to show (but then you wouldn't have the money to bribe anyone).

Re: Can AI do maths yet? Thoughts from a mathematician

#246

Post by meroes »

hyhconito wrote: post #186
busyant wrote: post #138

As someone who has a 18 yo son who wants to study math, this has me (and him) ... worried ... about becoming obsolete? But I'm wondering what other people think of this analogy. I used to be a bench scientist (molecular genetics). There were world class researchers who were more creative than I was. I even had a Nobel Laureate once tell me that my research was simply "dotting 'i's and crossing 't's". Nevertheless, I…

Let's put it this way, from another mathematician, and I'm sure I'll probably be shot for this one. Every LLM release moves half of the remaining way to the minimum viable goal of replacing a third class undergrad. If your business or research initiative is fine with that level of competence then you will find utility. The problem is that I don't know anyone who would find that useful. Nor does it fit within any exis…

Well said. As someone with only a math undergrad and as a math RLHF’er, this speaks to my experience the most.

That craving for an understanding an elegant proof is nowhere to be found with verifying an LLM’s proof.

Like sure, you could put together a car by first building an airplane, disassembling all of it minus the two front seats, and having zero elegance and still get a car at the end. But if you do all that and don’t provide novelty in results or useful techniques, there’s no business.

Hell, I can’t even get a model to calculate compound interest for me (save for the technicality of prompt engineering a python function to do it). What do I expect?

Re: Can AI do maths yet? Thoughts from a mathematician

#247

Post by Syzygies »

"Can AI do math for us" is the canonical wrong question. People want self-driving cars so they can drink and watch TV. We should crave tools that enhance our abilities, as tools have done since prehistoric times.

I'm a research mathematician. In the 1980's I'd ask everyone I knew a question, and flip through the hard bound library volumes of Mathematical Reviews, hoping to recognize something. If I was lucky, I'd get a hit in three weeks.

Internet search has shortened this turnaround. One instead needs to guess what someone else might call an idea. "Broken circuits?" Score! Still, time consuming.

I went all in on ChatGPT after hearing that Terry Tao had learned the Lean 4 proof assistant in a matter of weeks, relying heavily on AI advice. It's clumsy, but a very fast way to get suggestions.

Now, one can hold involved conversations with ChatGPT or Claude, exploring mathematical ideas. AI is often wrong, never knows when it's wrong, but people are like this too. Read how the insurance incidents for self-driving taxis are well below the human incident rates? Talking to fellow mathematicians can be frustrating, and so is talking with AI, but AI conversations go faster and can take place in the middle of the night.

I don't want AI to prove theorems for me, those theorems will be as boring as most of the dreck published by humans. I want AI to inspire bursts of creativity in humans.

Re: Can AI do maths yet? Thoughts from a mathematician

#248

Post by ninetyninenine »

Syzygies wrote: post #247

"Can AI do math for us" is the canonical wrong question. People want self-driving cars so they can drink and watch TV. We should crave tools that enhance our abilities, as tools have done since prehistoric times. I'm a research mathematician. In the 1980's I'd ask everyone I knew a question, and flip through the hard bound library volumes of Mathematical Reviews, hoping to recognize something. If I was lucky, I'd get…

Your optimism should be tempered with the downside of progress meaning that AI in the near future may not only inspire creativity in humans, but it can replace human creativity all together.

Why do I need to hire an artist for my movie/video game/advertisement when AI can replicate all the creativity I need.

Re: Can AI do maths yet? Thoughts from a mathematician

#249

Post by CamperBob2 »

kortilla wrote: post #232
CamperBob2 wrote: post #184

Earlier quoted context omitted.

You know, "You're using it wrong" is usually meant to carry an ironic or sarcastic tone, right? It dates back to Steve Jobs blaming an iPhone 4 user for "holding it wrong" rather than acknowledging a flawed antenna design that was causing dropped calls. The closest Apple ever came to admitting that it was their problem was when they subsequently ran an employment ad to hire a new antenna engineering lead. Maybe it's…

No, “holding it wrong” is the sarcastic version. “You’re using it wrong” is a super common way to tell people they are literally using something wrong.

But they're not using it wrong. They are using it as advertised by Wolfram themselves (read: himself).

The GP's rocket equation question is exactly the sort of use case for which Alpha has been touted for years.

Re: Can AI do maths yet? Thoughts from a mathematician

#250

Post by wnc3141 »

ninetyninenine wrote: post #248
Syzygies wrote: post #247

"Can AI do math for us" is the canonical wrong question. People want self-driving cars so they can drink and watch TV. We should crave tools that enhance our abilities, as tools have done since prehistoric times. I'm a research mathematician. In the 1980's I'd ask everyone I knew a question, and flip through the hard bound library volumes of Mathematical Reviews, hoping to recognize something. If I was lucky, I'd get…

Your optimism should be tempered with the downside of progress meaning that AI in the near future may not only inspire creativity in humans, but it can replace human creativity all together. Why do I need to hire an artist for my movie/video game/advertisement when AI can replicate all the creativity I need.

There is research on AI limiting creative output in completive arenas. Essentially it breaks expectancy therefore deteriorates iteration.

https://direct.mit.edu/rest/article-abstract/102/3/583/96779...

Post reply on HN