Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can AI do maths yet? Thoughts from a mathematician

xenaproject.wordpress.com

Post reply on HN
Print view View original

301–310 of 364 posts

Re: Can AI do maths yet? Thoughts from a mathematician

#301

Post by noFaceDiscoG668 »

tossandthrow wrote: post #11
noFaceDiscoG668 wrote: post #5

Earlier quoted context omitted.

I tried. I don't have the time to formulate and scrutinise adequate arguments, though. Do you? Anything anywhere you could point me to? The algorithms live entirely off the training data. They consistently fail to "abduct" (inference) beyond any language-in/of-the-training-specific information.

It is a gradual thing. Presumably the models are inferring things on runtime that was not a part of their training data. Anyhow, philosophically speaking you are also only exposed to what your senses pick up, but presumably you are able to infer things? As written: this is a dogma that stems from a limited understanding of what algorithmic processes are and the insistence that emergence can not happen from algorithmi…

[dead]

Re: Can AI do maths yet? Thoughts from a mathematician

#302

Post by earnestinger »

xmprt wrote: post #245
phkahler wrote: post #217

Earlier quoted context omitted.

>> It's not about training directly on the test set, it's about people discussing questions in the test set online Don't kid yourself. There are 10's of billions of dollars going into AI. Some of the humans involved would happily cheat on comparative tests to boost investment.

The incentives are definitely there, but even CEOs and VCs know that if they cheat the tests just to get more investment, they're only cheating themselves. No one is liquidating within the next 5 years so either they end up getting caught and lose everything or they spent all this energy trying to cheat while having a subpar model which results in them losing to competitors who actually invested in good technology. H…

People like to cheat. See the VW case. Company is big and established and still cheated.

It depends a lot on individuals making up the companies command chain and their values.

Re: Can AI do maths yet? Thoughts from a mathematician

#303

Post by mrbungie »

halayli wrote: post #280
amanda99 wrote: post #266

Earlier quoted context omitted.

> AI is often wrong, never knows when it's wrong, but people are like this too. When talking with various models of ChatGPT about research math, my biggest gripe is that it's either confidently right (10% of my work) or confidently wrong (90%). A human researcher would be right 15% of the time, unsure 50% of the time, and give helpful ideas that are right/helpful (25%) or wrong/a red herring (10%). And only 5% of the…

these numbers are just your perception. The way you ask the question will very much influence the output and certain topics more than others. I get much better results when I share my certainty levels in my questions and say things like "if at all", "if any" etc.

Yeah, blame the users for "using it wrong" (phrase of the week I would say after the o3 discussions), and then sell the solution as almost-AGI.

PS: I'm starting to see a lot of plausible deniability in some comments about LLMs capabilites. When LLMs do great => "cool, we are scaling AI". when LLMs do something wrong => "user problem", "skill issues", "don't judge a fish for its ability to fly".

Re: Can AI do maths yet? Thoughts from a mathematician

#304

Post by pontus »

Syzygies wrote: post #247

"Can AI do math for us" is the canonical wrong question. People want self-driving cars so they can drink and watch TV. We should crave tools that enhance our abilities, as tools have done since prehistoric times. I'm a research mathematician. In the 1980's I'd ask everyone I knew a question, and flip through the hard bound library volumes of Mathematical Reviews, hoping to recognize something. If I was lucky, I'd get…

I agree. I think it comes down to the motivation behind why one does mathematics (or any other field for that matter). If it's a means to an end, then sure have the AI do the work and get rid of the researchers. However, that's not why everyone does math. For many it's more akin to why an artist paints. People still paint today even though a camera can produce much more realistic images. It was probably the case (I'm guessing!) that there was a significant drop in jobs for artists-for-hire, for whom painting was just a means to an end (e.g. creating a portrait), but the artists who were doing it for the sake of art survived and were presumably made better by the ability to see photos of other places they want to paint or art from other artists due to the invention of the camera.

Re: Can AI do maths yet? Thoughts from a mathematician

#305

Post by whimsicalism »

didibus wrote: post #251
Syzygies wrote: post #247

"Can AI do math for us" is the canonical wrong question. People want self-driving cars so they can drink and watch TV. We should crave tools that enhance our abilities, as tools have done since prehistoric times. I'm a research mathematician. In the 1980's I'd ask everyone I knew a question, and flip through the hard bound library volumes of Mathematical Reviews, hoping to recognize something. If I was lucky, I'd get…

I think I'm missing your point? You still want to enjoy doing math yourself? Is that what you are saying? So you equate "Can AI do math in my place?" with "Can AI drink and watch TV in my place?"

Ingredients to a top HN comment on AI include some nominal expert explaining why actually labor won’t be replaced and it will be a collaborative process so you don’t need to worry sprinkled with a little bit of ‘the status quo will stay still even though this tech only appeared in the last 2 years’

Re: Can AI do maths yet? Thoughts from a mathematician

#306

Post by whimsicalism »

zifpanachr23 wrote: post #195
joe_the_user wrote: post #173

Earlier quoted context omitted.

The reddit thread is ... interesting (direct link[1]). It seems to be a debate among mathematicians some of whom do have access to the secret set. But they're debating publicly and so naturally avoiding any concrete examples that would give the set away so wind-up with fuzzy-fiddly language for the qualities of the problem tiers. The "reality" of keeping this stuff secret 'cause someone would train on it is itself bi…

Not having access to the dataset really makes the whole thing seem incredibly shady. Totally valid questions you are raising

it’s a key aspect of the entire project. we have gone through many cycles of evils where the dataset is public

Re: Can AI do maths yet? Thoughts from a mathematician

#307

Post by TheRealPomax »

busyant wrote: post #185
TheRealPomax wrote: post #180

Earlier quoted context omitted.

What part do you think is going to become obsolete? Because Math isn't about "working out the math", it's about finding the relations between seemingly unrelated things to bust open a problem. Short of AGI, there is no amount of neural net that's going to realize that a seemingly impossible probabilistic problem is actually equivalent to a projection of an easy to work with 4D geometry. "Doing the math" is what we ha…

> What part do you think is going to become obsolete? Thank you for the question. I guess what I'm saying is: Will LLMs (or whatever comes after them) be _so_ good and _so_ pervasive that we will simply be able to say, "Hey ChatGPT-9000, I'd like to see if the xyz conjecture is correct." And then ChatGPT-9000 just does the work without us contributing beyond asking a question. Or will the technology be limited/bound…

As amazing as they may seem, they're still just autocompletes, it's inherent to what an LLM is. So unless we come up with a completely new kind technology, I don't see "test this conjecture for me" becoming more real than the computer assisted proof tooling we already have.

Re: Can AI do maths yet? Thoughts from a mathematician

#308

Post by FiberBundle »

whimsicalism wrote: post #305
didibus wrote: post #251

Earlier quoted context omitted.

I think I'm missing your point? You still want to enjoy doing math yourself? Is that what you are saying? So you equate "Can AI do math in my place?" with "Can AI drink and watch TV in my place?"

Ingredients to a top HN comment on AI include some nominal expert explaining why actually labor won’t be replaced and it will be a collaborative process so you don’t need to worry sprinkled with a little bit of ‘the status quo will stay still even though this tech only appeared in the last 2 years’

It didn't appear in the last two years. We have had deep learning based autoregressive language models (like Word2Vec) for at least 10 years.

Re: Can AI do maths yet? Thoughts from a mathematician

#309

Post by exprofmaddy »

fire_lake wrote: post #300
exprofmaddy wrote: post #143

Earlier quoted context omitted.

Agreed. If someone believes the world is purely mechanistic, then it follows that a sufficiently large computing machine can model the world---like Leibniz's Ratiocinator. The intoxication may stem from the potential for predictability and control. The irony is: why would someone want control if they don't have true choice? Unfortunately, such a question rarely pierces the intoxicated mind when this mind is preoccupi…

> Agreed. If someone believes the world is purely mechanistic, then it follows that a sufficiently large computing machine can model the world---like Leibniz's Ratiocinator. I don’t think it does. Taking computers as an analogy… if you have a computer with 1GB memory, then you can’t simulate a computer with more than 1GB memory inside of it.

"sufficiently large machine" ... It's a thought experiment. Leibniz didn't have a computer, but he still imagined it.

Re: Can AI do maths yet? Thoughts from a mathematician

#310

Post by jvvw »

busyant wrote: post #138

As someone who has a 18 yo son who wants to study math, this has me (and him) ... worried ... about becoming obsolete? But I'm wondering what other people think of this analogy. I used to be a bench scientist (molecular genetics). There were world class researchers who were more creative than I was. I even had a Nobel Laureate once tell me that my research was simply "dotting 'i's and crossing 't's". Nevertheless, I…

Another PhD in maths here and I would say not to worry. It's the process of doing and understanding mathematics, and thinking mathematically that is ultimately important.

There's never been the equivalent of the 'bench scientist' in mathematics and there aren't many direct careers in mathematics, or pure mathematics at least - so very few people ultimately become researchers. Instead, I think you take your way of thinking and apply it to whatever else you do (and it certainly doesn't do any harm to understand various mathematical concepts incredibly well).

Post reply on HN