Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can AI do maths yet? Thoughts from a mathematician

xenaproject.wordpress.com

Post reply on HN
Print view View original

331–340 of 364 posts

Re: Can AI do maths yet? Thoughts from a mathematician

#331

Post by YeGoblynQueenne »

>> There were language models before ChatGPT, and on the whole they couldn’t even write coherent sentences and paragraphs. ChatGPT was really the first public model which was coherent.

If that's referring to Large Language Models, meaning everything after the fist GPT and BERT, then that's absolutely not right. The first LLM that demonstrated the ability to generate coherent, fluently grammatical English was GPT-2. That story about the unicorns- that was the first time a statistical language model was able to generate text that stayed on the subject over a long distance and made (some) sense.

GPT-2 was followed by GPT 3 and GPT 3.5 that turned the hype dial up to 11 and were certainly "public" at least if that means publicly available. They were coherent enough that many people predicted all sorts of fancy things, like the end of programming jobs and the end of journalist jobs and so on.

So, weird statement that one and it kind of makes me wary of Gell-Mann amnesia while reading the article.

Re: Can AI do maths yet? Thoughts from a mathematician

#332

Post by dinosaurdynasty »

staunton wrote: post #223
dinosaurdynasty wrote: post #164

Earlier quoted context omitted.

Quantum computing actually isn't super-Turing, it "just" computes some things faster. (Strictly speaking it's somewhere between a standard Turing machine and a nondeterministic Turing machine in speed, and the first can emulate the second.)

If we're nitpicking: quantum computing algorithms could (if implemented) compute certain things faster than the best classical algorithms we know . We don't know any quantum algorithms that are provably faster than all possible classical algorithms.

Well yeah, we haven't even proved that P != NP yet.

Re: Can AI do maths yet? Thoughts from a mathematician

#333

Post by amanda99 »

portaouflop wrote: post #271
amanda99 wrote: post #266

Earlier quoted context omitted.

> AI is often wrong, never knows when it's wrong, but people are like this too. When talking with various models of ChatGPT about research math, my biggest gripe is that it's either confidently right (10% of my work) or confidently wrong (90%). A human researcher would be right 15% of the time, unsure 50% of the time, and give helpful ideas that are right/helpful (25%) or wrong/a red herring (10%). And only 5% of the…

A human researcher that is basically right 40%-95% of the time would probably an Einstein level genius. Just assume that the LLM is wrong and test their assumptions - math is one of the few disciplines where you can do that easily

I think you are imagining a different class of "questions".

To clarify, I was doing research on applied math. My field is not analysis, but I needed to prove some bounds on certain messed up expressions (involving special functions, etc), and analyze an ODE that's not analytically solvable. I used the COT model a fair bit.

I would ask ChatGPT for hints/ideas/direction in proving various bounds, asking it for theorems or similar results in literature. This is exactly the kind of thing where a researcher would go "yeah this looks like X" or "I think I saw something like this in (book/article name)", or just know a method; or alternatively say they have no clue. ChatGPT most often will confidently give me a "solution", being right 10% of the time (when there's a pretty standard way to do it that I didn't see/know).

On the whole it was quite useful.

Re: Can AI do maths yet? Thoughts from a mathematician

#334

Post by amanda99 »

halayli wrote: post #280
amanda99 wrote: post #266

Earlier quoted context omitted.

> AI is often wrong, never knows when it's wrong, but people are like this too. When talking with various models of ChatGPT about research math, my biggest gripe is that it's either confidently right (10% of my work) or confidently wrong (90%). A human researcher would be right 15% of the time, unsure 50% of the time, and give helpful ideas that are right/helpful (25%) or wrong/a red herring (10%). And only 5% of the…

these numbers are just your perception. The way you ask the question will very much influence the output and certain topics more than others. I get much better results when I share my certainty levels in my questions and say things like "if at all", "if any" etc.

> these numbers are just your perception.

Of course they are, I hoped it was clear I was just sharing my experience trying to use it for research!

I did in general word it as I would a question to a researcher, which includes an uncertainty in it being true. E.g. this is from a recent prompt: "is this true in general, if not, what are the conditions for this to be true?"

Re: Can AI do maths yet? Thoughts from a mathematician

#335

Post by Quekid5 »

exprofmaddy wrote: post #315
Quekid5 wrote: post #181

Earlier quoted context omitted.

> If someone believes the world is purely mechanistic, then it follows that a sufficiently large computing machine can model the world Is this controversial in some way? The problem is that to simulate a universe you need a bigger universe -- which doesn't exist (or is certainly out of reach due to information theoretical limits) > ---like Leibniz's Ratiocinator. The intoxication may stem from the potential for predi…

People wish to feel safe. One path to safety is controlling or managing the environment. Lack of sufficient control produces anxiety. But control is only possible if the environment is predictable, i.e., relatively certain knowledge that if I do X then the environment responds with Y. Humans use models for prediction. Loosely speaking, if the universe is truly mechanistic/deterministic, then the goal of modeling is t…

> However, if we can't know whether the universe is truly deterministic, then modeling is a pragmatic exercise in control (or management).

What would you say if we can predict the outcome of an experiment with 51% probability. Is that enough to establish what you call "control"? What if we can repeat the experiment as many times as we like?

(I must admit, I still don't really understand what "control" means to you, but let's get the preliminaries out of the way first.)

Re: Can AI do maths yet? Thoughts from a mathematician

#336

Post by Quekid5 »

tananan wrote: post #198
Quekid5 wrote: post #181

Earlier quoted context omitted.

> If someone believes the world is purely mechanistic, then it follows that a sufficiently large computing machine can model the world Is this controversial in some way? The problem is that to simulate a universe you need a bigger universe -- which doesn't exist (or is certainly out of reach due to information theoretical limits) > ---like Leibniz's Ratiocinator. The intoxication may stem from the potential for predi…

> Is this controversial in some way? It’s not “controversial”, it’s just not a given that the universe is to be thought a deterministic machine. Not to everyone, at least.

That's fine and well, but AFAICT the only alternative is it being non-deterministic... which doesn't seem very satisfactory either.

Re: Can AI do maths yet? Thoughts from a mathematician

#337

Post by est »

int_19h wrote: post #210
Someone wrote: post #187

Earlier quoted context omitted.

Why is that unfair in reply to the claim “At this stage I assume everything having a sequencial pattern can and will be automated by LLM AIs.” ? I am not claiming LLMs aren’t or cannot be intelligent, not even that they cannot do magical things; I just rebuked a statement about the lack of limits of LLMs. > Naturally, humans couldn’t do it, even though they could edit the input to remove the X’s So, what are you clai…

If you have a million Xs on the end of each line, when a human is looking at that file, he's not looking at the entirety of it, but only at the part that is actually visible on-screen, so the equivalent task for an LLM would be to feed it the same subset as input. In which case they can all answer this question just fine.

> If you have a million Xs on the end of each line

Hmm, I wonder if adding a compression layer during encoding helps?

Re: Can AI do maths yet? Thoughts from a mathematician

#338

Post by xmprt »

sumeno wrote: post #314
xmprt wrote: post #245

Earlier quoted context omitted.

The incentives are definitely there, but even CEOs and VCs know that if they cheat the tests just to get more investment, they're only cheating themselves. No one is liquidating within the next 5 years so either they end up getting caught and lose everything or they spent all this energy trying to cheat while having a subpar model which results in them losing to competitors who actually invested in good technology. H…

Why is this any different from say, Theranos? CEOs and VCs will happily lie because they are convinced they are smarter than everyone else and will solve the problem before they get caught.

Theranos didn't have 10 different competitors doing the exact same thing. A new AI model which scores better on a random metric isn't going to suddenly make them the top model that everyone uses unless they're actually good. So while Theranos cheating would help put them in stores like CVS, an AI company cheating would just mean that they make a few sales before everyone realizes that their model is actually pretty bad compared to all the competitors.

Re: Can AI do maths yet? Thoughts from a mathematician

#339

Post by raincole »

vouaobrasil wrote: post #290
raincole wrote: post #244

Earlier quoted context omitted.

> Now most will just use some AI. Do people with PhD in math really ask AI to explain math concepts to them?

They will, when it becomes good enough to prove tricky things.

The parent comment said:

> Now most will just use some AI.

I'm genuninely wondering whether it's true. The "now" and "most" part.

Re: Can AI do maths yet? Thoughts from a mathematician

#340

Post by gldjmp »

aomix wrote: post #298

No comment on the article it's just always interesting to get hit with intense jargon from a field I know very little about. I understood the statements of all five questions. I could do the third one relatively quickly (I had seen the trick before that the function mapping a natural n to alpha^n was p-adically continuous in n iff the p-adic valuation of alpha-1 was positive)

Haha, the thing about jargon is that is typically hiding something not so bad. (At least in this case, the solution is something you could explain to a high-schooler).
Post reply on HN