Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can AI do maths yet? Thoughts from a mathematician

xenaproject.wordpress.com

Post reply on HN
Print view View original

181–190 of 364 posts

Re: Can AI do maths yet? Thoughts from a mathematician

#181

Post by Quekid5 »

exprofmaddy wrote: post #143
tananan wrote: post #89

Earlier quoted context omitted.

> Unless you believe in determinism and an overseeing god Or perhaps, determinism and mechanistic materialism - which in STEM-adjacent circles has a relatively prevalent adherence. Worldviews which strip a human being of agency in the sense you invoke crop up quite a lot today in such spaces. If you start of adopting a view like this, you have a deflationary sword which can cut down most any notion that's not mechani…

Agreed. If someone believes the world is purely mechanistic, then it follows that a sufficiently large computing machine can model the world---like Leibniz's Ratiocinator. The intoxication may stem from the potential for predictability and control. The irony is: why would someone want control if they don't have true choice? Unfortunately, such a question rarely pierces the intoxicated mind when this mind is preoccupi…

> If someone believes the world is purely mechanistic, then it follows that a sufficiently large computing machine can model the world

Is this controversial in some way? The problem is that to simulate a universe you need a bigger universe -- which doesn't exist (or is certainly out of reach due to information theoretical limits)

> ---like Leibniz's Ratiocinator. The intoxication may stem from the potential for predictability and control.

I really don't understand the 'control' angle here. It seems pretty obvious that even in a purely mechanistic view of the universe, information theory forbids using the universe to simulate itself. Limited simulations, sure... but that leaves lots of gaps wherein you lose determinism (and control, whatever that means).

Re: Can AI do maths yet? Thoughts from a mathematician

#182

Post by [deleted] »

amelius wrote: post #125
jampekka wrote: post #19

I just spent a few days trying to figure out some linear algebra with the help of ChatGPT. It's very useful for finding conceptual information from literature (which for a not-professional-mathematician at least can be really hard to find and decipher). But in the actual math it constantly makes very silly errors. E.g. indexing a vector beyond its dimension, trying to do matrix decomposition for scalars and insisting…

Don't most mathematical papers contain at least one such error?

[deleted]

Re: Can AI do maths yet? Thoughts from a mathematician

#183

Post by CamperBob2 »

xienze wrote: post #152
thomashop wrote: post #110

Earlier quoted context omitted.

Ask it to plot the graph with python plotting utilities. Not using its image generator. I think you need a ChatGPT subscription though for it to be able to run python code.

Shouldn’t ChatGPT be smart enough to know to do this automatically, based on context?

It was, for a while. I think this is an area where there may have been some regression. It can still write code to solve problems that are a poor fit for the language model, but you may need to ask it to do that explicitly.

Re: Can AI do maths yet? Thoughts from a mathematician

#184

Post by CamperBob2 »

bongodongobob wrote: post #158
GuB-42 wrote: post #136

Earlier quoted context omitted.

Wolfram Alpha can solve equations well, but it is terrible at understanding natural language. For example I asked Wolfram Alpha "How heavy a rocket has to be to launch 5 tons to LEO with a specific impulse of 400s", which is a straightforward application of the Tsiolkovsky rocket equation. Wolfram Alpha gave me some nonsense about particle physics (result: 95 MeV/c^2), GPT-4o did it right (result: 53.45 tons). Wolfra…

You're using it wrong, you can use natural language in your equation, but afaik it's not supposed to be able to do what you're asking of it.

You know, "You're using it wrong" is usually meant to carry an ironic or sarcastic tone, right?

It dates back to Steve Jobs blaming an iPhone 4 user for "holding it wrong" rather than acknowledging a flawed antenna design that was causing dropped calls. The closest Apple ever came to admitting that it was their problem was when they subsequently ran an employment ad to hire a new antenna engineering lead. Maybe it's time for Wolfram to hire a new language-model lead.

Re: Can AI do maths yet? Thoughts from a mathematician

#185

Post by busyant »

TheRealPomax wrote: post #180
busyant wrote: post #138

As someone who has a 18 yo son who wants to study math, this has me (and him) ... worried ... about becoming obsolete? But I'm wondering what other people think of this analogy. I used to be a bench scientist (molecular genetics). There were world class researchers who were more creative than I was. I even had a Nobel Laureate once tell me that my research was simply "dotting 'i's and crossing 't's". Nevertheless, I…

What part do you think is going to become obsolete? Because Math isn't about "working out the math", it's about finding the relations between seemingly unrelated things to bust open a problem. Short of AGI, there is no amount of neural net that's going to realize that a seemingly impossible probabilistic problem is actually equivalent to a projection of an easy to work with 4D geometry. "Doing the math" is what we ha…

> What part do you think is going to become obsolete?

Thank you for the question.

I guess what I'm saying is:

Will LLMs (or whatever comes after them) be _so_ good and _so_ pervasive that we will simply be able to say, "Hey ChatGPT-9000, I'd like to see if the xyz conjecture is correct." And then ChatGPT-9000 just does the work without us contributing beyond asking a question.

Or will the technology be limited/bound in some way such that we will still be able to use ChatGPT-9000 as a tool of our own intellectual augmentation and/or we could still contribute to research even without it.

Hopefully, my comment clarifies my original post.

Also, writing this stuff has helped me think about it more. I don't have any grand insight, but the more I write, the more I lean toward the outcome that these machines will allow us to augment our research.

Re: Can AI do maths yet? Thoughts from a mathematician

#186

Post by hyhconito »

busyant wrote: post #138

As someone who has a 18 yo son who wants to study math, this has me (and him) ... worried ... about becoming obsolete? But I'm wondering what other people think of this analogy. I used to be a bench scientist (molecular genetics). There were world class researchers who were more creative than I was. I even had a Nobel Laureate once tell me that my research was simply "dotting 'i's and crossing 't's". Nevertheless, I…

Let's put it this way, from another mathematician, and I'm sure I'll probably be shot for this one.

Every LLM release moves half of the remaining way to the minimum viable goal of replacing a third class undergrad. If your business or research initiative is fine with that level of competence then you will find utility.

The problem is that I don't know anyone who would find that useful. Nor does it fit within any existing working methodology we have. And on top of that the verification of any output can take considerably longer than just doing it yourself in the first place, particularly where it goes off the rails, which it does all the time. I mean it was 3 months ago I was arguing with a model over it not understanding place-value systems properly, something we teach 7 year olds here?

But the abstract problem is at a higher level. If it doesn't become a general utility for people outside of mathematics, which is very very evident at the moment by the poor overall adoption and very public criticism of the poor result quality, then the funding will dry up. Models cost lots of money to train and if you don't have customers it's not happening and no one is going to lend you the money any more. And then it's moot.

Re: Can AI do maths yet? Thoughts from a mathematician

#187

Post by Someone »

beng-nl wrote: post #99
Someone wrote: post #28

Earlier quoted context omitted.

I think that’s provably incorrect for the current approach to LLMs. They all have a horizon over which they correlate tokens in the input stream. So, for any LLM, if you intersperse more than that number of ‘X’ tokens between each useful token, they won’t be able to do anything resembling intelligence. The current LLMs are a bit like n-gram databases that do not use letters, but larger units.

It’s that a bit of an unfair sabotage? Naturally, humans couldn’t do it, even though they could edit the input to remove the X’s, but shouldn’t we evaluate the ability (even intelligent ability) of LLM’s on what they can generally do rather than amplify their weakness?

Why is that unfair in reply to the claim “At this stage I assume everything having a sequencial pattern can and will be automated by LLM AIs.”?

I am not claiming LLMs aren’t or cannot be intelligent, not even that they cannot do magical things; I just rebuked a statement about the lack of limits of LLMs.

> Naturally, humans couldn’t do it, even though they could edit the input to remove the X’s

So, what are you claiming: that they cannot or that they can? I think most people can and many would. Confronted with a file containing millions of X’s, many humans will wonder whether there’s something else than X’s in the file, do a ‘replace all’, discover the question hidden in that sea of X’s, and answer it.

There even are simple files where most humans would easily spot things without having to think of removing those X's. Consider a file

   How         X X X X X X
   many        X X X X X X
   days        X X X X X X
   are         X X X X X X
   there       X X X X X X
   in          X X X X X X
   a           X X X X X X
   week?       X X X X X X
with a million X’s on the end of each line. Spotting the question in that is easy for humans, but impossible for the current bunch of LLMs

Re: Can AI do maths yet? Thoughts from a mathematician

#188

Post by voiper1 »

seafoamteal wrote: post #48

I don't have much to opine from an advanced maths perspective, but I'd like to point out a couple examples of where ChatGPT made basic errors in questions I asked it as an undergrad CS student. 1. I asked it to show me the derivation of a formula for the efficiency of Stop-and-Wait ARQ and it seemed to do it, but a day later, I realised that in one of the steps, it just made a term vanish to get to the next step. Obv…

The first question is always: which model? Which fortunately you at least addressed: >free plan of ChatGPT, but I can remember if it was 4o or 4.

Since chatgpt-4o, there has been o1-preview, and o1 (full) is out. They just announced o3 got a 25% on frontiermath which is what this article is a reaction to. So, any tests on 4o are at least TWO (or three) AI releases with new capabilities.

Re: Can AI do maths yet? Thoughts from a mathematician

#189

Post by binarymax »

hyhconito wrote: post #186
busyant wrote: post #138

As someone who has a 18 yo son who wants to study math, this has me (and him) ... worried ... about becoming obsolete? But I'm wondering what other people think of this analogy. I used to be a bench scientist (molecular genetics). There were world class researchers who were more creative than I was. I even had a Nobel Laureate once tell me that my research was simply "dotting 'i's and crossing 't's". Nevertheless, I…

Let's put it this way, from another mathematician, and I'm sure I'll probably be shot for this one. Every LLM release moves half of the remaining way to the minimum viable goal of replacing a third class undergrad. If your business or research initiative is fine with that level of competence then you will find utility. The problem is that I don't know anyone who would find that useful. Nor does it fit within any exis…

This is a great point that nobody will shoot you over :)

But the main question is still: assuming you replace an undergrad with a model, who checks the work? If you have a good process around that already, and find utility as an augmented system, then get you’ll get value - but I still think it’s better for the undergrad to still have the job and be at the wheel, and does things faster and better when leveraging a powerful tool.

Re: Can AI do maths yet? Thoughts from a mathematician

#190

Post by HDThoreaun »

exprofmaddy wrote: post #143
tananan wrote: post #89

Earlier quoted context omitted.

> Unless you believe in determinism and an overseeing god Or perhaps, determinism and mechanistic materialism - which in STEM-adjacent circles has a relatively prevalent adherence. Worldviews which strip a human being of agency in the sense you invoke crop up quite a lot today in such spaces. If you start of adopting a view like this, you have a deflationary sword which can cut down most any notion that's not mechani…

Agreed. If someone believes the world is purely mechanistic, then it follows that a sufficiently large computing machine can model the world---like Leibniz's Ratiocinator. The intoxication may stem from the potential for predictability and control. The irony is: why would someone want control if they don't have true choice? Unfortunately, such a question rarely pierces the intoxicated mind when this mind is preoccupi…

Choice is over rated. This gets to an issue Ive long had with Nozicks experience machine. Not only would I happily spend my days in such a machine, Im pretty sure most other people would too. Maybe they say they wouldnt but if you let them try it out and then offered them the question again I think theyd say yes. The real conclusion of the experience machine is that the unknown is scary.
Post reply on HN