Live data from Hacker News

A misalignment of AI in mathematics

mathandai.org

821–830 of 1001 posts

Re: A misalignment of AI in mathematics

#821
post #241

This sounds a lot to me like people in the 90's complaining that computers were destroying chess. Thirty years later, chess is more popular than it ever was, and chess players are better than they ever have been. I wouldn't be surprised if there are now more chess books now than there ever have been. Furthermore, it turns out that a lot of chess books written before computers were just wrong about a lot of things. It…

[flagged]

Re: A misalignment of AI in mathematics

#822

Earlier quoted context omitted.

I’m no big AI cheerleader, but why don’t you try these models to see what they offer - they might suggest some things you haven’t thought of, or save you some time in your research. Are you sure they’d one-shot your algorithm? Maybe they’d come up with worse, maybe better, but you are in a perfect position to actually judge the validity of any claims a LLM makes in this realm, which most people are not. As to why bot…

>> I’m no big AI cheerleader, but why don’t you try these models to see what they offer - they might suggest some things you haven’t thought of, or save you some time in your research. One reason not to try is to create an air gap between the OP's ideas and the data that future models can train on. We saw that the mathematicians who trusted OpenAI and Anthropic with their preliminary work found the rug pulled out of…

Unless you’re very weak-willed, I wouldn’t worry about independence. LLMs make mistakes all the time, they are nothing like an all-knowing oracle nor are they going to replace humans despite the absurd fantasies of LLM fans and those with a vested interest.

Plagiarism is an interesting point, though honestly I think it would be fairly easy to work out who had published similar research if the LLM gives you an idea - personally I see this as the weakest argument against using them, as long as you are strict about attribution - all work like this depends heavily on the research of others - the LLM is just a tool to aid that research IMO.

Opsec is a fair point, and it might be worth avoiding the completely amoral OpenAI at this point for that reason. There are open models though.

Re: A misalignment of AI in mathematics

#823

Earlier quoted context omitted.

Of course you're not going to get rich with the kind of software that LLMs can one shot these days. But that kind of software like to-do lists or basic CRUD have been saturated for over a decade, way before LLMs. People overestimate how much you can one shot, yeah a good prompt can get you 90% there but that 10% remaining often takes months of extra work. Software has always progressed this way, lots of devs back the…

But what does the human researcher do in this future? If they are not needed to understand the result, then what's their role? Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves? And how long will they be needed to ask the right questions, how long until AI can do that too? > Software has always progr…

They are needed to guide the research in fruitful directions and understand the result.

LlMs don’t understand, they generate, though they have fooled a lot of people who should know better.

Re: A misalignment of AI in mathematics

#824
post #278

Earlier quoted context omitted.

I wonder if they will continue to dump proofs, though? Their point has been made, the novelty will wear off, and it maybe won't be a priority use of their resources to spend however many millions on another big proof--they will move on to the next thing to show off I'm sure. At that point, the ones generating proofs will be, I hope, mathematicians (professional and otherwise) that are more interested in the results a…

They aren't going to stop at one, that's for sure. They already claimed they have "made substantial progress" on another millenium problem. Let's say they bag another one (Hodge and/or BSD according to the rumors), if it looks like their internal model could solve P/NP or Riemann Hypothesis, you think they wouldn't take that chance ?

If it's a counterexample to BSD, that would be pretty surprising.

It would also be a considerably more impressive achievement, because experts had mostly shifted to Navier-Stokes regularity being false, while as far as I know almost everybody thinks BSD is true. Hodge people seem less sure about.

If either conjecture is true and they prove it, that would be an even bigger success, since the techniques might unlock any number of other theorems.

Re: A misalignment of AI in mathematics

#825
post #441

Earlier quoted context omitted.

If it's worth anything: I have a PhD in (theoretical) mathematics and I entirely stand by stabbles' comment. There is a real, undeniable possibility of AI becoming better at mathematics in the same way that it became better at chess and Go, and in such a scenario, one may expect the community's response to be comparable.

Stockfish isn't owned by a club of three trillionaires. It does not cost $15 million to achieve a result in Stockfish. Stockfish does not steal research or scoop researchers. The concentration of computing resources and capital should be examined by the math community.

Note that there are no trillionaires anymore - spacex stock went down and musk „lost” a lot of money so rejoice, poor must be much better off now that we do not have any trillionaire.

Re: A misalignment of AI in mathematics

#827
post #663

Earlier quoted context omitted.

that is incredibly depressing

Yep. Perhaps humanity would be better off if we instituted and enforced the notion of "Thou shalt not make a machine in the likeness of a human mind." It's worth thinking about; just because we can build AI systems doesn't mean we should.

I would remind you that the fictional society that did this was also described as a feudal monarchy in which a tiny class of nobles exploits the economic output of trillions of people reduced to serfs by controlling all means of interstellar trade. Oh, and they also have legal slavery.

Re: A misalignment of AI in mathematics

#828

Earlier quoted context omitted.

Of course you're not going to get rich with the kind of software that LLMs can one shot these days. But that kind of software like to-do lists or basic CRUD have been saturated for over a decade, way before LLMs. People overestimate how much you can one shot, yeah a good prompt can get you 90% there but that 10% remaining often takes months of extra work. Software has always progressed this way, lots of devs back the…

But what does the human researcher do in this future? If they are not needed to understand the result, then what's their role? Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves? And how long will they be needed to ask the right questions, how long until AI can do that too? > Software has always progr…

> Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves?

The right questions are very simple to ask.

How do I get food. How to cure aging. How to turn lead into gold. How to fly high. What is the ultimate theory of physics. Are there any odd perfect numbers. Is there a soul.

We have reached complicated questions requiring deep knowledge because we tried to solve the simpler ones and reached obstacles. For example to solve alchemy we had to develop nuclear physics (and in the process we got chemistry). If one has a genie able to solve questions, you won't have to think about the complicated ones because the genie will.

Re: A misalignment of AI in mathematics

#829
post #668

Earlier quoted context omitted.

> Many graduate students (I know) are having a crisis if any of their research worth it? If AI can (or will) do everything, what's the point of doing experiments and all? This will eventually deter a whole generation of curious minded students from research. Those who think it's me or the machine will fail. Those who realize how much you can accelerate your research with the help of AI will succeed.

> Those who think it's me or the machine will fail. > Those who realize how much you can accelerate your research with the help of AI will succeed. This is only true up until a point. If I treat a mid-sized model (say, Qwen3.8 Flash Next) like a pair programmer, then yes, it accelerates my work. But I can already see the next stage with Fable: If I give it a couple of paragraphs of spec and $50, then I can just leave…

> But I don't expect AI to accelerate humans or improve our productivity for long. I can already see the first signs of a future where the AI doesn't need us for anything at all.

The real question is, why is this a bad thing?

Every task that is automated is a task that humans no longer have to do. It doesn't mean that humans still can't do it for reasons other than "because it needs to be done".

And if the answer is "why bother if X does it better", then what does it say about the motivations of doing it in the first place?

Re: A misalignment of AI in mathematics

#830
post #363

Earlier quoted context omitted.

It's still an optimistic scenario. Artificial superintelligence may develop hypermathematics of a kind that never will be accesible to human mind, enhanced or not. One can't teach geometry to ants even if you put them on a Moebius strip. It would be more like Lem's novel where it completely disappears from the human horizon: https://en.wikipedia.org/wiki/Golem_XIV

Why would this be optimistic? This sounds extremely negative to me…

optimistic in comparison to the alternative, where humans can't understand anything anymore.

if a particularly intelligent sixth grader was highly motivated to understand chromatic homotopy and had a highly capable private teacher available to her 24/7, she might within a year get to the point where she could apply it by herself to figure out some simple but nontrivial topological properties. (nobody has tested this. maybe it would require at least three years instead of one.)

if a particularly intelligent chimpanzee was for some reason highly motivated to understand chromatic homotopy, no matter how many years the most incredible teachers spent explaining it to her, she would never comprehend anything about it.

that's the pessimistic scenario: humans will be to future AI like our closest evolutionary relatives are to us. or even more pessimistic: we will be to future AI like insects are to us.

Post reply on HN