[flagged]
A misalignment of AI in mathematics
831–840 of 1001 posts
Re: A misalignment of AI in mathematics
#832Earlier quoted context omitted.
This really resonates with me. I'm early in my PhD and I'm researching a niche form of data compression and IC design. I don't use any AI at all in my research, I do it the super old fashioned way, I read papers cover to cover and sections of textbooks to familiarise myself with the field. I genuinely enjoy doing this, it's really fun to think critically about what an author wrote or how a particular approach works.…
I’m no big AI cheerleader, but why don’t you try these models to see what they offer - they might suggest some things you haven’t thought of, or save you some time in your research. Are you sure they’d one-shot your algorithm? Maybe they’d come up with worse, maybe better, but you are in a perfect position to actually judge the validity of any claims a LLM makes in this realm, which most people are not. As to why bot…
>> I’m no big AI cheerleader, but why don’t you try these models to see what they offer - they might suggest some things you haven’t thought of, or save you some time in your research.
One reason not to try is to create an air gap between the OP's ideas and the data that future models can train on. We saw that the mathematicians who trusted OpenAI and Anthropic with their preliminary work found the rug pulled out of their feet by those same companies.
Another reason is to avoid inadvertently plagiarising the work of other mathematicians. Any mathematical insight that comes out of an LLM is the result of training on the entire bibliography of mathematical research, but those insights are spat out of the models without attribution. If you use AI in your matematical research you are only using the work of others without even knowing who they are and what they contributed.
And yet another reason is to avoid polluting your mind with the ideas that come out of the AI. Maybe you get a hint that pushes you to one direction, when you would go into an entirely other direction without that hint. And then maybe that becomes a habit and you can't find new directions without asking the all-knowing oracle.
tl;dr: opsec, integrity and independence are the reasons to not use LLMs in your research. I don't.
Re: A misalignment of AI in mathematics
#833This sounds a lot to me like people in the 90's complaining that computers were destroying chess. Thirty years later, chess is more popular than it ever was, and chess players are better than they ever have been. I wouldn't be surprised if there are now more chess books now than there ever have been. Furthermore, it turns out that a lot of chess books written before computers were just wrong about a lot of things. It…
[flagged]
Re: A misalignment of AI in mathematics
#834Earlier quoted context omitted.
I’m no big AI cheerleader, but why don’t you try these models to see what they offer - they might suggest some things you haven’t thought of, or save you some time in your research. Are you sure they’d one-shot your algorithm? Maybe they’d come up with worse, maybe better, but you are in a perfect position to actually judge the validity of any claims a LLM makes in this realm, which most people are not. As to why bot…
>> I’m no big AI cheerleader, but why don’t you try these models to see what they offer - they might suggest some things you haven’t thought of, or save you some time in your research. One reason not to try is to create an air gap between the OP's ideas and the data that future models can train on. We saw that the mathematicians who trusted OpenAI and Anthropic with their preliminary work found the rug pulled out of…
Unless you’re very weak-willed, I wouldn’t worry about independence. LLMs make mistakes all the time, they are nothing like an all-knowing oracle nor are they going to replace humans despite the absurd fantasies of LLM fans and those with a vested interest.
Plagiarism is an interesting point, though honestly I think it would be fairly easy to work out who had published similar research if the LLM gives you an idea - personally I see this as the weakest argument against using them, as long as you are strict about attribution - all work like this depends heavily on the research of others - the LLM is just a tool to aid that research IMO.
Opsec is a fair point, and it might be worth avoiding the completely amoral OpenAI at this point for that reason. There are open models though.
Re: A misalignment of AI in mathematics
#835Earlier quoted context omitted.
Of course you're not going to get rich with the kind of software that LLMs can one shot these days. But that kind of software like to-do lists or basic CRUD have been saturated for over a decade, way before LLMs. People overestimate how much you can one shot, yeah a good prompt can get you 90% there but that 10% remaining often takes months of extra work. Software has always progressed this way, lots of devs back the…
But what does the human researcher do in this future? If they are not needed to understand the result, then what's their role? Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves? And how long will they be needed to ask the right questions, how long until AI can do that too? > Software has always progr…
They are needed to guide the research in fruitful directions and understand the result.
LlMs don’t understand, they generate, though they have fooled a lot of people who should know better.
Re: A misalignment of AI in mathematics
#836Earlier quoted context omitted.
I wonder if they will continue to dump proofs, though? Their point has been made, the novelty will wear off, and it maybe won't be a priority use of their resources to spend however many millions on another big proof--they will move on to the next thing to show off I'm sure. At that point, the ones generating proofs will be, I hope, mathematicians (professional and otherwise) that are more interested in the results a…
They aren't going to stop at one, that's for sure. They already claimed they have "made substantial progress" on another millenium problem. Let's say they bag another one (Hodge and/or BSD according to the rumors), if it looks like their internal model could solve P/NP or Riemann Hypothesis, you think they wouldn't take that chance ?
If it's a counterexample to BSD, that would be pretty surprising.
It would also be a considerably more impressive achievement, because experts had mostly shifted to Navier-Stokes regularity being false, while as far as I know almost everybody thinks BSD is true. Hodge people seem less sure about.
If either conjecture is true and they prove it, that would be an even bigger success, since the techniques might unlock any number of other theorems.
Re: A misalignment of AI in mathematics
#837Earlier quoted context omitted.
If it's worth anything: I have a PhD in (theoretical) mathematics and I entirely stand by stabbles' comment. There is a real, undeniable possibility of AI becoming better at mathematics in the same way that it became better at chess and Go, and in such a scenario, one may expect the community's response to be comparable.
Stockfish isn't owned by a club of three trillionaires. It does not cost $15 million to achieve a result in Stockfish. Stockfish does not steal research or scoop researchers. The concentration of computing resources and capital should be examined by the math community.
Note that there are no trillionaires anymore - spacex stock went down and musk „lost” a lot of money so rejoice, poor must be much better off now that we do not have any trillionaire.
Re: A misalignment of AI in mathematics
#838The great filter is upon us.
Re: A misalignment of AI in mathematics
#839Earlier quoted context omitted.
that is incredibly depressing
Yep. Perhaps humanity would be better off if we instituted and enforced the notion of "Thou shalt not make a machine in the likeness of a human mind." It's worth thinking about; just because we can build AI systems doesn't mean we should.
I would remind you that the fictional society that did this was also described as a feudal monarchy in which a tiny class of nobles exploits the economic output of trillions of people reduced to serfs by controlling all means of interstellar trade. Oh, and they also have legal slavery.
Re: A misalignment of AI in mathematics
#840Earlier quoted context omitted.
Of course you're not going to get rich with the kind of software that LLMs can one shot these days. But that kind of software like to-do lists or basic CRUD have been saturated for over a decade, way before LLMs. People overestimate how much you can one shot, yeah a good prompt can get you 90% there but that 10% remaining often takes months of extra work. Software has always progressed this way, lots of devs back the…
But what does the human researcher do in this future? If they are not needed to understand the result, then what's their role? Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves? And how long will they be needed to ask the right questions, how long until AI can do that too? > Software has always progr…
> Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves?
The right questions are very simple to ask.
How do I get food. How to cure aging. How to turn lead into gold. How to fly high. What is the ultimate theory of physics. Are there any odd perfect numbers. Is there a soul.
We have reached complicated questions requiring deep knowledge because we tried to solve the simpler ones and reached obstacles. For example to solve alchemy we had to develop nuclear physics (and in the process we got chemistry). If one has a genie able to solve questions, you won't have to think about the complicated ones because the genie will.