Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI for math resources, and erdosproblems.com

terrytao.wordpress.com

Post reply on HN
Print view View original

11–20 of 39 posts

Re: AI for math resources, and erdosproblems.com

#11

Post by mindcrime »

benreesman wrote: post #6
upmind wrote: post #2

What does AI for Math mean? People doing Math research that want to transition to AI research?

I’m generally a bit of an LLM skeptic, at least as concerns extravagant claims around “AGI” or self-improvement of any of that nonsense. But there are activities where an inherently high-temperature assistant (e.g. some Instruct-inspired tune) are a great fit, and those are almost definitionally at the boundary of “objectively faithful” and “a bit stochastic”. I’ve never done any work in novel mathematics myself, but…

I’m generally a bit of an LLM skeptic, at least as concerns extravagant claims around “AGI” or self-improvement of any of that nonsense.

Of all the highly optimistic applications for 2024 LLMs, mathematics sounds pretty plausible?

You probably already know this, but I think it bears pointing out explicitly: in this context we're not talking (only) about LLM's. There's a LOT more to AI than just Large Language Models, and the list of resources linked reflects that.

Re: AI for math resources, and erdosproblems.com

#12

Post by adroniser »

kevinventullo wrote: post #9
benreesman wrote: post #6

Earlier quoted context omitted.

I’m generally a bit of an LLM skeptic, at least as concerns extravagant claims around “AGI” or self-improvement of any of that nonsense. But there are activities where an inherently high-temperature assistant (e.g. some Instruct-inspired tune) are a great fit, and those are almost definitionally at the boundary of “objectively faithful” and “a bit stochastic”. I’ve never done any work in novel mathematics myself, but…

FWIW as a former mathematician, I believe that if LLM’s can crack mathematics in the sense of say, solving the Riemann Hypothesis through chains of logical reasoning, then we’d be in the back half of reaching “AGI” in the sense of solving broad classes of general problems like drug discovery, or getting fusion to work, or even improving itself.

The curious thing is if you can ever hit a snowball type point, because by proving theorems you are generating training data, and maybe you can get to a point where you effectively have limitless data.

Re: AI for math resources, and erdosproblems.com

#13

Post by riku_iki »

cvoss wrote: post #3
upmind wrote: post #2

What does AI for Math mean? People doing Math research that want to transition to AI research?

Broadly, as you'll see in the Google doc linked from the announcement, they are referring to the use of computer systems to assist mathematical research, which falls into a two step process: 1) Identify a suitable language for use by both humans and computers for communication and manipulation of mathematical theorems and their proofs. 2) Employ both mathematicians and machine learning algorithms to generate interest…

> as you'll see in the Google doc linked from the announcement, they are referring to the use of computer systems to assist mathematical research, which falls into a two step process

I reviewed the doc, and it is just curated list of links on various projects (ML frameworks, proof assistant, some tutorials), and does not outline plan you described here. Or I missed something?..

Re: AI for math resources, and erdosproblems.com

#14

Post by quonn »

kevinventullo wrote: post #9
benreesman wrote: post #6

Earlier quoted context omitted.

I’m generally a bit of an LLM skeptic, at least as concerns extravagant claims around “AGI” or self-improvement of any of that nonsense. But there are activities where an inherently high-temperature assistant (e.g. some Instruct-inspired tune) are a great fit, and those are almost definitionally at the boundary of “objectively faithful” and “a bit stochastic”. I’ve never done any work in novel mathematics myself, but…

FWIW as a former mathematician, I believe that if LLM’s can crack mathematics in the sense of say, solving the Riemann Hypothesis through chains of logical reasoning, then we’d be in the back half of reaching “AGI” in the sense of solving broad classes of general problems like drug discovery, or getting fusion to work, or even improving itself.

Science needs experiments and those will not go faster because of AI. So why would fusion arrive any faster?

Re: AI for math resources, and erdosproblems.com

#15

Post by benreesman »

mindcrime wrote: post #11
benreesman wrote: post #6

Earlier quoted context omitted.

I’m generally a bit of an LLM skeptic, at least as concerns extravagant claims around “AGI” or self-improvement of any of that nonsense. But there are activities where an inherently high-temperature assistant (e.g. some Instruct-inspired tune) are a great fit, and those are almost definitionally at the boundary of “objectively faithful” and “a bit stochastic”. I’ve never done any work in novel mathematics myself, but…

I’m generally a bit of an LLM skeptic, at least as concerns extravagant claims around “AGI” or self-improvement of any of that nonsense. Of all the highly optimistic applications for 2024 LLMs, mathematics sounds pretty plausible? You probably already know this, but I think it bears pointing out explicitly: in this context we're not talking (only) about LLM's. There's a LOT more to AI than just Large Language Models,…

I’m an advocate for shifting resources from pure-scale transformer work to other promising avenues in both representation and reinforcement learning.

Re: AI for math resources, and erdosproblems.com

#17

Post by kevinventullo »

quonn wrote: post #14
kevinventullo wrote: post #9

Earlier quoted context omitted.

FWIW as a former mathematician, I believe that if LLM’s can crack mathematics in the sense of say, solving the Riemann Hypothesis through chains of logical reasoning, then we’d be in the back half of reaching “AGI” in the sense of solving broad classes of general problems like drug discovery, or getting fusion to work, or even improving itself.

Science needs experiments and those will not go faster because of AI. So why would fusion arrive any faster?

Yeah I did say “back half”. I just think that if we can build an AI that can logically reason better than any human, combined with the total knowledge of all humans, it can start running experiments and doing “science” itself.

Re: AI for math resources, and erdosproblems.com

#18

Post by benreesman »

kevinventullo wrote: post #17
quonn wrote: post #14

Earlier quoted context omitted.

Science needs experiments and those will not go faster because of AI. So why would fusion arrive any faster?

Yeah I did say “back half”. I just think that if we can build an AI that can logically reason better than any human, combined with the total knowledge of all humans, it can start running experiments and doing “science” itself.

In general I think I agree with all your points.

The word “if” in both the parent and GP is doing a lot of lifting here though, if they can do amazing feat X, then they will likewise do amazing feat Y. And I think in every instance you’ve made a credible argument.

But I don’t think we have the if yet. If I could just slip GPT-4 an executive summary of the code I meant to write and the emails I meant to send on a given day I would be doing it.

But it can’t: it’s catastrophically wrong with extreme confidence routinely. These things are easy to cherry-pick in citation but still fuck up on HellaSwag in instances a child would not.

Re: AI for math resources, and erdosproblems.com

#19

Post by kevinventullo »

benreesman wrote: post #18
kevinventullo wrote: post #17

Earlier quoted context omitted.

Yeah I did say “back half”. I just think that if we can build an AI that can logically reason better than any human, combined with the total knowledge of all humans, it can start running experiments and doing “science” itself.

In general I think I agree with all your points. The word “if” in both the parent and GP is doing a lot of lifting here though, if they can do amazing feat X, then they will likewise do amazing feat Y. And I think in every instance you’ve made a credible argument. But I don’t think we have the if yet. If I could just slip GPT-4 an executive summary of the code I meant to write and the emails I meant to send on a give…

Oh, I 100% agree we are not there. Every time a new model comes out I cook up a few undergrad math homework questions to see if they’ve gotten any better, and so far they haven’t. But, the thing I like about “reducing” AGI to solving math is that it kind of sidesteps these thorny questions about interacting with the real world to conduct experiments, etc. It’s something that, in principle, can be done with the current text interfaces, and it’s pretty easy to tell whether or not we’ve gotten there.

Re: AI for math resources, and erdosproblems.com

#20

Post by kubrickslair »

benreesman wrote: post #6
upmind wrote: post #2

What does AI for Math mean? People doing Math research that want to transition to AI research?

I’m generally a bit of an LLM skeptic, at least as concerns extravagant claims around “AGI” or self-improvement of any of that nonsense. But there are activities where an inherently high-temperature assistant (e.g. some Instruct-inspired tune) are a great fit, and those are almost definitionally at the boundary of “objectively faithful” and “a bit stochastic”. I’ve never done any work in novel mathematics myself, but…

The impact extends beyond mathematics. We have seen some pretty positive outcomes for LLMs as an assistant for scientists - with a lot of custom agent dev though - and not off-the-shelf models.

This is already being used by a few hundred scientists in a lab. We are aiming to extend to thousands next year with a focus on CS, climate & bio.

Post reply on HN