Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reimagining mathematics in a world of reasoning machines [video]

youtube.com

Post reply on HN
Print view View original

11–20 of 43 posts

Re: Reimagining mathematics in a world of reasoning machines [video]

#11

Post by pegasus »

m3kw9 wrote: post #9

Just waiting on what Terrance Tao thinks of the new o3 intern

When can infer somewhat from this section in the FrontierMath paper [1]:

"Assessment of FrontierMath difficulty

All four mathematicians characterized the research problems in the FrontierMath benchmark as exceptionally challenging, noting that the most difficult questions require deep domain expertise and significant time investment. For example, referring to a selection of several questions from the dataset, Tao remarked, "These are extremely challenging. I think that in the near term basically the only way to solve them, short of having a real domain expert in the area, is by a combination of a semi-expert like a graduate student in a related field, maybe paired with some combination of a modern AI and lots of other algebra packages...”

However, some mathematicians pointed out that the numerical format of the questions feels somewhat contrived. Borcherds, in particular, mentioned that the benchmark problems “aren’t quite the same as coming up with original proofs.”

It sounds like it will be able to crack some hard math problems, but not actually do mathematics. Which makes sense to me, given how these oN models are being trained. Synthetic data is bound to be in some (more or less obvious) way contrieved, since it's not like we can just generate tons of new/natural/original mathematical results to train the model on.

[1] https://arxiv.org/pdf/2411.04872

Re: Reimagining mathematics in a world of reasoning machines [video]

#13

Post by versteegen »

pegasus wrote: post #11
m3kw9 wrote: post #9

Just waiting on what Terrance Tao thinks of the new o3 intern

When can infer somewhat from this section in the FrontierMath paper [1]: "Assessment of FrontierMath difficulty All four mathematicians characterized the research problems in the FrontierMath benchmark as exceptionally challenging, noting that the most difficult questions require deep domain expertise and significant time investment. For example, referring to a selection of several questions from the dataset, Tao rem…

> It sounds like it will be able to crack some hard math problems, but not actually do mathematics.

What, do you say that because format of the FrontierMath problems is a bit contrived? I think I must misunderstanding you; a benchmark can't rule out something it doesn't test. And saying solving these problems requiring "deep domain expertise" isn't real mathematics sure sounds like a No True Scotsman argument. Why shouldn't o3 be able to produce novel mathematical proofs? It's got plenty of maths textbooks and papers to train on, the same thing mathematicians train on.

Re: Reimagining mathematics in a world of reasoning machines [video]

#14

Post by godelski »

Al0neStar wrote: post #10
godelski wrote: post #3

Earlier quoted context omitted.

I'm curious, what do you see as "gatekeeping layers" in math? Books? Material? Pedagogy? Nomenclature? Something else?

Not really gatekeeping, but as someone who likes to self-study, not having solutions is very annoying.

I totally get that. I was fortunate to get properly educated up through essentially the level of an undergraduate math degree (minus maybe typology), but then continued learning a lot on my own. It’s common to hear that the struggle is part of the learning process. The more I’ve advanced the more I find this to be true. It’s that struggle that makes you pay attention to the small details that are so critical. I was also fortunate to have a professor who would pester me and he later told me he wanted me to be confident in my results, because eventually I would have no one to double check (and he was right). The struggle really helps with this.

I think the main difference between learning provisioning and math is a compiler. To learn either you can only learn by doing. Reading and lectures aren’t enough. What is hard to learn in math is to be the compiler yourself. To be able to verify “programs” (do I even need quotes here?). This is a very powerful tool to add to your tool belt and one I think even helps in programming.

I hope others can add advice here and words of encouragement. The struggle is real, but it is part of the process, for better or for worse.

Re: Reimagining mathematics in a world of reasoning machines [video]

#15

Post by pegasus »

versteegen wrote: post #13
pegasus wrote: post #11

Earlier quoted context omitted.

When can infer somewhat from this section in the FrontierMath paper [1]: "Assessment of FrontierMath difficulty All four mathematicians characterized the research problems in the FrontierMath benchmark as exceptionally challenging, noting that the most difficult questions require deep domain expertise and significant time investment. For example, referring to a selection of several questions from the dataset, Tao rem…

> It sounds like it will be able to crack some hard math problems, but not actually do mathematics. What, do you say that because format of the FrontierMath problems is a bit contrived? I think I must misunderstanding you; a benchmark can't rule out something it doesn't test. And saying solving these problems requiring "deep domain expertise" isn't real mathematics sure sounds like a No True Scotsman argument. Why sh…

Those proofs are not by far enough - which is why the raw frontier models don't do that well on this test. The oN models are trained on synthetic data: so generated mathematical theorems and associated proofs. If we could generate legitimately new math we would already have accomplished the hard task, so the generated theorems are going to feel contrieved. I.e. they won't be theorems that ever make it into a future math book.

Doesn't mean they won't be useful to mathematicians, they will, just like computers in general are useful in doing maths today.

Re: Reimagining mathematics in a world of reasoning machines [video]

#16

Post by pegasus »

versteegen wrote: post #13
pegasus wrote: post #11

Earlier quoted context omitted.

When can infer somewhat from this section in the FrontierMath paper [1]: "Assessment of FrontierMath difficulty All four mathematicians characterized the research problems in the FrontierMath benchmark as exceptionally challenging, noting that the most difficult questions require deep domain expertise and significant time investment. For example, referring to a selection of several questions from the dataset, Tao rem…

> It sounds like it will be able to crack some hard math problems, but not actually do mathematics. What, do you say that because format of the FrontierMath problems is a bit contrived? I think I must misunderstanding you; a benchmark can't rule out something it doesn't test. And saying solving these problems requiring "deep domain expertise" isn't real mathematics sure sounds like a No True Scotsman argument. Why sh…

Look at the sample problems and it will be obvious. For example, look at all the arbitrary constants here:

"Let an for n ∈ Z be the sequence of integers satisfying the recurrence formula1 an = (1.981 × 1011)an−1 + (3.549 × 1011)an−2 − (4.277 × 1011)an−3 + (3.706 × 108 )an−4 with initial conditions ai = i for 0 ≤ i ≤ 3. Find the smallest prime p ≡ 4 mod 7 for which the function Z → Z given by n 7→ an can be extended to a continuous function on Zp."

Re: Reimagining mathematics in a world of reasoning machines [video]

#17

Post by Nevermark »

random3 wrote: post #4
revskill wrote: post #2

My wish is for math to remove all the gatekeeping layers and one day, everyone can do maths, just like with programming.

Programming should remove all gatekeeping one day so everyone can do programming

And gatekeeping, so many of us want to gatekeep but there too, other gatekeepers stand in our way

Re: Reimagining mathematics in a world of reasoning machines [video]

#18

Post by nicf »

Al0neStar wrote: post #10
godelski wrote: post #3

Earlier quoted context omitted.

I'm curious, what do you see as "gatekeeping layers" in math? Books? Material? Pedagogy? Nomenclature? Something else?

Not really gatekeeping, but as someone who likes to self-study, not having solutions is very annoying.

I work as a private tutor for proof-based math, and I have a lot of students who've spent some time self-studying before coming to me. The comment by godelski matches my experience: the biggest obstacle seems to be the fact that it's hard to learn how to check your own proofs if no one has ever taught you how. I see a lot of variation in how well people have managed to develop that skill on their own.

Having more textbooks with solutions to the exercises would probably help a lot with this, especially if you used the solutions judiciously. I think the fact that this isn't more common sadly has a lot to do with their role in undergraduate teaching: every exercise that has a solution in the back of the book is one that college students can very easily cheat on. I definitely agree that it's frustrating that the product has to be made worse for everyone else just because some people would misuse the better version. Far from the only such case in the world!

Re: Reimagining mathematics in a world of reasoning machines [video]

#19

Post by amelius »

Nevermark wrote: post #17
random3 wrote: post #4

Earlier quoted context omitted.

Programming should remove all gatekeeping one day so everyone can do programming

And gatekeeping, so many of us want to gatekeep but there too, other gatekeepers stand in our way

Just build a gate around their gate.

Re: Reimagining mathematics in a world of reasoning machines [video]

#20

Post by godelski »

wslh wrote: post #5
godelski wrote: post #3

Earlier quoted context omitted.

I'm curious, what do you see as "gatekeeping layers" in math? Books? Material? Pedagogy? Nomenclature? Something else?

Not OP, but I'd like to add the term “learning complexity reduction” which relates closely to “pedagogy”. Reflecting on my experience as a CS graduate with a strong interest in math (though not very advanced knowledge), I realize I would have benefited from a more intuitive, black-box approach when first engaging with complex topics, rather than diving straight into their intricacies.

I think this is kinda like the C++ vs Python as your first language question. C++ is better if you want to really learn programming but it’s harder to get to the magic of your first real program. Python gets people making programs sooner but it’s harder to build good habits and get that deep understanding. It depends what your goals are. There’s stuff like this for math but it’s kinda like trying to learn math from 3Blue1Brown videos. They sure help, but you eventually need the rigor
Post reply on HN