Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Caltech Mathathon – first hackathon ever devoted to research level mathematics

mathathonchallenge.com

Post reply on HN
Print view View original

51–60 of 109 posts

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#51

Post by a2ff6eeb0 »

bwfan123 wrote: post #37
youoy wrote: post #10

Should I read this as the big labs trying to move maths forward? Or the big labs trying to use professional mathematitians as (cheap?) Labour for validating LLM outputs? On yesterdays "An Alien Mind" post from openAI they openly said that maths is not a priority for them, so I personally know what to think...

> Or the big labs trying to use professional mathematitians as (cheap?) Labour for validating LLM outputs? I am told AGI has been achieved. If so, shouldnt these systems be out and about on their own ? Looking at 1st proof submissions in batch 2 it is clear that fully autonomous AI systems have a long way to go. AI harnessing human labor with the incentive of 2M in free tokens is the way my skeptic eye sees it, or hu…

It's very clear that the AI still has no motivation beyond its prompts. Humans can mostly outsource their thinking today across a wide variety of topics, but they still need to express their desires.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#52

Post by fred123123 »

brian-bfz wrote: post #49
fred123123 wrote: post #47

Earlier quoted context omitted.

Hey! Any indication on what area of mathematics theses questions are from?

You pick your own problem! You can even formulate your own conjecture and then prove it. Picking an impactful problem is part of our judging criteria.

That sound cool!, do you have hints on the cash prizes the website said something like 2 M ???

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#53

Post by a2ff6eeb0 »

jhonof wrote: post #14
a2ff6eeb0 wrote: post #8

Earlier quoted context omitted.

It seems like making progress on math is letting the AI run fully autonomously for a few days, occasionally asking it to keep going. I'm not sure people need to organize a mathathon to wait for a computer to give a printout. They mainly need tokens.

I think the purpose of an event like this would be to optimize the process so that it isn't just occasionally asking an AI to keep going.

That sounds like adding a bottleneck, unless you mean writing a harness that automatically asks the model to keep going, so that there's no humans involved at all?

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#54

Post by maypop »

brian-bfz wrote: post #42

Hey guys, I'm one of the organizers. AMA. - We are a team of undergrads at Caltech. We don't represent Caltech, any Caltech departments, or any of our sponsors. - We don't receive monetary compensation. All the funding raised goes toward paying our judges and participants. - Our goal is to promote responsible AI use. You can read more about our commitments here: https://mathathonchallenge.com/faq.html

Is this hackathon only for those with formal math backgrounds?

I've seen a few instances of AI assisted advances math and cs this year that were _not_ published by authors with formal backgrounds in those fields (or even institutional affiliation). Which makes me wonder if they would have a place at the event.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#55

Post by jegutman »

fred123123 wrote: post #52
brian-bfz wrote: post #49

Earlier quoted context omitted.

You pick your own problem! You can even formulate your own conjecture and then prove it. Picking an impactful problem is part of our judging criteria.

That sound cool!, do you have hints on the cash prizes the website said something like 2 M ???

That’s tokens available I assume for all competitors during the competition.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#56

Post by brian-bfz »

maypop wrote: post #54
brian-bfz wrote: post #42

Hey guys, I'm one of the organizers. AMA. - We are a team of undergrads at Caltech. We don't represent Caltech, any Caltech departments, or any of our sponsors. - We don't receive monetary compensation. All the funding raised goes toward paying our judges and participants. - Our goal is to promote responsible AI use. You can read more about our commitments here: https://mathathonchallenge.com/faq.html

Is this hackathon only for those with formal math backgrounds? I've seen a few instances of AI assisted advances math and cs this year that were _not_ published by authors with formal backgrounds in those fields (or even institutional affiliation). Which makes me wonder if they would have a place at the event.

Yes. To us, solving a problem with AI doesn't matter as much as selecting the right problem and understanding the proof. A participant must have a good mathematical intuition.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#57

Post by youoy »

brian-bfz wrote: post #50
youoy wrote: post #10

Should I read this as the big labs trying to move maths forward? Or the big labs trying to use professional mathematitians as (cheap?) Labour for validating LLM outputs? On yesterdays "An Alien Mind" post from openAI they openly said that maths is not a priority for them, so I personally know what to think...

We are a student-led initiative. Our sponsors don't pay us and don't have a say in our decisions. All of our funding goes toward our judges and participants.

Hey! Thanks for answering, i appreciate it. Dont get me wrong, this is what you should be doing, understanding what these models are good (and most importantly bad) for. My observation is that this is very very valuable for the labs, and in an ideal world they should be paying you to do this, not just the tokens and "prize" for the winners.

I am also a bit frustrated seeing maths go in the direction of prompt enginnering. I am afraid of a world were a math phd student cannot go one week thinking about a problem without prompting an LLM to give him/her an invented answer. Something is lost along the way.

For me maths is not Lean, or formal systems, or an agent reasoning about formal systems to join literature from different fields. I see the value of it, but i think it will make it way more difficult for students (and profesional mathematitians) to see beyond that. And i see us heading into a reality were those who think like me will in practice remain a minority for quite a few years/decades because the low hanging fruit of LLMs will be to vast to ignore.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#58

Post by youoy »

a2ff6eeb0 wrote: post #45
MostlyStable wrote: post #38

Earlier quoted context omitted.

That's how several major AI advancements have happened. I have seen no evidence that that is the fastest way to make progress right now. I expect that, much like chess engines, it will not take too long before AI is significantly better than AI + human. But right now, my bet is that we are still safely within the window where an AI + human mathematician team is still better than AI alone (at least for the case where…

I suspect that the best progress will be made by a team that purely spends their time taking a list of open problems and promoting "solve ", without actually trying to understand anything. Just keep as many problems in flight as you can across as many sessions as you can. You can probably ask the AI to come up with a list of problems itself, and rank them by the likelihood of progress.

Then 20 years go by and you wake up one day with questions that you cannot get out of your mind: why did I start prompting the LLM for? Why did I need these random proofs for? What do i do with my repo with 2billion lines of Lean?

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#59

Post by brian-bfz »

youoy wrote: post #57
brian-bfz wrote: post #50

Earlier quoted context omitted.

We are a student-led initiative. Our sponsors don't pay us and don't have a say in our decisions. All of our funding goes toward our judges and participants.

Hey! Thanks for answering, i appreciate it. Dont get me wrong, this is what you should be doing, understanding what these models are good (and most importantly bad) for. My observation is that this is very very valuable for the labs, and in an ideal world they should be paying you to do this, not just the tokens and "prize" for the winners. I am also a bit frustrated seeing maths go in the direction of prompt enginne…

Agreed. I think we share the same concerns about LLM use. We just drew different conclusions.

Mathathon's goal is to reshape rather than stop LLM use. Can we set high standards for LLM use? Can we highlight the roles of a mathematician beyond proof generation? Can we redesign our incentives to promote these standards and roles?

I'd love to hear your thoughts on how to improve this event. We're very open to criticisms.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#60

Post by charlieyu1 »

a2ff6eeb0 wrote: post #36
hgoel wrote: post #32

Earlier quoted context omitted.

Same here, it's one thing if I'm just screwing around, but if I'm trying to do anything serious, I need to at least have a handle on what it's doing, and, when thinking traces are available, keeping track of any logical errors in the model's reasoning.

you know the thinking traces are redacted and summarized using another model, right? The actual thinking trace looks something like: 7♣-removal-IS-the-prerequisite-for-10♠/9♥!!)-⟹-OVERLAP-(ii)+(iv):-{6♠ J♦ 9♥ 2♣}-=-FOUR--—-UNLESS-7♣'s-seat-8♥-...-and-2♣-drains-only-at-crack-:-⟹-2♣-celled-+-9♥-celled-simultaneously-UNAVOIDABLE-in-t8-dig--—-BREAK:-9♥-drains-to-10♠-THE-MOMENT-10♠-is-free:-t8-dig-order:-[K♣→t2]-[2♣→cell]…

Why is the model playing poker
Post reply on HN