Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Caltech Mathathon – first hackathon ever devoted to research level mathematics

mathathonchallenge.com

Post reply on HN
Print view View original

81–90 of 113 posts

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#81

Post by ajkjk »

queuebert wrote: post #61
brian-bfz wrote: post #42

Hey guys, I'm one of the organizers. AMA. - We are a team of undergrads at Caltech. We don't represent Caltech, any Caltech departments, or any of our sponsors. - We don't receive monetary compensation. All the funding raised goes toward paying our judges and participants. - Our goal is to promote responsible AI use. You can read more about our commitments here: https://mathathonchallenge.com/faq.html

Serious question: Are you sure solving abstract mathematical problems with AI is responsible use? You will likely put mathematicians out of jobs, and I doubt solving the Collatz conjecture is urgent or will save lives. It also robs a future Fields medalist of the pride of doing all by themselves. All things AI seems to assume that more and faster is better, but there is no justification of that assumption. As a biolo…

I dunno, I like math and I still think automatically solving problems is worth doing. If problem-solving is a hobby then it will still be a hobby after all this. If it's about doing things for humanity--which it often is since many thousands of people are paid to do it--then doing it more efficiently benefits humanity. If the value on the other hand comes from people being extremely good at math, rather than from them solving novel open problems, then we should pay them to do that, which is independent of whether the open problems are solved or not. There's just no version of this where "not solving the problems" is morally justifiable.

Now, I think it's the case that professional mathematics spends way too much money on open problems and way less than it should on pedagogy, exposition, mastery, etc. But that has always been a problem, even decades ago (I've been complaining about it my whole life). AI just finally puts pressure on the world to do something about it. I find it relieving, honestly. And I'm an AI skeptic in many other ways; it's not an AI-maximalism thing. I genuinely think the state of the field of mathematics has been something of a disaster for a long time (thanks, largely, due to the academic incentive structure which heavily favors novel results, no matter how esoteric).

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#82

Post by brian-bfz »

ajkjk wrote: post #80
brian-bfz wrote: post #78

Earlier quoted context omitted.

Solving a problem is the easy part with AI. Selecting a problem worth solving is half of the challenge. We allow prior work as long as it's labelled. We record chat logs, so it's easy to verify what's prior work. When we evaluate the significance of a result, we focus on the part produced at the event.

it seems like a person could cheat by finding an open problem that's AI-solvable ahead of time (by trying many problems), and then pretend to do it for the first time in the competition. You can't verify past chat logs so make sure they didn't already realize it would work. just something to think about

We are giving each team $20k AI credits. I hope they will use it to solve problems beyond the reach of regular AI plans.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#83

Post by tacomonstrous »

queuebert wrote: post #61
brian-bfz wrote: post #42

Hey guys, I'm one of the organizers. AMA. - We are a team of undergrads at Caltech. We don't represent Caltech, any Caltech departments, or any of our sponsors. - We don't receive monetary compensation. All the funding raised goes toward paying our judges and participants. - Our goal is to promote responsible AI use. You can read more about our commitments here: https://mathathonchallenge.com/faq.html

Serious question: Are you sure solving abstract mathematical problems with AI is responsible use? You will likely put mathematicians out of jobs, and I doubt solving the Collatz conjecture is urgent or will save lives. It also robs a future Fields medalist of the pride of doing all by themselves. All things AI seems to assume that more and faster is better, but there is no justification of that assumption. As a biolo…

FWIW even Astra hasn't been able to solve the problems I care about, which are less about proving theorems and more about understanding the right way to think about already existing stories (and thus permitting extensions to new contexts). However it's been a more than a capable interlocutor to test my ideas with and see if they actually have any content. It's also great for parsing possible mistakes in long technical arguments that at least my brain isn't wired to verify completely satisfactorily. Personally, I think it's good to know what is made trivial (meaning depending only on token expenditure) vs what remains a real hard kernel.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#84

Post by tacomonstrous »

a2ff6eeb0 wrote: post #76
mattmcal wrote: post #75

Earlier quoted context omitted.

If you're interested in the topic enough to comment on it, you'll probably find it worthwhile reading a mathematician's perspective. Here's the prolific Terry Tao: https://mathstodon.xyz/@tao/117219548485446992

They don't actually say anything about why anyone should fund this, though. I don't get why a society should worry about progress in mathematics if there's no practical benefit expected. Maybe there's two kinds of math that we need? Useful math and navel gazing, and we can hand the first to the machines, and let hobbyists do the second in their free to entertain themselves?

It's hard to know what math is 'useful' a priori. That's always been the argument for supporting basic research. This is not why I am a mathematician however. I think there's intrinsic value into understanding something of depth and meaning, but the societal setup we have now that mostly agrees this is valuable is probably a very contingent phenomenon that is unlikely to last much longer.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#85

Post by ewjt »

a2ff6eeb0 wrote: post #36
hgoel wrote: post #32

Earlier quoted context omitted.

Same here, it's one thing if I'm just screwing around, but if I'm trying to do anything serious, I need to at least have a handle on what it's doing, and, when thinking traces are available, keeping track of any logical errors in the model's reasoning.

you know the thinking traces are redacted and summarized using another model, right? The actual thinking trace looks something like: 7♣-removal-IS-the-prerequisite-for-10♠/9♥!!)-⟹-OVERLAP-(ii)+(iv):-{6♠ J♦ 9♥ 2♣}-=-FOUR--—-UNLESS-7♣'s-seat-8♥-...-and-2♣-drains-only-at-crack-:-⟹-2♣-celled-+-9♥-celled-simultaneously-UNAVOIDABLE-in-t8-dig--—-BREAK:-9♥-drains-to-10♠-THE-MOMENT-10♠-is-free:-t8-dig-order:-[K♣→t2]-[2♣→cell]…

Anthropic specifically calls out that example as

>"illegible reasoning in a few reinforcement-learning environments over long rollout"

Yet, I get the point that you're making: those tokens essentially are an internal scratchpad for the LLM which isn't required to logically lead to the output.

This video presentation of the paper you linked was interesting: https://www.youtube.com/watch?v=hUp3zh23aHw

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#86

Post by suriyaG »

hatsix wrote: post #77
lovasoa wrote: post #65

Earlier quoted context omitted.

steam engines solved a problem people had

Don't respond to a strawman argument with another strawman. The post you are responding to ignored the reasons given in the second paragraph. They're just trying to score points by preaching to the choir, not engage with the concern.

what strawman?

do you mean,

> All things AI seems to assume that more and faster is better, but there is no justification of that assumption.

is good argument?

of course faster discovery without human in the loop is better. is that not what humans have been optimizing for the past few thousand years ? faster mobility, faster communication, faster medical recovery etc. everything modern civilization has to offer is because of a rush to get better and faster. for example, discovering penicillin 2 years early would've saved ~15 million people more.

why is that not worthy enough to pursue?

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#87

Post by suriyaG »

lovasoa wrote: post #65
suriyaG wrote: post #63

Earlier quoted context omitted.

> you will likely put mathematicians out of jobs. why is that a concern in this context? would you have asked the same about steam engines and horses? this is a really cool concept, organized very well. and that is very commendable.

steam engines solved a problem people had

I suppose iphones solved a problem people had as well?

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#89

Post by danielmarkbruce »

brian-bfz wrote: post #82
ajkjk wrote: post #80

Earlier quoted context omitted.

it seems like a person could cheat by finding an open problem that's AI-solvable ahead of time (by trying many problems), and then pretend to do it for the first time in the competition. You can't verify past chat logs so make sure they didn't already realize it would work. just something to think about

We are giving each team $20k AI credits. I hope they will use it to solve problems beyond the reach of regular AI plans.

Ok. This all makes sense I guess.

Good going, hope it goes well.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#90

Post by smcin »

brian-bfz wrote: post #56
maypop wrote: post #54

Earlier quoted context omitted.

Is this hackathon only for those with formal math backgrounds? I've seen a few instances of AI assisted advances math and cs this year that were _not_ published by authors with formal backgrounds in those fields (or even institutional affiliation). Which makes me wonder if they would have a place at the event.

Yes. To us, solving a problem with AI doesn't matter as much as selecting the right problem and understanding the proof. A participant must have a good mathematical intuition.

How much formal math background is required? Undergrad/graduate/PhD/...? For one person on each team or everyone?
Post reply on HN