Live data from Hacker News

Caltech Mathathon – first hackathon ever devoted to research level mathematics

mathathonchallenge.com

21–30 of 108 posts

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#21
post #5

Won't deny that this is an interesting idea, but I feel like waiting on the output of an LLM for 40 hours feels like it is completely antithetical to what makes classic Hackathons appealing / educative. More generally, I don't think the shape of a hackathon (intensely working for a short timespan) maps at all onto the way LLM Math progress has seemingly been made so far; AFAIK it mostly involves picking out something…

Have you done any math hacking with sol/astra or fable? It’s more fun than using them for coding. The models are great at the monotony, like constructing a Gröbner-basis, etc. But they’re all still absolutely awful at coming up with new ideas, new proof methods, or new constructive forms. So you spend all your time on coming up with novel hypotheses yourself and handing off the rote work to an agent.

It’s also quite fun to get instant results by finding isomorphisms into unfamiliar areas of mathematics that previously would’ve required some networking in order to build a collaborative relationship.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#22
post #5

Won't deny that this is an interesting idea, but I feel like waiting on the output of an LLM for 40 hours feels like it is completely antithetical to what makes classic Hackathons appealing / educative. More generally, I don't think the shape of a hackathon (intensely working for a short timespan) maps at all onto the way LLM Math progress has seemingly been made so far; AFAIK it mostly involves picking out something…

But it's not "waiting on the output of an LLM for 40 hours" any more than a regular hackathon is "waiting for my damn teammates to finish their part for 40 hours". From my experience using agentic coding for hackathons, the best teams are those that coordinate with the AI agents in relatively quick cadence, generally giving it small tasks and steering it often. Teams may want to run some long-running sessions too, especially closer to the deadline, but even then, they'd probably want to run and follow several sessions in parallel, and continuously inspect their work so that they have reasonable confidence that their main efforts will wrap up before the deadline. There is an art to it.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#24
post #21
post #5

Won't deny that this is an interesting idea, but I feel like waiting on the output of an LLM for 40 hours feels like it is completely antithetical to what makes classic Hackathons appealing / educative. More generally, I don't think the shape of a hackathon (intensely working for a short timespan) maps at all onto the way LLM Math progress has seemingly been made so far; AFAIK it mostly involves picking out something…

Have you done any math hacking with sol/astra or fable? It’s more fun than using them for coding. The models are great at the monotony, like constructing a Gröbner-basis, etc. But they’re all still absolutely awful at coming up with new ideas, new proof methods, or new constructive forms. So you spend all your time on coming up with novel hypotheses yourself and handing off the rote work to an agent. It’s also quite…

"A mathematician is a person who can find analogies between theorems; a better mathematician is one who can see analogies between proofs and the best mathematician can notice analogies between theories. One can imagine that the ultimate mathematician is one who can see analogies between analogies."

I wonder how models perform on finding analogies between analogies

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#25

Earlier quoted context omitted.

If the goal is to accomplish something then why limit yourself with available tools? I’m not a full on AI optimist but it is absolutely the most powerful tool in a host of applications. From a Hackathon perspective, obviously in the 90s it was much more unorganized, but the same ethos existed. Use all available tools to accomplish the goal/task, it’s where a lot of incredible learning came out of. The same will hopef…

If the goal is to run 42km why limit yourself? Use a car and win.

But the goal here is not to run 42km; to stay with the outdoors metaphor, it's more like deciding where and how to set up a bivouac - use whatever tools you have at your disposal to analyze the area you're in, and find the best site to stay in overnight.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#26
post #21

Earlier quoted context omitted.

Have you done any math hacking with sol/astra or fable? It’s more fun than using them for coding. The models are great at the monotony, like constructing a Gröbner-basis, etc. But they’re all still absolutely awful at coming up with new ideas, new proof methods, or new constructive forms. So you spend all your time on coming up with novel hypotheses yourself and handing off the rote work to an agent. It’s also quite…

"A mathematician is a person who can find analogies between theorems; a better mathematician is one who can see analogies between proofs and the best mathematician can notice analogies between theories. One can imagine that the ultimate mathematician is one who can see analogies between analogies." I wonder how models perform on finding analogies between analogies

> I wonder how models perform on finding analogies between analogies

Load-bearingly verbose, in my experience.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#27
post #15

Earlier quoted context omitted.

Are they actually autonomous? I’d say subject knowledge at the prompt stage plays a large part towards getting proper results

Prompting for some of the results was almost the "Computer, do a breakthrough. Make no mistakes." meme. Just someone telling the model to keep trying a couple of times. Unfortunately we don't actually know what kind of prompting was done for the more prominent results.

It’s definitely not how I work, I’d need to read the model responses and set a direction for the model to go

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#28
post #15

Earlier quoted context omitted.

Prompting for some of the results was almost the "Computer, do a breakthrough. Make no mistakes." meme. Just someone telling the model to keep trying a couple of times. Unfortunately we don't actually know what kind of prompting was done for the more prominent results.

It’s definitely not how I work, I’d need to read the model responses and set a direction for the model to go

[deleted]

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#29
post #21

Earlier quoted context omitted.

Have you done any math hacking with sol/astra or fable? It’s more fun than using them for coding. The models are great at the monotony, like constructing a Gröbner-basis, etc. But they’re all still absolutely awful at coming up with new ideas, new proof methods, or new constructive forms. So you spend all your time on coming up with novel hypotheses yourself and handing off the rote work to an agent. It’s also quite…

"A mathematician is a person who can find analogies between theorems; a better mathematician is one who can see analogies between proofs and the best mathematician can notice analogies between theories. One can imagine that the ultimate mathematician is one who can see analogies between analogies." I wonder how models perform on finding analogies between analogies

Well, I suppose that explains monads. It’s one thing to see an analogy and quite another to make it the basis of an API.

Re: Caltech Mathathon – first hackathon ever devoted to research level mathematics

#30
post #5

Won't deny that this is an interesting idea, but I feel like waiting on the output of an LLM for 40 hours feels like it is completely antithetical to what makes classic Hackathons appealing / educative. More generally, I don't think the shape of a hackathon (intensely working for a short timespan) maps at all onto the way LLM Math progress has seemingly been made so far; AFAIK it mostly involves picking out something…

Are you sure it’s letting it run and not going back and forth interactively?
Post reply on HN