Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI solves International Math Olympiad problems at silver medal level

deepmind.google

Post reply on HN
Print view View original

351–360 of 564 posts

Re: AI solves International Math Olympiad problems at silver medal level

#351

Post by tardygrade »

necovek wrote: post #337

This is certainly impressive, but whenever IMO is brought up, a caveat should be put out: medals are awarded to 50% of the participants (high school students), with 1:2:3 ratio between gold, silver and bronze. That puts all gold and silver medalists among the top 25% of the participants. That means that "AI solves IMO problems better than 75% of the students", which is probably even more impressive. But, "minutes for…

One key difference between giving humans more time and giving computer programs more time is that historically we have had more success making the latter run faster than the former.

Re: AI solves International Math Olympiad problems at silver medal level

#352

Post by acchow »

necovek wrote: post #337

This is certainly impressive, but whenever IMO is brought up, a caveat should be put out: medals are awarded to 50% of the participants (high school students), with 1:2:3 ratio between gold, silver and bronze. That puts all gold and silver medalists among the top 25% of the participants. That means that "AI solves IMO problems better than 75% of the students", which is probably even more impressive. But, "minutes for…

But computers get faster each year, so even with zero progress in actual AI, this will reach human-student speeds in a few years (need a 40x speed up)

Re: AI solves International Math Olympiad problems at silver medal level

#353

Post by necovek »

gallerdude wrote: post #130

Sometimes I wonder if in 100 years, it's going to be surprising to people that computers had a use before AI...

AI is simply another form of what we've been doing since the dawn of computers: expressing real world problems in the form of computations.

While there are certainly some huge jumps in compute power, theory of data transformation and availability of data to transform, it would surprise me if computers in a 100 years do not still rely on a combination of well-defined and well-understood algorithms and AI-inspired tools that do the same thing but on a much bigger scale.

If not for any other reason, then because there are so many things where you can easily produce a great, always correct result simply by doing very precise, obvious and simple computation.

We've had computers and digital devices for a long while now, yet we still rely heavily on mechanical contraptions. Sure, we improve them with computers (eg. think brushless motors), but I don't think anyone would be surprised today about how did anyone design these same devices (hair dryers, lawn mowers, internal combustion engines...) before computers?

Re: AI solves International Math Olympiad problems at silver medal level

#354

Post by lmm »

Smaug123 wrote: post #143
pclmulqdq wrote: post #123

Earlier quoted context omitted.

Comparing "turn photo into LaTeX" to "translate theorems into Lean" is like comparing a child's watercolor drawing to the Mona Lisa.

… no? After the LaTeX output, I told stock GPT4o that the answer was "all even integers", and asked for the statement in Lean. I had to make two changes to its output (both of which were compile-time errors, not misformalisations), and it gave me the formalisation of the difficult direction of the problem. Both changes were trivial: it had one incorrect (but unnecessary) import, and it used the syntax from Lean 3 ins…

> I am quite confident that this formalisation failure mode is not hard to solve in a system that is, like, actually trained to do this.

Why?

Re: AI solves International Math Olympiad problems at silver medal level

#355

Post by richard___ »

dinobones wrote: post #346

I see DeepMind is still playing around with RL + search algorithms, except now it looks like they're using an LLM to generate state candidates. I don't really find that this impressive. With enough compute you could just do n-of-10,000 LLM generations to "brute force" a difficult problem and you'll get there eventually.

Sigh. Just wrong

Re: AI solves International Math Olympiad problems at silver medal level

#356

Post by necovek »

Davidzheng wrote: post #153
majikaja wrote: post #115

Earlier quoted context omitted.

I found those, I just would have appreciated if the content of the mathematics wasn't sidelined to a separate download as if it's not important. I felt the explanation on the page was shallow, as if they just want people to accept it's a black box. All I've learnt from this is that they used an unstated amount of computational resources just to basically brute force what a human already is capable of doing in far les…

Very few humans can after years of training. Please don't trivialize.

Very few humans go after this type of the training. In my "math talent" school (most of the Serbian/Yugoslavian medal winners came from it), at most a dozen students "trained" for this over 4 high school generations (500 students).

Problems are certainly not trivial, but humans are not really putting all their effort into it either, and the few that do train for it, on average medal 50% of the time and get a silver or better 25% of the time (by design) with much less time available to do the problems.

Re: AI solves International Math Olympiad problems at silver medal level

#357

Post by refulgentis »

pishpash wrote: post #306
refulgentis wrote: post #257

Earlier quoted context omitted.

[flagged]

[flagged]

That wasn't very nice. Are you curious about anything? Happy to help. I'd proactively do it, but I don't want to guess at whats in your mind. My initial guess is you think I think that engaging with the public is an infinite loop. I don't!

Re: AI solves International Math Olympiad problems at silver medal level

#358

Post by dooglius »

Davidzheng wrote: post #341
necovek wrote: post #337

This is certainly impressive, but whenever IMO is brought up, a caveat should be put out: medals are awarded to 50% of the participants (high school students), with 1:2:3 ratio between gold, silver and bronze. That puts all gold and silver medalists among the top 25% of the participants. That means that "AI solves IMO problems better than 75% of the students", which is probably even more impressive. But, "minutes for…

In my opinion (not Google s) the only reason they didn't get gold this year (apart from being unlucky on problem selection) is that they didn't want to try for any partial credit in P3 and P5. They are so close to the cut off and usually contestants with a little bit of progress can get 1 point. But i guess they didn't want to get a gold on a technicality--it would be bad press. So they settled in a indisputable silv…

I don't believe anything was graded by the IMO, Google is just giving itself 7 for anything proved in Lean (which is reasonable IMO), so they can't really try for partial credit so much as choose not to report a higher self-graded number.

Re: AI solves International Math Olympiad problems at silver medal level

#359

Post by rich_sasha »

I'm actually not that surprised. Maths Olympiads IME have always been 80% preparation, 20% skill - if not more heavily tuned to preparation. It was all about solving as many problems as possible ahead of the papers, and having a good short term memory. Since Olympiads are for kids, the amount of actual fundamental mathematical theorems required is actually not that great.

Sounds perfect for a GPT model, with lots of input training data (problem books and solutions).

Re: AI solves International Math Olympiad problems at silver medal level

#360

Post by llwu »

Davidzheng wrote: post #341
necovek wrote: post #337

This is certainly impressive, but whenever IMO is brought up, a caveat should be put out: medals are awarded to 50% of the participants (high school students), with 1:2:3 ratio between gold, silver and bronze. That puts all gold and silver medalists among the top 25% of the participants. That means that "AI solves IMO problems better than 75% of the students", which is probably even more impressive. But, "minutes for…

In my opinion (not Google s) the only reason they didn't get gold this year (apart from being unlucky on problem selection) is that they didn't want to try for any partial credit in P3 and P5. They are so close to the cut off and usually contestants with a little bit of progress can get 1 point. But i guess they didn't want to get a gold on a technicality--it would be bad press. So they settled in a indisputable silv…

Partial credit is quite challenging to earn, per: https://www.imo-official.org/year_statistics.aspx?year=2024
Post reply on HN