Live data from Hacker News

Early Evals for OpenAI O3

1–5 of 5 posts

Post by maurycy » Fri, Dec 20, 2024, 6:06 PM UTC

Re: Early Evals for OpenAI O3

Post by [deleted] » Fri, Dec 20, 2024, 6:11 PM UTC

[deleted]

Post by famouswaffles » Fri, Dec 20, 2024, 6:13 PM UTC

New SOTA's on:

SWE-Bench - 71.7

Competition Code - 2727

ARC (Semi Private Eval) - 75.7 on low, 87.5% on high compute

Frontier Math (previous SOTA was 2%) - 25% on high compute

Post by [deleted] » Fri, Dec 20, 2024, 6:13 PM UTC

[deleted]

Post by blurbleblurble » Fri, Dec 20, 2024, 8:05 PM UTC

Wow, the demo where the user asks for untraceable payments shows some pretty sophisticated reasoning. The word "crafty" comes to mind.