Claude 4 • Hacker News Board

Re: Claude 4

#31

Post by minimaxir » Thu, May 22, 2025, 4:49 PM UTC

sigmoid10 wrote: post #18

Sooo... it can play Pokemon. Feels like they had to throw that in after Google IO yesterday. But the real question is now can it beat the game including the Elite Four and the Champion. That was pretty impressive for the new Gemini model.

That Google IO slide was somewhat misleading as the maintainer of Gemini Plays Pokemon had a much better agentic harness that was constantly iterated upon throughout the runtime (e.g. the maintainer had to give specific instructions on how to use Strength to get past Victory Road), unlike Claude Plays Pokemon.

The Elite Four/Champion was a non-issue in comparison especially when you have a lv. 81 Blastoise.

Re: Claude 4

#32

Post by oofbaroomf » Thu, May 22, 2025, 4:50 PM UTC

Wonder why they renamed it from Claude (e.g. Claude 3.7 Sonnet) to Claude (Claude Opus 4).

Re: Claude 4

#33

Post by jeremyjh » Thu, May 22, 2025, 4:51 PM UTC

htrp wrote: post #19

Allegedly Claude 4 Opus can run autonomously for 7 hours (basically automating an entire SWE workday).

Which sort of workday? The sort where you rewrite your code 8 times and end the day with no marginal business value produced?

Re: Claude 4

#34

Post by hansmayer » Thu, May 22, 2025, 4:51 PM UTC

[flagged]

Re: Claude 4

#35

Post by Squarex » Thu, May 22, 2025, 4:51 PM UTC

i_love_retros wrote: post #23

[flagged]

You are on a website dedicated to technology news. What's the surprise?

Re: Claude 4

#36

Post by oofbaroomf » Thu, May 22, 2025, 4:51 PM UTC

Interesting how Sonnet has a higher SWE-bench Verified score than Opus. Maybe says something about scaling laws.

Re: Claude 4

#37

Post by mxwsn » Thu, May 22, 2025, 4:51 PM UTC

throwaway314155 wrote: post #28

sigmoid10 wrote: post #18

Sooo... it can play Pokemon. Feels like they had to throw that in after Google IO yesterday. But the real question is now can it beat the game including the Elite Four and the Champion. That was pretty impressive for the new Gemini model.

Gemini can beat the game?

Gemini has beat it already, but using a different and notably more helpful harness. The creator has said they think harness design is the most important factor right now, and that the results don't mean much for comparing Claude to Gemini.

Re: Claude 4

#38

Post by speedgoose » Thu, May 22, 2025, 4:52 PM UTC

htrp wrote: post #19

Allegedly Claude 4 Opus can run autonomously for 7 hours (basically automating an entire SWE workday).

Easy, I can also make a nanoGPT run for 7 hours when inferring on a 68k, and make it produce as much value as I usually do.

Re: Claude 4

#39

Post by Artgor » Thu, May 22, 2025, 4:52 PM UTC

OpenIA's Codex-1 isn't so cool anymore. If it was ever cool.

And Claude Code used Opus 4 now!

Re: Claude 4

#40

Post by boh » Thu, May 22, 2025, 4:53 PM UTC

Can't wait to hear how it breaks all the benchmarks but have any differences be entirely imperceivable in practice.