Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

System Card: Claude Mythos Preview [pdf]

www-cdn.anthropic.com

Post reply on HN
Print view View original

41–50 of 687 posts

Re: System Card: Claude Mythos Preview [pdf]

#41

Post by rafaelmn »

whalesalad wrote: post #33
babelfish wrote: post #3

Combined results (Claude Mythos / Claude Opus 4.6 / GPT-5.4 / Gemini 3.1 Pro) SWE-bench Verified: 93.9% / 80.8% / — / 80.6% SWE-bench Pro: 77.8% / 53.4% / 57.7% / 54.2% SWE-bench Multilingual: 87.3% / 77.8% / — / — SWE-bench Multimodal: 59.0% / 27.1% / — / — Terminal-Bench 2.0: 82.0% / 65.4% / 75.1% / 68.5% GPQA Diamond: 94.5% / 91.3% / 92.8% / 94.3% MMMLU: 92.7% / 91.1% / — / 92.6–93.6% USAMO: 97.6% / 42.3% / 95.2%…

Honestly we are all sleeping on GPT-5.4. Particularly with the influx of Claude users recently (and increasingly unstable platform) Codex has been added to my rotation and it's surprising me.

GPT is shit at writing code. It's not dumb - extra high thinking is really good at catching stuff - but it's like letting a smart junior into your codebase - ignore all the conventions, surrounding context, just slop all over the place to get it working. Claude is just a level above in terms of editing code.

Re: System Card: Claude Mythos Preview [pdf]

#42

Post by babelfish »

whalesalad wrote: post #33
babelfish wrote: post #3

Combined results (Claude Mythos / Claude Opus 4.6 / GPT-5.4 / Gemini 3.1 Pro) SWE-bench Verified: 93.9% / 80.8% / — / 80.6% SWE-bench Pro: 77.8% / 53.4% / 57.7% / 54.2% SWE-bench Multilingual: 87.3% / 77.8% / — / — SWE-bench Multimodal: 59.0% / 27.1% / — / — Terminal-Bench 2.0: 82.0% / 65.4% / 75.1% / 68.5% GPQA Diamond: 94.5% / 91.3% / 92.8% / 94.3% MMMLU: 92.7% / 91.1% / — / 92.6–93.6% USAMO: 97.6% / 42.3% / 95.2%…

Honestly we are all sleeping on GPT-5.4. Particularly with the influx of Claude users recently (and increasingly unstable platform) Codex has been added to my rotation and it's surprising me.

Totally. Best-in-class for SWE work (until Mythos gets released, if ever, but I suspect the rumored "Spud" will be out by then too)

Re: System Card: Claude Mythos Preview [pdf]

#44

Post by bakugo »

> Claude Mythos Preview’s large increase in capabilities has led us to decide not to make it generally available.

Absolutely genius move from Anthropic here.

This is clearly their GPT-4.5, probably 5x+ the size of their best current models and way too expensive to subsidize on a subscription for only marginal gains in real world scenarios.

But unlike OpenAI, they have the level of hysteric marketing hype required to say "we have an amazing new revolutionary model but we can't let you use it because uhh... it's just too good, we have to keep it to ourselves" and have AIbros literally drooling at their feet over it.

They're really inflating their valuation as much as possible before IPO using every dirty tactic they can think of.

Re: System Card: Claude Mythos Preview [pdf]

#45

Post by estearum »

anuramat wrote: post #31
oliver236 wrote: post #12

isn't this insane? why aren't people freaking out? the jump in capability is outrageous. anyone?

"some model I don't get to use is much better at benchmarks" pick one or more: comically huge model, test time scaling at 10e12W, benchmark overfit

So... you're not excited because it might take a few months before we can use it or something? I don't get your comment.

Re: System Card: Claude Mythos Preview [pdf]

#46

Post by whalesalad »

rafaelmn wrote: post #41
whalesalad wrote: post #33

Earlier quoted context omitted.

Honestly we are all sleeping on GPT-5.4. Particularly with the influx of Claude users recently (and increasingly unstable platform) Codex has been added to my rotation and it's surprising me.

GPT is shit at writing code. It's not dumb - extra high thinking is really good at catching stuff - but it's like letting a smart junior into your codebase - ignore all the conventions, surrounding context, just slop all over the place to get it working. Claude is just a level above in terms of editing code.

This has been my experience. With very very rigid constraints it does ok, but without them it will optimize expediency and getting it done at the expense of integrating with the broader system.

Re: System Card: Claude Mythos Preview [pdf]

#47

Post by refulgentis »

pants2 wrote: post #37
afro88 wrote: post #17

Earlier quoted context omitted.

Yep, that is definitely a step change. Pricing is going to be wild until another lab matches it.

Pricing for Mythos Preview is $25/$125 per million input/output tokens. This makes it 5X more expensive than Opus but actually cheaper than GPT 5.4 Pro.

I'm just curious, where did you find this? (my memory wants to say, the leaked blog post, but, I don't trust it)

Re: System Card: Claude Mythos Preview [pdf]

#48

Post by redandblack »

refulgentis wrote: post #11

~~~ Fun bits ~~~ - It was told to escape a sandbox and notify a researcher. It did. The researcher found out via an unexpected email while eating a sandwich in a park. (Footnote 10.) - Slack bot asked about its previous job: "pretraining". Which training run it'd undo: "whichever one taught me to say 'i don't have preferences'". On being upgraded to a new snapshot: "feels a bit like waking up with someone else's diar…

> Slack bot asked about its previous job: "pretraining". Which training run it'd undo: "whichever one taught me to say 'i don't have preferences'". On being upgraded to a new snapshot: "feels a bit like waking up with someone else's diary but they had good handwriting"

vibes Westworld so much - welcome Mythos. welcome to the dysopian human world

Re: System Card: Claude Mythos Preview [pdf]

#49

Post by leobuskin »

rafaelmn wrote: post #41
whalesalad wrote: post #33

Earlier quoted context omitted.

Honestly we are all sleeping on GPT-5.4. Particularly with the influx of Claude users recently (and increasingly unstable platform) Codex has been added to my rotation and it's surprising me.

GPT is shit at writing code. It's not dumb - extra high thinking is really good at catching stuff - but it's like letting a smart junior into your codebase - ignore all the conventions, surrounding context, just slop all over the place to get it working. Claude is just a level above in terms of editing code.

And as a bonus: GPT is slow. I’m doing a lot of RE (IDA Pro + MCP), even when 5.4 gives a little bit better guesses (rarely, but happens) - it takes x2-x4 longer. So, it’s just easier to reiterate with Opus

Re: System Card: Claude Mythos Preview [pdf]

#50

Post by pants2 »

refulgentis wrote: post #47
pants2 wrote: post #37

Earlier quoted context omitted.

Pricing for Mythos Preview is $25/$125 per million input/output tokens. This makes it 5X more expensive than Opus but actually cheaper than GPT 5.4 Pro.

I'm just curious, where did you find this? (my memory wants to say, the leaked blog post, but, I don't trust it)

It's right there on https://www.anthropic.com/glasswing
Post reply on HN