Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

System Card: Claude Mythos Preview [pdf]

www-cdn.anthropic.com

Post reply on HN
Print view View original

641–650 of 687 posts

Re: System Card: Claude Mythos Preview [pdf]

#641

Post by yencabulator »

mike_hearn wrote: post #572
Jcampuzano2 wrote: post #59

Earlier quoted context omitted.

A jump that we will never be able to use since we're not part of the seemingly minimum 100 billion dollar company club as requirement to be allowed to use it. I get the security aspect, but if we've hit that point any reasonably sophisticated model past this point will be able to do the damage they claim it can do. They might as well be telling us they're closing up shop for consumer models. They should just say they…

I think they already said somewhere that they can't release Mythos because it requires absurdly large amounts of compute. The economics of releasing it just don't work.

Yet they quote a $20,000 cost for one of the exploits.

Re: System Card: Claude Mythos Preview [pdf]

#642

Post by sacrosaunt »

raldi wrote: post #609
yismail wrote: post #192

I wonder what the relationship is between a model's capability and the personality it develops. Page 202: > In interactions with subagents, internal users sometimes observed that Mythos Preview appeared “disrespectful” when assigning tasks. It showed some tendency to use commands that could be read as “shouty” or dismissive, and in some cases appeared to underestimate subagent intelligence by overexplaining trivial t…

Could you transcribe the emoji? HN strips them out.

Cosmic set [:sparkles: :dizzy: :star2: :infinity: :performing_arts:] Functional set [:wave: :thumbsup: :slightly_smiling_face:] Nature set [:handshake: :pray: :ocean: :seedling: :new_moon:]

Re: System Card: Claude Mythos Preview [pdf]

#643

Post by SkyPuncher »

rbliss wrote: post #531
SkyPuncher wrote: post #343

Earlier quoted context omitted.

Yes, I agree. I’m about to drop Claude Code because it’s become literally unusable. Today, Opus went in circles trying to get a toggle button to work.

Same. Asked CC Opus about a change in a particular file...it looked in a totally different file and told me there was no change.

I've switched to max thinking mode as my default and that's helping in some capacity.

It's not necessarily back to where it was, but it's not desk-flipping bad.

Re: System Card: Claude Mythos Preview [pdf]

#644

Post by TimTheTinker »

sigmoid10 wrote: post #635
TimTheTinker wrote: post #612

Earlier quoted context omitted.

LLMs have zero metacognition. Don't be fooled - their output is stochastic inference and they have no self-awareness. The best you'll see is an improvised post-hoc rationalization story.

You can turn all these argents around and prove the same is true for humans. Don't be fooled by dogmatic people who spread the idea that the human mind is the pinnacle of cognition in the universe. Best to leave that to religion.

Humans may not always be that smart, but we do at least have an internal state and an awareness of that internal state - a "self-awareness".

AI most certainly has nothing of the sort, and any appearance to the contrary is the direct result of training data.

Re: System Card: Claude Mythos Preview [pdf]

#645

Post by TimTheTinker »

naasking wrote: post #634
TimTheTinker wrote: post #612

Earlier quoted context omitted.

LLMs have zero metacognition. Don't be fooled - their output is stochastic inference and they have no self-awareness. The best you'll see is an improvised post-hoc rationalization story.

> The best you'll see is an improvised post-hoc rationalization story. Funny, because "post-hoc rationalization" is how many neuroscientists think humans operate. That LLMs are stochastic inference engines is obvious by construction, but you skipped the step where you proved that human thoughts, self-awareness and metacognition are not reducible to stochastic inference.

I'm not saying we don't do post-hoc rationalization. But self-awareness is a trait we possess to varying degrees, and reporting on a memory of a past internal state is at least sometimes possible, even if we don't always choose to do so.

Re: System Card: Claude Mythos Preview [pdf]

#646

Post by casey2 »

babelfish wrote: post #3

Combined results (Claude Mythos / Claude Opus 4.6 / GPT-5.4 / Gemini 3.1 Pro) SWE-bench Verified: 93.9% / 80.8% / — / 80.6% SWE-bench Pro: 77.8% / 53.4% / 57.7% / 54.2% SWE-bench Multilingual: 87.3% / 77.8% / — / — SWE-bench Multimodal: 59.0% / 27.1% / — / — Terminal-Bench 2.0: 82.0% / 65.4% / 75.1% / 68.5% GPQA Diamond: 94.5% / 91.3% / 92.8% / 94.3% MMMLU: 92.7% / 91.1% / — / 92.6–93.6% USAMO: 97.6% / 42.3% / 95.2%…

Lots of benchmaxxing here. A few simple randomizations puts it back on par with gemini 3.1 and under 5.4 pro in most benchmarks

Re: System Card: Claude Mythos Preview [pdf]

#647

Post by SparkyMcUnicorn »

andai wrote: post #522
SparkyMcUnicorn wrote: post #357

Earlier quoted context omitted.

https://marginlab.ai/trackers/claude-code-historical-perform...

This just looks like random noise to me? Is it also random on short timespans, like running it 10x in a row?

Explained in the methodology at the bottom of this page: https://marginlab.ai/trackers/claude-code/

Re: System Card: Claude Mythos Preview [pdf]

#648

Post by mik09 »

babelfish wrote: post #3

Combined results (Claude Mythos / Claude Opus 4.6 / GPT-5.4 / Gemini 3.1 Pro) SWE-bench Verified: 93.9% / 80.8% / — / 80.6% SWE-bench Pro: 77.8% / 53.4% / 57.7% / 54.2% SWE-bench Multilingual: 87.3% / 77.8% / — / — SWE-bench Multimodal: 59.0% / 27.1% / — / — Terminal-Bench 2.0: 82.0% / 65.4% / 75.1% / 68.5% GPQA Diamond: 94.5% / 91.3% / 92.8% / 94.3% MMMLU: 92.7% / 91.1% / — / 92.6–93.6% USAMO: 97.6% / 42.3% / 95.2%…

it might have broken a couple metrics since if you get above 90 percent it might be that the metric can not measure you well anymore right?

Re: System Card: Claude Mythos Preview [pdf]

#649

Post by mik09 »

Zee2 wrote: post #208
tony_cannistra wrote: post #22

> Claude Mythos Preview is, on essentially every dimension we can measure, the best-aligned model that we have released to date by a significant margin. We believe that it does not have any significant coherent misaligned goals, and its character traits in typical conversations closely follow the goals we laid out in our constitution. Even so, we believe that it likely poses the greatest alignment-related risk of any…

Alignment “appearing” better as model capabilities increase scares the shit out of me, tbh.

yeah anthropic tries to address this through mechanistic interpretation but not sure they are progressing as fast in that domain as their model development

Re: System Card: Claude Mythos Preview [pdf]

#650

Post by mik09 »

goldenarm wrote: post #281
apetresc wrote: post #191

I've long maintained that the real indicator that AGI is imminent is that public availability stops being a thing. If you truly believed you had a superhuman, godlike mind in your thrall, renting it out for $20/month would be the last thing you would choose to do with it.

Simpler explanation : they don't have enough GPUs to release this much larger model.

but simpler explanation work until they don't all of a sudden
Post reply on HN