Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Assessing Claude Mythos Preview's cybersecurity capabilities

red.anthropic.com

Post reply on HN
Print view View original

11–20 of 59 posts

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#11

Post by rfoo »

staticassertion wrote: post #10
rfoo wrote: post #9

Earlier quoted context omitted.

> Mythos Preview identified a memory-corruption vulnerability in a production memory-safe VMM. This vulnerability has not been patched, so we neither name the project nor discuss details of the exploit. Good morning Sir. > Has anything changed here? I don't pay much attention but KASLR was considered basically useless for preventing LPE a few years ago. No. It's still like this. Bonus point that there are always free…

Oh, that. That's true, I didn't know Mythos found that one. I guess I will not comment further on it until there's a write up (edited out a bit more). > It is easy to turn this into a denial-of-service attack on the host, and conceivably could be used as part of an exploit chain. So yeah, perhaps some evidence to what I'm getting at. Bug density is too low in that project, it's high enough in others. I'll be way way…

> I read about 30% and got bored.

I was lucky then :) Somehow I saw this first. And then the "somewhat reliably writing exploits for SpiderMonkey" part, and then the crypto libraries part. Finally I wonder why is there a Linux LPE mini writeup and realized it's the "automatically turn a syzkaller report to a working exploit" part.

Now that I read the first few things (meh bugs in OpenBSD, FFmpeg, FreeBSD etc) they are indeed all pretty boring!

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#12

Post by staticassertion »

rfoo wrote: post #11
staticassertion wrote: post #10

Earlier quoted context omitted.

Oh, that. That's true, I didn't know Mythos found that one. I guess I will not comment further on it until there's a write up (edited out a bit more). > It is easy to turn this into a denial-of-service attack on the host, and conceivably could be used as part of an exploit chain. So yeah, perhaps some evidence to what I'm getting at. Bug density is too low in that project, it's high enough in others. I'll be way way…

> I read about 30% and got bored. I was lucky then :) Somehow I saw this first. And then the "somewhat reliably writing exploits for SpiderMonkey" part, and then the crypto libraries part. Finally I wonder why is there a Linux LPE mini writeup and realized it's the "automatically turn a syzkaller report to a working exploit" part. Now that I read the first few things (meh bugs in OpenBSD, FFmpeg, FreeBSD etc) they ar…

If people want exploitable syzkaller reports, following spender is free!

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#13

Post by esafak »

awestroke wrote: post #4

This is becoming a bit scary. I almost hope we'll reach some kind of plateau for llm intelligence soon.

We need to promote alignment and other ethics benchmarks; we can't change what we don't measure. I don't even know any off the top of my head.

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#14

Post by pants2 »

AntiDyatlov wrote: post #2

A very good outcome for AI safety would be if when improved models get released, malicious actors use them to break society in very visible ways. Looks like we're getting close to that world.

It would certainly be good news for cybersecurity employment!

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#15

Post by hibikir »

awestroke wrote: post #4

This is becoming a bit scary. I almost hope we'll reach some kind of plateau for llm intelligence soon.

On a topic like cybersecurity, we never win by not looking: One needs top of the line knowledge of how to break a system to be able to protect it. We have that dilemma dealing with human experts: The same government sponsored unit that tells you that you need to update your encryption can hold on to the information and use it to exploit it at their leisure.

Given that it's absolutely impossible to stop people not aligned with us (for any definition of us) from doing AI research, the most reasonable way forward is to dedicate compute resources to the frontier, and to automatically send reasonable disclosures to major projects. It could in itself be a pretty reasonable product. Just like you pay for dubious security scans and publish that you are making them, an LLM company could offer actually expensive security reviews with a preview model, and charge accordingly.

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#16

Post by lebovic »

awestroke wrote: post #4

This is becoming a bit scary. I almost hope we'll reach some kind of plateau for llm intelligence soon.

A plateau is unlikely, at least for cybersecurity. RL scales well here and is replicable outside of Anthropic (rewards are verifiable, so setting up the training environment doesn't require that much cleverness).

The post also points out that the model wasn't trained specifically on cybersecurity, and that it was just a side-effect – so I think there's still a lot of headroom.

It's scary, but there's also some room for cautious non-pessimism. More people than ever can cause billions of dollars of damage in attacks now [1], but the same tools can be used for defensive use. For that reason, I'm more optimistic about mitigations in security vs. other risk areas like biosecurity.

[1]: https://www.noahlebovic.com/testing-an-autonomous-hacker/

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#17

Post by jryio »

staticassertion wrote: post #6

I'd love to see them point at a target that's not a decades old C/C++ codebase. Of the targets, only browsers are what should be considered hardened, and their biggest lever is sandboxing, which requires a lot of chained exploits to bypass - we're seeing that LLMs are fast to discover bugs, which means they can chain more easily. But bug density in these code bases is known to be extremely high - especially the under…

The majority of vulnerabilities are in newly committed lines of code. This has been shown again and again [1] [2]

From a marketing standpoint Anthropic is showing that they're able to direct 'compute' to find vulnerabilities where human time/cost is not efficient or effective.

Project Glasswing is attempting to pay off as many of these old vulnerabilities as possible now so the low-hanging fruit has already been picked.

The next generation of Mythos and real world vulnerabilities exploits are going to be in newly committed code...

[1]: https://dl.acm.org/doi/epdf/10.1145/2635868.2635880

[2]: https://arxiv.org/abs/2601.22196

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#18

Post by staticassertion »

jryio wrote: post #17
staticassertion wrote: post #6

I'd love to see them point at a target that's not a decades old C/C++ codebase. Of the targets, only browsers are what should be considered hardened, and their biggest lever is sandboxing, which requires a lot of chained exploits to bypass - we're seeing that LLMs are fast to discover bugs, which means they can chain more easily. But bug density in these code bases is known to be extremely high - especially the under…

The majority of vulnerabilities are in newly committed lines of code. This has been shown again and again [1] [2] From a marketing standpoint Anthropic is showing that they're able to direct 'compute' to find vulnerabilities where human time/cost is not efficient or effective. Project Glasswing is attempting to pay off as many of these old vulnerabilities as possible now so the low-hanging fruit has already been pick…

> The majority of vulnerabilities are in newly committed lines of code. This has been shown again and again

That's fine, I wouldn't argue against that. It doesn't really change things, right?

> From a marketing standpoint Anthropic is showing that they're able to direct 'compute' to find vulnerabilities where human time/cost is not efficient or effective.

Yes, they've demonstrated that.

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#20

Post by dang »

Related ongoing threads:

System Card: Claude Mythos Preview [pdf] - https://news.ycombinator.com/item?id=47679258

Project Glasswing: Securing critical software for the AI era - https://news.ycombinator.com/item?id=47679121

I can't tell which of the current threads, if any, should be merged - they all seem significant. Anyone?

Post reply on HN