Small models also found the vulnerabilities that Mythos found

> We took the specific vulnerabilities Anthropic showcases in their announcement, isolated the relevant code, and ran them through small, cheap, open-weights models. Those models recovered much of the same analysis. Eight out of eight models detected Mythos's flagship FreeBSD exploit, including one with only 3.6 billion active parameters costing $0.11 per million tokens.

Impressive, and very valuable work, but isolating the relevant code changes the situation so much that I'm not sure it's much of the same use case.

Being able to dump an entire code base and have the model scan it is they type of situation where it opens up vulnerability scans to an entirely larger class of people.

Re: Small models also found the vulnerabilities that Mythos found

#6

Post by MaxLeiter » Sat, Apr 11, 2026, 5:19 PM UTC

I think they key thing here is they "isolated the relevant code"

If the exploits exist in e.g. one file, great. But many complex zerodays and exploits are chains of various bugs/behaviors in complex systems.

Important research but I don’t think it dispels anything about Mythos

Re: Small models also found the vulnerabilities that Mythos found

#7

Post by JackYoustra » Sat, Apr 11, 2026, 5:21 PM UTC

> Isolated the relevant code

I mean isn't that most of it? If you put a snippet of code in front of me and said "there's probably a vulnerability here" I could probably spend a few hours (a much lower METR time!) and find it. It's a whole other ballgame to ask me with no context to come up with an exploit.

Re: Small models also found the vulnerabilities that Mythos found

#8

Post by neuzhou » Sat, Apr 11, 2026, 5:22 PM UTC

[dead]

Re: Small models also found the vulnerabilities that Mythos found

#9

Post by amazingamazing » Sat, Apr 11, 2026, 5:22 PM UTC

Did mythos isolate the code to begin with? Without a clear methodology that can be attempted with another model the whole thing is meaningless

Re: Small models also found the vulnerabilities that Mythos found

#10

Post by odie5533 » Sat, Apr 11, 2026, 5:23 PM UTC

epistasis wrote: post #5

> We took the specific vulnerabilities Anthropic showcases in their announcement, isolated the relevant code, and ran them through small, cheap, open-weights models. Those models recovered much of the same analysis. Eight out of eight models detected Mythos's flagship FreeBSD exploit, including one with only 3.6 billion active parameters costing $0.11 per million tokens. Impressive, and very valuable work, but isolat…

Isn't the difference just harness then? I can write a harness that chunks code into individual functions or groups of functions and then feed it into a vulnerability analysis agent.