Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small models also found the vulnerabilities that Mythos found

aisle.com

Post reply on HN
Print view View original

41–50 of 372 posts

Re: Small models also found the vulnerabilities that Mythos found

#41

Post by bhouston »

kennywinker wrote: post #29
chirau wrote: post #13

Their isolation approach is totally different from Mythos approach though. Mythos had to evaluate whole code bases rather than isolated sections. It's like saying one dog walked into the Amazon jungle and found a tennis ball and then another team isolated a 1 square kilometer radius that they knew the ball was definitely in and found the same ball.

I don’t think mythos can ingest an entire codebase into context. So it’s spinning off sub-agents to process chunks. Which supports their thesis: the harness is the moat. The tooling is whats important, the model is far far less important.

Mythos was clear it was one agent per chunk. But this positive confirming results do not actually disprove anytime with Mythos, because it is only one side of the discriminator challenge - you got positives, but we do not know your false positive rate and your false negative rate.

Re: Small models also found the vulnerabilities that Mythos found

#42

Post by loeg »

amazingamazing wrote: post #9

Did mythos isolate the code to begin with? Without a clear methodology that can be attempted with another model the whole thing is meaningless

No. How would it? Before the vulns were identified by Mythos, no one knew what the relevant portion to isolate was.

Re: Small models also found the vulnerabilities that Mythos found

#43

Post by bhouston »

amazingamazing wrote: post #9

Did mythos isolate the code to begin with? Without a clear methodology that can be attempted with another model the whole thing is meaningless

They did do one agent per code chunk, yes. But key is that their agent had to identify when there was a vulnerability and when there wasn't. This "small model" test only had to label the known positive cases as positive -- which any function that simply returns "true" can do. This whole test setup is annoying because it proves nothing.

Re: Small models also found the vulnerabilities that Mythos found

#45

Post by kennywinker »

bhouston wrote: post #41
kennywinker wrote: post #29

Earlier quoted context omitted.

I don’t think mythos can ingest an entire codebase into context. So it’s spinning off sub-agents to process chunks. Which supports their thesis: the harness is the moat. The tooling is whats important, the model is far far less important.

Mythos was clear it was one agent per chunk. But this positive confirming results do not actually disprove anytime with Mythos, because it is only one side of the discriminator challenge - you got positives, but we do not know your false positive rate and your false negative rate.

In TFA they talk a fair bit about how different models perform wrt false positives:

“The results show something close to inverse scaling: small, cheap models outperform large frontier ones.”

Re: Small models also found the vulnerabilities that Mythos found

#48

Post by SCHiM »

TacticalCoder wrote: post #38
elicash wrote: post #12

Earlier quoted context omitted.

This is from the first of the caveats that they list: > Scoped context: Our tests gave models the vulnerable function directly, often with contextual hints (e.g., "consider wraparound behavior"). A real autonomous discovery pipeline starts from a full codebase with no hints. The models' performance here is an upper bound on what they'd achieve in a fully autonomous scan. That said, a well-designed scaffold naturally…

> That's why their point is what the subheadline says, that the moat is the system, not the model. Can you expand a bit more on this? What is the system then in this case? And how was that model created? By AI? By humans?

You can imagine a pipeline that looks at individual source files or functions. And first "extracts" what is going on. You ask the model:

- "Is the code doing arithmetic in this file/function?" - "Is the code allocating and freeing memory in this file/function?" - "Is the code the code doing X/Y/Z? etc etc"

For each question, you design the follow-up vulnerability searchers.

For a function you see doing arithmetic, you ask:

- "Does this code look like integer overflow could take place?",

For memory:

- "Do all the pointers end up being freed?" _or_ - "Do all pointers only get freed once?"

I think that's the harness part in terms of generating the "bug reports". From there on, you'll need a bunch of tools for the model to interact with the code. I'd imagine you'll want to build a harness/template for the file/code/function to be loaded into, and executed under ASAN.

If you have an agent that thinks it found a bug: "Yes file xyz looks like it could have integer overflow in function abc at line 123, because...", you force another agent to load it in the harness under ASAN and call it. If ASAN reports a bug, great, you can move the bug to the next stage, some sort of taint analysis or reach-ability analysis.

So at this point you're running a pipeline to: 1) Extract "what this code does" at the file, function or even line level. 2) Put code you suspect of being vulnerable in a harness to verify agent output. 3) Put code you confirmed is vulnerable into a queue to perform taint analysis on, to see if it can be reached by attackers.

Traditionally, I guess a fuzzer approached this from 3 -> 2, and there was no "stage 1". Because LLMs "understand" code, you can invert this system, and work if up from "understanding", i.e. approach it from the other side. You ask, given this code, is there a bug, and if so can we reach it?, instead of asking: given this public interface and a bunch of data we can stuff in it, does something happen we consider exploitable?

Re: Small models also found the vulnerabilities that Mythos found

#49

Post by SoftTalker »

johnfn wrote: post #15

The Anthropic writeup addresses this explicitly: > This was the most critical vulnerability we discovered in OpenBSD with Mythos Preview after a thousand runs through our scaffold. Across a thousand runs through our scaffold, the total cost was under $20,000 and found several dozen more findings. While the specific run that found the bug above cost under $50, that number only makes sense with full hindsight. Like any…

How much of that is simply scale? Anthropic threw probably an entire data center at analyzing a code base. Has anyone done the same with a "small" model?

Re: Small models also found the vulnerabilities that Mythos found

#50

Post by vmg12 »

The technique Anthropic uses was demonstrated by Nicholas Carlini in a talk he gave 2 weeks ago and it's very simple, when asking LLMs to review code, ask them to focus its review on one file in a single session. Here is the video with the timestamp (watch through to ~5:30, they show two different ways of prompting claude).

https://youtu.be/1sd26pWhfmg?t=204

https://youtu.be/1sd26pWhfmg?t=273

IMO the big "innovation" being shown by Mythos is the effectiveness with prompting LLMs to look for security vulnerabilities by focusing on specific files one at a time and automating this prompting with a simple script.

Prompting Mythos to focus on a single file per session is why I suspect it cost Anthropic $20k to find some of the bugs in these codebases. I know this same technique is effective with Opus 4.6 and GPT 5.4 because I've been using it on my own code. If you just ask the agent to review your pr with a low effort prompt they are not exhaustive, they will not actually read each changed file and look at how it interacts with the system as a whole. If the entire session is to review the changes for a single file, the llm will do much more work reviewing it.

Edit: I changed my phrasing, it's not about restricting its entire context to one file but focusing it on one file but still allowing it to look at how other files interact with it.

Post reply on HN