Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small models also found the vulnerabilities that Mythos found

aisle.com

Post reply on HN
Print view View original

201–210 of 372 posts

Re: Small models also found the vulnerabilities that Mythos found

#201

Post by andy_ppp »

johnfn wrote: post #15

The Anthropic writeup addresses this explicitly: > This was the most critical vulnerability we discovered in OpenBSD with Mythos Preview after a thousand runs through our scaffold. Across a thousand runs through our scaffold, the total cost was under $20,000 and found several dozen more findings. While the specific run that found the bug above cost under $50, that number only makes sense with full hindsight. Like any…

I wonder if you could just setup a small model and suggest a load of things and try every file and it might still end up being cheaper and just as good as Mythos at a specific task. Maybe this will be something that holds true for more things, formulating a small model to do specific things may well end up being as effective/efficient as a larger model looking at a huge solution space.

Re: Small models also found the vulnerabilities that Mythos found

#202

Post by tehryanx »

cyanydeez wrote: post #147
tehryanx wrote: post #127

Earlier quoted context omitted.

newer models have larger context windows, and more stable reasoning across larger context windows. If you point your model directly at the thing you want it to assess, and it doesn't have to gather any additional context you're not really testing those things at all. Say you point kimi and opus at some code and give them an agentic looping harness with code review tools. They're going to start digging into the code g…

Harnesses are basically doing this better than just adding more context. Every time, REGARDLESS OF MODEL SIZE, you add context, you are increasing the odds the model will get confused about any set of thoughts. So context size is no longer some magic you just sprinkle on these things and they suddenly dont imagine things. So, it's the old ML join: It's just a bunch of if statements. As others are pointing out, it's q…

I know you're right that there's a saturation point for context size, but it's not just context size that the larger models have, it's better grounding within that as a result of stronger, more discriminative attention patterns.

I'm not saying you're not going to drive confusion by overloading context, but the number of tokens required to trigger that failure mode in opus is going to be a lot higher than the number for gpt-oss-20b.

I'm pretty sure a model that can run on a cellphone is going to cap out it's context window long before opus or mythos would hit the point of diminishing returns on context overload. I think using a lower quality model with far fewer / noisier weights and less precise attention is going to drive false positives way before adding context to a SOTA model will.

You can even see here, AISLE had to print a retraction because someone checked their work and found that just pointing gpt-oss-20b at the patched version generated FP consistently: https://x.com/ChaseBrowe32432/status/2041953028027379806

Re: Small models also found the vulnerabilities that Mythos found

#203

Post by johnfn »

cyanydeez wrote: post #112
Aurornis wrote: post #76

Earlier quoted context omitted.

How is that a direct comparison? The link you gave has a quote that says it’s not: > Scoped context: Our tests gave models the vulnerable function directly, often with contextual hints (e.g., "consider wraparound behavior"). A real autonomous discovery pipeline starts from a full codebase with no hints They pointed the models at the known vulnerable functions and gave them a hint. The hint part is what really breaks…

Does no one defending mythos understand how nested foreloops work? loop through each repo: loop through each file: opencode command /find_wraparoundvulnerability next file next repo I can run this on my local LLM and sure, I gotta wait some time for it to complete, but I see zero distinguishing facts here.

No one is saying your nested for loop idea because it won't actually work in practice. In short, the signal to noise ratio will be too high - you will need to comb through a ton of false positives in order to find anything valuable, at which point it stops looking like "automated security research" and it starts looking like "normal security research".

If you don't believe me, you should try it yourself, it's only a couple of dollars. Hey, maybe you're right, and you can prove us all wrong. But I'd bet you on great odds that you're not.

Re: Small models also found the vulnerabilities that Mythos found

#204

Post by nodja »

mirsadm wrote: post #80
vmg12 wrote: post #50

The technique Anthropic uses was demonstrated by Nicholas Carlini in a talk he gave 2 weeks ago and it's very simple, when asking LLMs to review code, ask them to focus its review on one file in a single session. Here is the video with the timestamp (watch through to ~5:30, they show two different ways of prompting claude). https://youtu.be/1sd26pWhfmg?t=204 https://youtu.be/1sd26pWhfmg?t=273 IMO the big "innovation"…

How is that going to find anything that interacts across files?

You misunderstood.

Instead of asking the model: "Here's this codebase, report any vulnerability." you ask. "Here's this codebase, report any vulnerability in module\main.c".

The model can still explore references and other files inside the codebase, but you start over a new context/session for each file in the codebase.

Re: Small models also found the vulnerabilities that Mythos found

#205

Post by nozzlegear »

woeirua wrote: post #194
apgwoz wrote: post #171

Earlier quoted context omitted.

The benefit here is reducing the time to find vulnerabilities; faster than humans, right? So if you can rig a harness for each function in the system, by first finding where it’s used, its expected input, etc, and doing that for all functions, does it discover vulnerabilities faster than humans? Doesn’t matter that they isolated one thing. It matters that the context they provided was discoverable by the model.

There is absolutely zero reason to believe you could use this same approach to find and exploit vulns without Mythos finding them first. We already know that older LLMs can’t do what Mythos has done. Anthropic and others have been trying for years.

> There is absolutely zero reason to believe you could use this same approach to find and exploit vulns without Mythos finding them first.

There's one huge reason to believe it: we can actually use small models, but we cant use Anthropic's special marketing model that's too dangerous for mere mortals.

Re: Small models also found the vulnerabilities that Mythos found

#206

Post by ijk »

philipallstar wrote: post #197
sumeno wrote: post #164

Earlier quoted context omitted.

Or $200,000 for consumers when they have to make a profit

Good point. This is why consumer phones have got much worse since 2005 and now cost millions of dollars.

With the way the chip shortage the way it is, I'm a little concerned that my next phone will be worse and more expensive...

Re: Small models also found the vulnerabilities that Mythos found

#207

Post by bloaf »

johnfn wrote: post #154
sweezyjeezy wrote: post #125

Earlier quoted context omitted.

> But the entire value is that it can be automated. If you try to automate a small model to look for vulnerabilities over 10,000 files, it's going to say there are 9,500 vulns. Or none. 'Or none' is ruled out since it found the same vulnerability - I agree that there is a question on precision on the smaller model, but barring further analysis it just feels like '9500' is pure vibes from yourself? Also (out of intere…

Admittedly just vibes from me, having pointed small models at code and asked them questions, no extensive evaluation process or anything. For instance, I recall models thinking that every single use of `eval` in javascript is a security vulnerability, even something obviously benign like `eval("1 + 1")`. But then I'm only posting comments on HN, I'm not the one writing an authoritative thinkpiece saying Mythos actual…

I remember a study from a while back that found something like "50% of 2nd graders think that french fries are made out of meat instead of potatoes. Methodology: we asked kids if french fries were meat or potatoes."

Everyone was going around acting like this meant 50% of 2nd graders were stupid with terrible parents. (Or, conversely, that 50% of 2nd graders were geniuses for "knowing" it was potatoes at all)

But I think that was the wrong conclusion.

The right conclusion was that all the kids guessed and they had a 50% chance of getting it right.

And I think there is probably an element of this going on with the small models vs big models dichotomy.

Re: Small models also found the vulnerabilities that Mythos found

#208

Post by Oranguru »

mattmanser wrote: post #181
wat10000 wrote: post #57

Earlier quoted context omitted.

I meant, if the claim here is that small models can accomplish the same things with good scaffolding, why didn’t they demonstrate finding those problem with good scaffolding rather than directly pointing them at the problem?

They don't have to. Lot of people in this thread don't seem to be getting that. If another model can find the vulnerability if you point it at the right place, it would also find the vulnerability if you scanned each place individually. People are talking about false positives, but that also doesn't matter. Again, they're not thinking it through. False positives don't matter, as you can just automatically try and exp…

[dead]

Re: Small models also found the vulnerabilities that Mythos found

#209

Post by cyanydeez »

tredre3 wrote: post #133
cyanydeez wrote: post #108

Earlier quoted context omitted.

so what you're saying is no one could ever write a loop like: for githubProject in githubProjects opencode command /findvulnerability end for Seems like a silly thing to try and back up.

What he's saying is that you should read the "Caveats and limitations" section of the article. Here's the first one: > Our tests gave models the vulnerable function directly, often with contextual hints (e.g., "consider wraparound behavior"). Mythos did no such thing, it was cut lose and told to find vulnerabilities. If the intent was to prove that small models are just as good, they haven't demonstrated that at all.…

ok, but you're missing the obvious: I could also give it the vulnerable function byt just looping over all functions and providing a small hint about what to look at.

Until "Mythos" is compared with the most bland and straight forward harness vs small model, there's no great context god that can't be emulated with deterministic scanning and context pulls.

Re: Small models also found the vulnerabilities that Mythos found

#210

Post by thmoonbus »

philipallstar wrote: post #197
sumeno wrote: post #164

Earlier quoted context omitted.

Or $200,000 for consumers when they have to make a profit

Good point. This is why consumer phones have got much worse since 2005 and now cost millions of dollars.

Now do uber rides
Post reply on HN