Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small models also found the vulnerabilities that Mythos found

aisle.com

Post reply on HN
Print view View original

241–250 of 372 posts

Re: Small models also found the vulnerabilities that Mythos found

#241

Post by windexh8er »

tptacek wrote: post #20

If you cut out the vulnerable code from Heartbleed and just put it in front of a C programmer, they will immediately flag it. It's obvious. But it took Neel Mehta to discover it. What's difficult about finding vulnerabilities isn't properly identifying whether code is mishandling buffers or holding references after freeing something; it's spotting that in the context of a large, complex program, and working out how a…

What's weird is that Google, Anthropic and OpenAI are claiming the model is the powerhouse, when what Aisle is stating is very much not the case.

It almost seems like a coordinated effort (Google in January, Anthropic and OAI in April) building out gated models that will eventually be very expensive. Yet, here we are: Aisle is saying that's not required to get there.

I don't think it's weird at all. It seems to me the Frontier providers are just trying to find, still unsuccessfully, a moat to make their unsustainable business model... Well. Sustainable.

Re: Small models also found the vulnerabilities that Mythos found

#242

Post by wat10000 »

mattmanser wrote: post #181
wat10000 wrote: post #57

Earlier quoted context omitted.

I meant, if the claim here is that small models can accomplish the same things with good scaffolding, why didn’t they demonstrate finding those problem with good scaffolding rather than directly pointing them at the problem?

They don't have to. Lot of people in this thread don't seem to be getting that. If another model can find the vulnerability if you point it at the right place, it would also find the vulnerability if you scanned each place individually. People are talking about false positives, but that also doesn't matter. Again, they're not thinking it through. False positives don't matter, as you can just automatically try and exp…

It’s strange to see this constant “I could do that too, I just don’t want to” response.

Finding an important decades-old vulnerability in OpenBSD is extremely impressive. That’s the sort of thing anyone would be proud to put on their resume. Small models are available for anyone to use. Scaffolding isn’t that hard to build. So why didn’t someone use this technique to find this vulnerability and make some headlines before Anthropic did? Either this technique with small models doesn’t actually work, or it does work but nobody’s out there trying it for some reason. I find the second possibility a lot less plausible than the first.

Re: Small models also found the vulnerabilities that Mythos found

#243

Post by Sparkyte »

johnfn wrote: post #15

The Anthropic writeup addresses this explicitly: > This was the most critical vulnerability we discovered in OpenBSD with Mythos Preview after a thousand runs through our scaffold. Across a thousand runs through our scaffold, the total cost was under $20,000 and found several dozen more findings. While the specific run that found the bug above cost under $50, that number only makes sense with full hindsight. Like any…

Why not just write many small models for explicit tasks than running one bigger model anyway? I prefer the agentic subject matter expert design anyway. I suppose because it wants to look at the whole code base?

Re: Small models also found the vulnerabilities that Mythos found

#244

Post by apgwoz »

woeirua wrote: post #194
apgwoz wrote: post #171

Earlier quoted context omitted.

The benefit here is reducing the time to find vulnerabilities; faster than humans, right? So if you can rig a harness for each function in the system, by first finding where it’s used, its expected input, etc, and doing that for all functions, does it discover vulnerabilities faster than humans? Doesn’t matter that they isolated one thing. It matters that the context they provided was discoverable by the model.

There is absolutely zero reason to believe you could use this same approach to find and exploit vulns without Mythos finding them first. We already know that older LLMs can’t do what Mythos has done. Anthropic and others have been trying for years.

Why? They claim this small model found a bug given some context. I assume the context wasn’t “hey! There’s a very specific type of bug sitting in this function when certain conditions are met.”

We keep assuming that the models need to get bigger and better, and the reality is we’ve not exhausted the ways in which to use the smaller models. It’s like the Playstation 2 games that came out 10 years later. Well now all the tricks were found, and everything improved.

Re: Small models also found the vulnerabilities that Mythos found

#245

Post by bloppe »

windexh8er wrote: post #241
tptacek wrote: post #20

If you cut out the vulnerable code from Heartbleed and just put it in front of a C programmer, they will immediately flag it. It's obvious. But it took Neel Mehta to discover it. What's difficult about finding vulnerabilities isn't properly identifying whether code is mishandling buffers or holding references after freeing something; it's spotting that in the context of a large, complex program, and working out how a…

What's weird is that Google, Anthropic and OpenAI are claiming the model is the powerhouse, when what Aisle is stating is very much not the case. It almost seems like a coordinated effort (Google in January, Anthropic and OAI in April) building out gated models that will eventually be very expensive. Yet, here we are: Aisle is saying that's not required to get there. I don't think it's weird at all. It seems to me th…

I agree that the apocalyptic messaging about mythos is eye-rolling, but the thesis of the article that "the moat is the system, not the model" is weird because the point is that the model is the whole system. A little Bash loop that just tells the model to "look at this file" for every file is clearly not a "moat" of a system

Re: Small models also found the vulnerabilities that Mythos found

#246

Post by Filligree »

nozzlegear wrote: post #205
woeirua wrote: post #194

Earlier quoted context omitted.

There is absolutely zero reason to believe you could use this same approach to find and exploit vulns without Mythos finding them first. We already know that older LLMs can’t do what Mythos has done. Anthropic and others have been trying for years.

> There is absolutely zero reason to believe you could use this same approach to find and exploit vulns without Mythos finding them first. There's one huge reason to believe it: we can actually use small models, but we cant use Anthropic's special marketing model that's too dangerous for mere mortals.

If all you have is a spade, that is _not_ evidence that spades are good for excavating an entire hill.

Re: Small models also found the vulnerabilities that Mythos found

#247

Post by airstrike »

bloppe wrote: post #245
windexh8er wrote: post #241

Earlier quoted context omitted.

What's weird is that Google, Anthropic and OpenAI are claiming the model is the powerhouse, when what Aisle is stating is very much not the case. It almost seems like a coordinated effort (Google in January, Anthropic and OAI in April) building out gated models that will eventually be very expensive. Yet, here we are: Aisle is saying that's not required to get there. I don't think it's weird at all. It seems to me th…

I agree that the apocalyptic messaging about mythos is eye-rolling, but the thesis of the article that "the moat is the system, not the model" is weird because the point is that the model is the whole system. A little Bash loop that just tells the model to "look at this file" for every file is clearly not a "moat" of a system

While I agree this is true of coding, there are other domains and paradigms in which the loop is more involved than a bash loop.

Realizing this fact explains:

1. why software development is first to get disrupted by AI

2. other domains that are easily loopable like contract review are also quite easy to deploy AI into, so you get all these "AI for Law" running around doing essentially the same thing

3. domains that are not easily loopable are much harder to figure out leading people to believe AI can't be useful, when in fact it's a failure of the application layer

Re: Small models also found the vulnerabilities that Mythos found

#248

Post by johnfn »

jorvi wrote: post #227
johnfn wrote: post #154

Earlier quoted context omitted.

Admittedly just vibes from me, having pointed small models at code and asked them questions, no extensive evaluation process or anything. For instance, I recall models thinking that every single use of `eval` in javascript is a security vulnerability, even something obviously benign like `eval("1 + 1")`. But then I'm only posting comments on HN, I'm not the one writing an authoritative thinkpiece saying Mythos actual…

My proof-in-pudding test is still the fact that we haven't seen gigantic mass firings at tech companies, nor a massive acceleration on quality or breadth (not quantity!) of development. Microsoft has been going heavy on AI for 1y+ now. But then they replace their cruddy native Windows Copilot application with an Electron one. If tests and dev only has marginal cost now, why aren't they going all in on writing extreme…

Your proof-in-pudding test seems to assume that AI is binary -- either it accelerates everyone's development 100x ("let's rewrite every app into bug-free native applications") or nothing ("there hasn't been anything to show for that in years"). I posit reality is somewhere in between the two.

Re: Small models also found the vulnerabilities that Mythos found

#249

Post by klempner »

johnfn wrote: post #15

The Anthropic writeup addresses this explicitly: > This was the most critical vulnerability we discovered in OpenBSD with Mythos Preview after a thousand runs through our scaffold. Across a thousand runs through our scaffold, the total cost was under $20,000 and found several dozen more findings. While the specific run that found the bug above cost under $50, that number only makes sense with full hindsight. Like any…

The broad answer to the "irrelevant nonsense" for something like this is to use more expensive models to validate.

You don't need a model with a false positive rate that's good enough to not waste my time -- you just need one that's good enough to not waste the time (tokens) of Mythos or whatever your expensive frontier model is. Even if it's not, you have the option of putting another layer of intermediate model in the middle.

Re: Small models also found the vulnerabilities that Mythos found

#250

Post by usef- »

apgwoz wrote: post #244
woeirua wrote: post #194

Earlier quoted context omitted.

There is absolutely zero reason to believe you could use this same approach to find and exploit vulns without Mythos finding them first. We already know that older LLMs can’t do what Mythos has done. Anthropic and others have been trying for years.

Why? They claim this small model found a bug given some context. I assume the context wasn’t “hey! There’s a very specific type of bug sitting in this function when certain conditions are met.” We keep assuming that the models need to get bigger and better, and the reality is we’ve not exhausted the ways in which to use the smaller models. It’s like the Playstation 2 games that came out 10 years later. Well now all t…

If this were true, we're essentially saying that no one tried to scan vulnerabilities using existing models, despite vulnerabilities being extremely lucrative and a large professional industry. Vulnerability research has been one of the single most talked about risks of powerful AI so it wasn't exactly a novel concept, either.

If it is true that existing models can do this, it would imply that LLMs are being under marketed, not over marketed, since industry didn't think this was worth trying previously(?). Which I suspect is not the opinion of HN upvoters here.

Post reply on HN