Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small models also found the vulnerabilities that Mythos found

aisle.com

Post reply on HN
Print view View original

351–360 of 372 posts

Re: Small models also found the vulnerabilities that Mythos found

#351

Post by shmagadee »

johnfn wrote: post #15

The Anthropic writeup addresses this explicitly: > This was the most critical vulnerability we discovered in OpenBSD with Mythos Preview after a thousand runs through our scaffold. Across a thousand runs through our scaffold, the total cost was under $20,000 and found several dozen more findings. While the specific run that found the bug above cost under $50, that number only makes sense with full hindsight. Like any…

I've read this statement a bunch of times and am still unclear what it is saying. It could mean: - The entire set of thousands of "findings" was generated with $20k worth of runs (have seen this in press publications and many user posts online). - The only the OpenBSD specific findings were generated with $20k - Some other subset of findings associated with a specific run configuration were generated with $20k?

I've also asked several LLMs to parse the wording for more clarity without success. They all highlight it as ambiguous wording. Why not use more direct language and provide the supporting data? They also stated that they are providing $100M in credits to their partners. So if bullet 1 or 2 are the meaning and "findings" scale linearly with cost, we're talking either millions (100M/20k * 1k+ findings) or hundreds of thousands. Does that make any sense? Or is the idea that all of these companies will run scans across their critical codebases continuously? Anyone else have a better sense of the math going on here?

Re: Small models also found the vulnerabilities that Mythos found

#353

Post by jgalt212 »

sweezyjeezy wrote: post #125
johnfn wrote: post #87

Earlier quoted context omitted.

> Wasn't the scaffolding for the Mythos run basically a line of bash that loops through every file of the codebase and prompts the model to find vulnerabilities in it? That sounds pretty close to "any gold there?" to me, only automated. But the entire value is that it can be automated. If you try to automate a small model to look for vulnerabilities over 10,000 files, it's going to say there are 9,500 vulns. Or none.…

> But the entire value is that it can be automated. If you try to automate a small model to look for vulnerabilities over 10,000 files, it's going to say there are 9,500 vulns. Or none. 'Or none' is ruled out since it found the same vulnerability - I agree that there is a question on precision on the smaller model, but barring further analysis it just feels like '9500' is pure vibes from yourself? Also (out of intere…

or run multiple cheap models in parallel: MOE^n, in effect.

Re: Small models also found the vulnerabilities that Mythos found

#354

Post by gspetr »

naasking wrote: post #336
jorvi wrote: post #227

Earlier quoted context omitted.

My proof-in-pudding test is still the fact that we haven't seen gigantic mass firings at tech companies, nor a massive acceleration on quality or breadth (not quantity!) of development. Microsoft has been going heavy on AI for 1y+ now. But then they replace their cruddy native Windows Copilot application with an Electron one. If tests and dev only has marginal cost now, why aren't they going all in on writing extreme…

> My proof-in-pudding test is still the fact that we haven't seen gigantic mass firings at tech companies Jevon's paradox.

For Jevons paradox to be a win-win, you need these 3 statements to be true:

1)Workers get more productive thanks to AI.

2)Higher worker productivity translates into lower prices.

3)Most importantly, consumer demand needs to explode in reaction to lower prices. And we're finding out in real-time that the demand is inelastic.

Around 1900, 40% of American workers worked in agriculture. Today, it's Which is similar to what we see with coding: The increase in demand has not exploded enough to offset the job-killing of each farmer being able to produce more food.

Re: Small models also found the vulnerabilities that Mythos found

#355

Post by AbstractH24 »

tombert wrote: post #92
tptacek wrote: post #20

If you cut out the vulnerable code from Heartbleed and just put it in front of a C programmer, they will immediately flag it. It's obvious. But it took Neel Mehta to discover it. What's difficult about finding vulnerabilities isn't properly identifying whether code is mishandling buffers or holding references after freeing something; it's spotting that in the context of a large, complex program, and working out how a…

It's weird, because when working on a big project, taking a break for a week or two, and returning to it, I will find a bug and will see hundreds of lines of code that are absolutely terrible, and I will tell myself "Tom you know better than to do this, this is a rookie mistake". I think people forget that it's hard to be clever and tidy 100% of the time. Big programs take a lot of discipline and an understanding of…

> I think people forget that it's hard to be clever and tidy 100% of the time

People on the outside with imposter syndrome also need to remember this.

Any mature codebase is a bit messy.

Re: Small models also found the vulnerabilities that Mythos found

#356

Post by AbstractH24 »

johnfn wrote: post #15

The Anthropic writeup addresses this explicitly: > This was the most critical vulnerability we discovered in OpenBSD with Mythos Preview after a thousand runs through our scaffold. Across a thousand runs through our scaffold, the total cost was under $20,000 and found several dozen more findings. While the specific run that found the bug above cost under $50, that number only makes sense with full hindsight. Like any…

So the real learning here is the cost of “using” GenAI to do things is declining at a rapid speed.

We’re not doing anything that couldn’t be done before, we’re just doing it faster, easier and cheaper.

Sounds like a recipe for a lot of junk being built. Also sounds like something that’s been true since the beginning of humanity.

In the more near term, sounds like a reminder the datacenters and processing boom will look at lot like the fiber one.

Re: Small models also found the vulnerabilities that Mythos found

#358

Post by epistasis »

zelphirkalt wrote: post #325
epistasis wrote: post #119

Earlier quoted context omitted.

If that's the point they are making, let's see their false positive rate that it produces on the entire codebase. They measured false negatives on a handful of cases, but that is not enough to hint at the system you suggest. And based on my experiences with $$$ focused eval products that you can buy right now, e.g. greptile, the false positive rate will be so high that it won't be useful to do full codebase scans thi…

How do we know the false positives for this "Mythos" thingamabob? Since they didn't release it, and we cannot reproduce it, are we to simply believe their word on this? What if the author of the featured article simply made a claim about that? We also simply believe their word? To me these AI tech companies are not any more trustworthy than a random blog author, maybe even less so, due to all the shady stuff they are…

That they were able to use it for security scanning puts the false positive rate at a useable level, inherently.

Maybe they spent more on labor to comb through reports than they did on the hardware costs of discovery, but if so I think we'd be hearing from third parties about how useless those millions in Mythos credits were that they got.

Re: Small models also found the vulnerabilities that Mythos found

#359

Post by fred_is_fred »

tptacek wrote: post #20

If you cut out the vulnerable code from Heartbleed and just put it in front of a C programmer, they will immediately flag it. It's obvious. But it took Neel Mehta to discover it. What's difficult about finding vulnerabilities isn't properly identifying whether code is mishandling buffers or holding references after freeing something; it's spotting that in the context of a large, complex program, and working out how a…

>If you cut out the vulnerable code from Heartbleed and just put it in front of a C programmer, they will immediately flag it. It's obvious.

Genuinely curious - why couldn't a static analyzer also find the issue then? Those have been worked on for 50+ years at this point, maybe longer.

Re: Small models also found the vulnerabilities that Mythos found

#360

Post by direwolf20 »

idopmstuff wrote: post #160
sweezyjeezy wrote: post #125

Earlier quoted context omitted.

> But the entire value is that it can be automated. If you try to automate a small model to look for vulnerabilities over 10,000 files, it's going to say there are 9,500 vulns. Or none. 'Or none' is ruled out since it found the same vulnerability - I agree that there is a question on precision on the smaller model, but barring further analysis it just feels like '9500' is pure vibes from yourself? Also (out of intere…

> 'Or none' is ruled out since it found the same vulnerability It's not, though. It wasn't asked to find vulnerabilities over 10,000 files - it was asked to find a vulnerability in the one particular place in which the researchers knew there was a vulnerability. That's not proof that it would have found the vulnerability if it had been given a much larger surface area to search.

Mythos was also asked to find a vulnerability in one file, in turn for each file. Maybe the small model needs to be asked about each function instead of each file. Okay, you can still automate that.
Post reply on HN