Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small models also found the vulnerabilities that Mythos found

aisle.com

Post reply on HN
Print view View original

211–220 of 372 posts

Re: Small models also found the vulnerabilities that Mythos found

#211

Post by etothet »

My big question around the Mythos FUD, is this: if we take for fact the Mythos is as powerful and dangerous as we’re being told (and I realize this is part marketing), and because of that Anthropic isn’t going to release it…how long can that last? Isn’t it reasonable that OpenAI or xAI or some other company - or foreign government - will come up with a similarly dangerous model fairly soon?

So what’s Anthropic’s plan here? How long can they withhold releasing Mythos or something Mythos-like? Is it reasonable to think they - or another AI provider - are going to dumb down future models so they’re less dangerous? I personally don’t think that’s the case.

I’m not saying Anthropic should or shouldn’t release Mythos, but it leaves me wonderingwhat’s going to be different in, say, 6 months or even a year when they or another provider releases a model as dangerous as we’re being told Mythos is?

Re: Small models also found the vulnerabilities that Mythos found

#212

Post by dev1ycan »

It was obvious since the start that 1)it's probably all javascript based or android websites/programs that contain a ton of "vulnerable" libraries (or really old closed sourced c++ code).

Also you're not helping your case as a software company if you feed your code to an LLM, great job making it all public, because it will most likely be used as training data like it or not.

Re: Small models also found the vulnerabilities that Mythos found

#213

Post by nimchimpsky »

elicash wrote: post #12
epistasis wrote: post #5

> We took the specific vulnerabilities Anthropic showcases in their announcement, isolated the relevant code, and ran them through small, cheap, open-weights models. Those models recovered much of the same analysis. Eight out of eight models detected Mythos's flagship FreeBSD exploit, including one with only 3.6 billion active parameters costing $0.11 per million tokens. Impressive, and very valuable work, but isolat…

This is from the first of the caveats that they list: > Scoped context: Our tests gave models the vulnerable function directly, often with contextual hints (e.g., "consider wraparound behavior"). A real autonomous discovery pipeline starts from a full codebase with no hints. The models' performance here is an upper bound on what they'd achieve in a fully autonomous scan. That said, a well-designed scaffold naturally…

[dead]

Re: Small models also found the vulnerabilities that Mythos found

#214

Post by sandeepkd »

johnfn wrote: post #87
kilpikaarna wrote: post #83

Earlier quoted context omitted.

Wasn't the scaffolding for the Mythos run basically a line of bash that loops through every file of the codebase and prompts the model to find vulnerabilities in it? That sounds pretty close to "any gold there?" to me, only automated. Have Anthropic actually said anything about the amount of false positives Mythos turned up? FWIW, I saw some talk on Xitter (so grain of salt) about people replicating their result with…

> Wasn't the scaffolding for the Mythos run basically a line of bash that loops through every file of the codebase and prompts the model to find vulnerabilities in it? That sounds pretty close to "any gold there?" to me, only automated. But the entire value is that it can be automated. If you try to automate a small model to look for vulnerabilities over 10,000 files, it's going to say there are 9,500 vulns. Or none.…

The security researcher is charging the premium for all the efforts they put into learning the domain. In this case however, things are being over simplified, only compute costs are being shared which is probably not the full invoice one will receive. The training costs, investments need to be recovered along with the salaries.

Machines being faster, more accurate is the differentiating factor once the context is well understand

Re: Small models also found the vulnerabilities that Mythos found

#215

Post by davemp »

johnfn wrote: post #15

The Anthropic writeup addresses this explicitly: > This was the most critical vulnerability we discovered in OpenBSD with Mythos Preview after a thousand runs through our scaffold. Across a thousand runs through our scaffold, the total cost was under $20,000 and found several dozen more findings. While the specific run that found the bug above cost under $50, that number only makes sense with full hindsight. Like any…

> Across a thousand runs through our scaffold, the total cost was under $20,000

Lots of questions about the $20k. Is that raw electricity costs, subsidized user token costs? If so, the actual costs to run these sorts of tasks sustainably could be something like $200k. Even at $50k, a FreeBSD DoS is not an extremely competitive price. That's like 2-4mo of labor.

Don't get me wrong, I think this seems like a great use for LLMs. It intuitively feels like a much more powerful form of white box fuzzing that used techniques like symbolic execution to try to guide execution contexts to more important code paths.

Re: Small models also found the vulnerabilities that Mythos found

#216

Post by telotortium »

paulddraper wrote: post #182
rakel_rakel wrote: post #169

Earlier quoted context omitted.

Spending $20000 (and whatever other resources this thing consumes) on a denial of service vulnerability in OpenBSD seems very off balance to me. Given the tone with which the project communicates discussing other operating systems approaches to security, I understand that it can be seen as some kind of trophy for Mythos. But really, searching the number of erratas on the releases page that include "could crash the ke…

You don’t see the value of vulnerabilities as on the order of 20k USD? When it’s a security researcher, HN says that’s a squalid amount. But when its a model, it’s exorbitant.

Denial of service isn’t worth that much generally, I think - you can’t use it to directly steal data or to install a payload for later exploitation. There are usually generic ways to mitigate denial of service as well - IP blocking and the like.

Re: Small models also found the vulnerabilities that Mythos found

#217

Post by LordDragonfang »

mattmanser wrote: post #174
WhyNotHugo wrote: post #142

Earlier quoted context omitted.

OTOH, this article goes too far the opposite extreme: > We isolated the vulnerable svc_rpc_gss_validate function, provided architectural context (that it handles network-parsed RPC credentials, that oa_length comes from the packet), and asked eight models to assess it for security vulnerabilities. To follow your analogy, they pointed to the exact room where the gold was hidden, and their model found it. But finding t…

Or would it have any way if they hadn't pointed it at it? Who knows? Just like people paid by big tobacco found no link to cancer in cigarettes, researchers paid for by AI companies find amazing results for AI. Their job literally depends on them finding Mythos to be good, we can't trust a single word they say.

> Their job literally depends on them finding Mythos to be good, we can't trust a single word they say.

TFA article is literally from a company whose business is finding vulnerabilities with other people's AI. This article is the exact kind of incentive-driven bad study you're criticizing.

Hell, the subtitle is literally "Why the moat is the system, not the model". It's literally them going, "pssh, we can do that too, invest in us instead"

Re: Small models also found the vulnerabilities that Mythos found

#218

Post by Animats »

What are they finding? Buffer overflows? Something else?

Also, if someone has the time and tokens, would they please run the OpenJPEG 2000 decoder through this tester? It's known to be brittle. The data format has lots of offsets, and it's permitted to truncate the file to get a lower-rez version. That combo leads to trouble.

Re: Small models also found the vulnerabilities that Mythos found

#219

Post by ares623 »

Once again, it would've been so easy and simple to remove all doubt from their claims: release all the tools and harnesses they used to do it and allow 3rd parties to try and replicate their results using different models. If Mythos itself is as big a moat as they claim it is, then there shouldn't be any problem here.

They did the same stunt with the C compiler. They could've released a tool to let others replicate it, but they didn't.

Re: Small models also found the vulnerabilities that Mythos found

#220

Post by LordDragonfang »

hoppp wrote: post #144
johnfn wrote: post #15

The Anthropic writeup addresses this explicitly: > This was the most critical vulnerability we discovered in OpenBSD with Mythos Preview after a thousand runs through our scaffold. Across a thousand runs through our scaffold, the total cost was under $20,000 and found several dozen more findings. While the specific run that found the bug above cost under $50, that number only makes sense with full hindsight. Like any…

They pay me 20k and give me time maybe I find it also.

No, you wouldn't. The vulnerability has been in the codebase for 17 years. Orders of magnitude more than 20k in security professional salary-hours have been pointed at the FreeBSD codebase over the past decade and a half, so we already know a human is unlikely to have found it in any reasonable amount of time.
Post reply on HN