Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small models also found the vulnerabilities that Mythos found

aisle.com

Post reply on HN
Print view View original

281–290 of 372 posts

Re: Small models also found the vulnerabilities that Mythos found

#281

Post by ehnto »

tptacek wrote: post #20

If you cut out the vulnerable code from Heartbleed and just put it in front of a C programmer, they will immediately flag it. It's obvious. But it took Neel Mehta to discover it. What's difficult about finding vulnerabilities isn't properly identifying whether code is mishandling buffers or holding references after freeing something; it's spotting that in the context of a large, complex program, and working out how a…

So it follows that the most efficient time to discover bugs is when you first write them.

Re: Small models also found the vulnerabilities that Mythos found

#282

Post by ipaddr »

eiens wrote: post #262
johnfn wrote: post #248

Earlier quoted context omitted.

Your proof-in-pudding test seems to assume that AI is binary -- either it accelerates everyone's development 100x ("let's rewrite every app into bug-free native applications") or nothing ("there hasn't been anything to show for that in years"). I posit reality is somewhere in between the two.

LLM’s are capable of searching information spaces and generating some outputs that one can use to do their job. But it’s not taking anyone’s job, ever. People are not bots, a lot of the work they do is tacit and goes well beyond the capabilities and abilities of llm’s. Many tech firms are essentially mature and are currently using too much labour. This will lead to a natural cycle of lay offs if they cannot figure ou…

"it’s not taking anyone’s job, ever"

It has already and that doesn't mean new jobs haven't been created or that those new jobs went to those who lost their jobs.

Re: Small models also found the vulnerabilities that Mythos found

#283

Post by JSR_FDED »

muyuu wrote: post #255

I think the "Mythos" name is genius. The people at Anthropic make a bunch of claims and the public is expected to just believe them without any possibility of testing those claims or reproducing those results, and since so many people are invested in this saviour for the Global economy, or in the industry in general, or in hype to feed their engagement-based income sources, then there is faith to spare. Meanwhile thi…

One very clever consequence of Anthropic's guarded release of the Mythos model is that they've kind of claimed the position of best in class here, and also positioned themselves as the responsible vendor in this space in one fell swoop.

Re: Small models also found the vulnerabilities that Mythos found

#285

Post by solatic »

Most commenters here: "Mythos is powerful because you can point it at a whole codebase, if you point the smaller models at a whole codebase and iterate through small sections of code, you'll get too many false-positives to handle."

This misses the point entirely. You pay $20k as a one-time fee to establish a baseline. Your codebase develops one PR at a time, which... updates isolated sections of code. Which means you don't need Mythos for a PR, just small, open-weight models. Maybe you run Mythos once a year to ensure that you keep your baseline updated and reduce the risk that the open-weights models missed anything.

Seeing this as anything but a huge win for open-weights models and a huge loss for Anthropic misses the point entirely. Mythos isn't something you can persuade Fortune 500 companies to spend $20k/day or even $20k/week to spend on, like they were hoping for. $20k/year is a lot less valuable, and it won't justify development costs or Anthropic's growth multiple.

Re: Small models also found the vulnerabilities that Mythos found

#286

Post by bottlepalm »

None of these comments will age well. I don't know if it is denial, or cope, or being threatened by AI or what, but no one is taking AI serious enough. Simply take what is being presented at face value, stop thinking everything is a conspiracy and realize the implications. Zero days in software are one thing, it's a hop skip and jump from there to zero days in biology - and no one will be laughing about that.

Re: Small models also found the vulnerabilities that Mythos found

#288

Post by dbdr »

eiens wrote: post #266
MidnightRider39 wrote: post #236

Earlier quoted context omitted.

Leadership is also a very human thing. I think most people would balk at the idea of being led by an LLM. One of the main functions of leaders (should be) is to assume responsibility for decisions and outcomes. A computer cant do that. And finally why should someone in power choose to replace themselves?

Someone in power doesn’t get to choose - the board of directors do. Who’s job is to act in the best interest of shareholders. Firms tend to follow peers in an industry - once one blinks the rest follow.

> Someone in power doesn’t get to choose - the board of directors do

Since the board of directors can decide to replace the CEO, it's not the CEO who holds the (ultimate) power, it's the board of directors.

Re: Small models also found the vulnerabilities that Mythos found

#289

Post by muyuu »

JSR_FDED wrote: post #283
muyuu wrote: post #255

I think the "Mythos" name is genius. The people at Anthropic make a bunch of claims and the public is expected to just believe them without any possibility of testing those claims or reproducing those results, and since so many people are invested in this saviour for the Global economy, or in the industry in general, or in hype to feed their engagement-based income sources, then there is faith to spare. Meanwhile thi…

One very clever consequence of Anthropic's guarded release of the Mythos model is that they've kind of claimed the position of best in class here, and also positioned themselves as the responsible vendor in this space in one fell swoop.

OpenAI pulled the same trick with GPT3. It's amazing how well it's working judging by the comments I'm hearing from people I know exist. Because out there on social media, who knows.

Re: Small models also found the vulnerabilities that Mythos found

#290

Post by cycomanic »

woeirua wrote: post #194
apgwoz wrote: post #171

Earlier quoted context omitted.

The benefit here is reducing the time to find vulnerabilities; faster than humans, right? So if you can rig a harness for each function in the system, by first finding where it’s used, its expected input, etc, and doing that for all functions, does it discover vulnerabilities faster than humans? Doesn’t matter that they isolated one thing. It matters that the context they provided was discoverable by the model.

There is absolutely zero reason to believe you could use this same approach to find and exploit vulns without Mythos finding them first. We already know that older LLMs can’t do what Mythos has done. Anthropic and others have been trying for years.

From the article:

>At AISLE, we've been running a discovery and remediation system against live targets since mid-2025: 15 CVEs in OpenSSL (including 12 out of 12 in a single security release, with bugs dating back 25+ years and a CVSS 9.8 Critical), 5 CVEs in curl, over 180 externally validated CVEs across 30+ projects spanning deep infrastructure, cryptography, middleware, and the application layer.

So there is pretty good evidence that yes you can use this approach. In fact I would wager that running a more systematic approach will yield better results than just bruteforcing, by running the biggest model across everything. It definitely will be cheaper.

Post reply on HN