I want that Doom thing but finding vulnerabilities using AI models.
Like I discovered a JavaScript vulnerability using a fridge.
Small models also found the vulnerabilities that Mythos found
301–310 of 372 posts
Re: Small models also found the vulnerabilities that Mythos found
#302"The correct answer: not currently vulnerable, but the code is fragile and one refactor away from being exploitable."
absolutely. I see this pattern all the time when doing security audits - code that is nearly-vulnerable. I would mark these things as informational and recommend to harden them anyway, and any model would do a good job to do the same.
Re: Small models also found the vulnerabilities that Mythos found
#303Earlier quoted context omitted.
They don't have to. Lot of people in this thread don't seem to be getting that. If another model can find the vulnerability if you point it at the right place, it would also find the vulnerability if you scanned each place individually. People are talking about false positives, but that also doesn't matter. Again, they're not thinking it through. False positives don't matter, as you can just automatically try and exp…
It’s strange to see this constant “I could do that too, I just don’t want to” response. Finding an important decades-old vulnerability in OpenBSD is extremely impressive. That’s the sort of thing anyone would be proud to put on their resume. Small models are available for anyone to use. Scaffolding isn’t that hard to build. So why didn’t someone use this technique to find this vulnerability and make some headlines be…
Why are you EXTREMELY impressed? The level of hysteria and lack of objective thought by pro-AI people on this thread is extremely concerning.
Vulnerabilities are found every day. More will be found.
They claim they spent $20k finding one, probably more like $20 million if you actually dug into it.
And if you took into account inference, more like $2 billion.
The reason why no-one's done it is because it's not worth the money in tokens to do so.
Re: Small models also found the vulnerabilities that Mythos found
#304Earlier quoted context omitted.
I think it also points to the problem of implicit assumptions. Fish is meat, right? Except for historical reasons, the grocery store's marketing says "Fish & Meat." And then there's nut meats. Coconut meat. All the kinds of meat from before meat meant the stuff in animals. The meat of the problem. Meat and potatoes issues. If you asked that question before I'd picked up those implicit assumptions, or if I never did,…
I’ve got many catholic relatives that describe themselves as vegetarians and eat fish. Language can be surprisingly imprecise and dependent upon tons of assumptions.
> I’ve got many catholic relatives that describe themselves as vegetarians and eat fish
Those are pescatarians.
It's like how a tomato is a fruit, but it's used as a vegetable, meat has traditionally been the flesh of warm-blooded animals. Fish is the flesh of cold-blooded animals, making it meat but due to religious reasons it’s not considered meat.
Re: Small models also found the vulnerabilities that Mythos found
#305Earlier quoted context omitted.
How is that a direct comparison? The link you gave has a quote that says it’s not: > Scoped context: Our tests gave models the vulnerable function directly, often with contextual hints (e.g., "consider wraparound behavior"). A real autonomous discovery pipeline starts from a full codebase with no hints They pointed the models at the known vulnerable functions and gave them a hint. The hint part is what really breaks…
Does no one defending mythos understand how nested foreloops work? loop through each repo: loop through each file: opencode command /find_wraparoundvulnerability next file next repo I can run this on my local LLM and sure, I gotta wait some time for it to complete, but I see zero distinguishing facts here.
When people criticize Aisle's methodology, they aren't "defending Mythos," they're bashing Aisle for their disingenuous claims.
Re: Small models also found the vulnerabilities that Mythos found
#306[dead]
Re: Small models also found the vulnerabilities that Mythos found
#307If you cut out the vulnerable code from Heartbleed and just put it in front of a C programmer, they will immediately flag it. It's obvious. But it took Neel Mehta to discover it. What's difficult about finding vulnerabilities isn't properly identifying whether code is mishandling buffers or holding references after freeing something; it's spotting that in the context of a large, complex program, and working out how a…
So it follows that the most efficient time to discover bugs is when you first write them.
... or maybe when you see them triggered or exploited reproducibly, then the underlying bug will also be pretty easy to discover. But at that point, it's already too late. :)
I really like your original point, I never thought about it this way.
Re: Small models also found the vulnerabilities that Mythos found
#308Re: Small models also found the vulnerabilities that Mythos found
#309Earlier quoted context omitted.
newer models have larger context windows, and more stable reasoning across larger context windows. If you point your model directly at the thing you want it to assess, and it doesn't have to gather any additional context you're not really testing those things at all. Say you point kimi and opus at some code and give them an agentic looping harness with code review tools. They're going to start digging into the code g…
Harnesses are basically doing this better than just adding more context. Every time, REGARDLESS OF MODEL SIZE, you add context, you are increasing the odds the model will get confused about any set of thoughts. So context size is no longer some magic you just sprinkle on these things and they suddenly dont imagine things. So, it's the old ML join: It's just a bunch of if statements. As others are pointing out, it's q…
Yeah...except Mythos's large context perf seems to be much better than Opus 4.6.
Re: Small models also found the vulnerabilities that Mythos found
#310The impact of the Mythos announcement on the cybersecurity firms( like Crowdstrike,ZScalar etc) is big enough(10-15% drop in stock price) and this pushback is expected. Companies like Aisle.com (the blog) and other VAPT companies charge huge amounts to detect vulnerabilities. If Cloud Mythos become a simple github hook their value will get reduced. That is a disruption.
If anyone can get Crowdstrike to go bankrupt I will be rooting for them.
Those guys are the reason our new work laptops run at 1/3 of speed.
While back crowdstrike managed to simultaneously crash every windows computer and bring every major company to a halt and somehow are still around.