I worry that cybersecurity as target is all fine and good, but it’s looking for your keys under the streetlight. We are all familiar with computers. The problem is likely to be humans, especially in automated programmatic manipulation. The risk is that the next level of AI is going to make Fox News and other mass manipulation efforts look like kindergarten.
Project Glasswing: An Initial Update
161–170 of 345 posts
Re: Project Glasswing: An Initial Update
#162Earlier quoted context omitted.
In the article they describe how all the vulns are actually exploitable end to end and >1000 have been independently verified as critical. These aren't unreachable vulns.
Where is the link to the advisories then? :/
> The software industry’s longstanding convention is to disclose new vulnerabilities 90 days after they’re discovered (or, if a patch is created before the 90 days is up, around 45 days after the patch becomes available). This allows time for end users to update their software before a vulnerability can be exploited by attackers. Our own Coordinated Vulnerability Disclosure policy takes this approach.
> However, this means that disclosed vulnerabilities are a lagging indicator of the accelerating frontier of AI models’ cyber capabilities: we’re not yet at the point where we can fully detail our partners’ findings with Mythos Preview without putting end users at risk. Instead, we provide illustrative examples of the model’s performance, along with aggregate statistics on our progress to date. Once patches for the vulnerabilities that Mythos Preview has discovered are widely deployed, we’ll provide much more detail about what we’ve learned.
Re: Project Glasswing: An Initial Update
#163Earlier quoted context omitted.
Where is the link to the advisories then? :/
As the article explains, they mostly haven't been disclosed, because they're not fixed. They're giving people 90 days, or 45 after a patch is made.
> haven't been disclosed, because they're not fixed.
That's convinient.
But wait, don't they have this amazing AI that can fix all the issues itself with a single /goal command? What's the holdup?
Re: Project Glasswing: An Initial Update
#164Earlier quoted context omitted.
I think you just aren't reading the post, or any of the Glasswing partner's posts. You have this view in your head of what Mythos is, and nobody can say anything dissuade you from it.
"Partners" is the important word in your comment. I am reading all of it, but I have a huge barrel of salt to consume along with everything that I read, because I see conflicts of interest everywhere I go, with fancy words and no means to verify. If I was given free access to any frontier model to use on my projects, equivalent of millions of dollars in AI credits, I sure hope people didn't trust anything that came o…
I don't think Firefox or The Linux Foundation have conflicts of interest here. They've said in their contracts that they get the tokens irrespective of what they say about Mythos. Additionally, the findings speak for themselves.
This just seems overly conspiratorial to me. I don't remember Anthropic ever lying in their blog posts. They've been about as consistent as Apple when it comes to product claims.
Re: Project Glasswing: An Initial Update
#165Earlier quoted context omitted.
> I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO. So, how is that supposed to work? Claude Code generates security bugs, then Claude Security finds them, then Claude Code generate fix, spend tokens, profit?
Yeah, with a budget assigned. This is actually just software development and security right? Developers create software, which has bugs. Users (including bad guys, pen testers, QA folks, automated scans etc, etc, etc) find bugs, including security bugs, Developers fix bugs and maybe make more. It's an OODA loop, and continues until the developers decide to stop supporting the software. Whether that fits into the busi…
It's pretty absurd to do it on AI-generated code though. If there is now an automated way to find vulnerabilities, coding models can be pretty easily trained to not introduce them
Re: Project Glasswing: An Initial Update
#166Earlier quoted context omitted.
You’re right, it’s a valid data point. But the U.K. government report is also a data point, and the Firefox report is a data point, and they suggest that it is, indeed, significantly better than current generation models. Maybe curl is significantly better hardened than most projects? In any event, it barely matters. As Anthropic acknowledges, next level models are comings, theirs is only one of them. Current generat…
> Maybe curl is significantly better hardened than most projects? Meanwhile from [1]: "Not even half-way through this #curl release cycle we are already at 11 confirmed vulnerabilities - and there are three left in the queue to assess and new reports keep arriving at a pace of more than one/day." "The simple reason is: the (AI powered) tools are this good now. And people use these tools against curl source code.They…
I don't think anyone has claimed that Mythos finds all vulns in all projects. But it's very good if Mozilla's blog posts are anything to go by.
Re: Project Glasswing: An Initial Update
#167You can get a taste of this today yourself with Codex Security. I turned it on just as an experiment and in less than a week it has now become essential to all of us. I was shocked how accurate it is, how many security issues it found in existing code, how it continually finds them as we commit, and how NO ONE is immune from making these mistakes. I'd say it is about 90% accurate for us. Often even the "Low" findings…
> I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO. So, how is that supposed to work? Claude Code generates security bugs, then Claude Security finds them, then Claude Code generate fix, spend tokens, profit?
Yes. Up until this point the bottleneck was how many developers you could convince to help you. Now it's how much money you can dump into it. Like everything else, software is becoming a game where the winner is the organization most willing to spend money. It'll be like bombs or tanks - you need smart people to advance in the war, but you also need money and material, the material is just compute infra.
Re: Project Glasswing: An Initial Update
#168Earlier quoted context omitted.
Yeah, with a budget assigned. This is actually just software development and security right? Developers create software, which has bugs. Users (including bad guys, pen testers, QA folks, automated scans etc, etc, etc) find bugs, including security bugs, Developers fix bugs and maybe make more. It's an OODA loop, and continues until the developers decide to stop supporting the software. Whether that fits into the busi…
It's pretty absurd to do it on AI-generated code though. If there is now an automated way to find vulnerabilities, coding models can be pretty easily trained to not introduce them
Tell me you don’t know how AI works without telling me you don’t know how AI works.
Re: Project Glasswing: An Initial Update
#169You can get a taste of this today yourself with Codex Security. I turned it on just as an experiment and in less than a week it has now become essential to all of us. I was shocked how accurate it is, how many security issues it found in existing code, how it continually finds them as we commit, and how NO ONE is immune from making these mistakes. I'd say it is about 90% accurate for us. Often even the "Low" findings…
Did you need to do anything special to get access to Codex Security?
Re: Project Glasswing: An Initial Update
#170Earlier quoted context omitted.
Somehow this reminded me of the historical efforts of some government bounty collections for mouse tails which were discontinued due to fraud (such as hunters breeding mice to collect the reward). There is a reason why/how devs and QA keep each other in check. Guess in case of LLM writing code, one has to use different models for dev and security checks. On other hand, in real world, the developers learn from mistake…
Are you thinking of the cobra effect (aka https://en.wikipedia.org/wiki/Perverse_incentive ) where people in India started breeding cobras to get the reward?
Plenty of examples abound:
https://en.wikipedia.org/wiki/Great_Hanoi_Rat_Massacre
> Today, the events are often used as an example of a perverse incentive, commonly referred to as the cobra effect. The modern discoverer of this event, American historian Michael G. Vann argues that the cobra example from the British Raj cannot be proven, but that the rats in the Vietnam case can be proven, so the term should be changed to the Rat Effect.