Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Glasswing: An Initial Update

anthropic.com

Post reply on HN
Print view View original

101–110 of 345 posts

Re: Project Glasswing: An Initial Update

#101

Post by ygjb »

sandeepkd wrote: post #87
ygjb wrote: post #67

Earlier quoted context omitted.

Yeah, with a budget assigned. This is actually just software development and security right? Developers create software, which has bugs. Users (including bad guys, pen testers, QA folks, automated scans etc, etc, etc) find bugs, including security bugs, Developers fix bugs and maybe make more. It's an OODA loop, and continues until the developers decide to stop supporting the software. Whether that fits into the busi…

Somehow this reminded me of the historical efforts of some government bounty collections for mouse tails which were discontinued due to fraud (such as hunters breeding mice to collect the reward). There is a reason why/how devs and QA keep each other in check. Guess in case of LLM writing code, one has to use different models for dev and security checks. On other hand, in real world, the developers learn from mistake…

> the developers learn from mistakes and avoid them in the future

No. Humans learn from mistakes and try to avoid them in the future, but there is a whole pile of other stuff in the bag of neurons between our ears that prevent us from avoiding repetition of errors.

I have seen extremely talented engineers write trivial to avoid memory corruption bugs because they were thinking about the problem they were trying to solve, and not the pitfalls they could fall into. I would argue that the vast majority of software defects in released code are written by people that know better, but the bug introduced was orthogonal to the problem they were trying to solve, or was for an edge case that was not considered in the requirements.

Unless you are writing a software component specifically to be resilient against memory corruption, preventing memory corruption issues aren't top of mind when writing code, and that is ok since humans, like the machines we build, have a limit to the amount of context/content/problem space that we can hold and evaluate at once.

Separately, you don't necessarily need to use different models to generate code vs conduct security checks, but you should be using different prompts, steering, specs, skills and agents for the two tasks because of how the model and agents interpret the instructions given.

Re: Project Glasswing: An Initial Update

#102

Post by SkyPuncher »

demorro wrote: post #86

If you're not already applying static analysis and linters to your codebase (and I know many of you aren't), ask yourself why you would bother to apply an expensive LLM tool? Not to say these things won't catch vulnerabilities static tools cannot, I think they can, it's just we already have the capability to automatically catch a large surface area of common vulns, and have chosen not to, often for expense reasons. I…

> If you're not already applying static analysis and linters to your codebase

Because most issues are in business logic that static analyzers aren't going to catch.

Re: Project Glasswing: An Initial Update

#103

Post by piker »

We have been working with the consumer-grade frontier models to develop what we call "lexploits" in legaltech, and they are insanely good at finding bugs across integrated pipelines. They're also surprisingly good at mitigating them!

Security vulnerabilities are one thing, but in legal we offer up a concept of "knowledge security" which goes to protecting the fidelity of the agent's legal context. Software bugs seem much more tractable because they're managed by software engineers, as opposed to the pipeline "vulnerabilities" we're finding. We wrote a little about one vector here where legal documents aren't quite what they seem: https://tritium.legal/blog/noroboto

No doubt there are many such knowledge domains exposed today. These are more concerning because they're understaffed and managed by non-technical people for the most part. No Mythos required.

Re: Project Glasswing: An Initial Update

#104

Post by jimbokun »

jimmy2times wrote: post #58
winstonwinston wrote: post #54

Earlier quoted context omitted.

> I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO. So, how is that supposed to work? Claude Code generates security bugs, then Claude Security finds them, then Claude Code generate fix, spend tokens, profit?

The AIs have already figured out how to succeed in a software job: 1. Ship bugs 2. Fix them 3. You're the hero!

Dilbert beat you to it:

https://english.stackexchange.com/questions/488178/what-does...

Re: Project Glasswing: An Initial Update

#105

Post by gck1 »

why_only_15 wrote: post #45
lukeschlather wrote: post #39

Earlier quoted context omitted.

It seems like Mythos is often (or typically?) costing $20k per vulnerability, so I don't think there will be enough compute capacity in the world any time soon to let a lot more people use it the way Glasswing is using it. That is not to say I think they are exaggerating its capabilities. That $20k is presumably the rough cost of renting the GPUs, and there are not enough GPUs in the world.

what's the origin of your $20k/vuln estimate?

It's the same as the origin of "Codex/Opus subscription usage is heavily subsidized" - the sales departments equipped with AI agents with the prompt: "use anonymous accounts on the internet to make it easy for me to sell it at $price".

Re: Project Glasswing: An Initial Update

#106

Post by Amekedl »

I don't buy it. A lot of stuff this finds is also just simply wrong, benignly reported as true, despite upper/lower layers in the code burying the possibility of a vulnerability actually being exploited. It's a performance/security trade-off too, it always has been. Additional checks and other measures do in fact need to be performed for security purposes.

Great marketing as always, but the rose-tinted view many have seems vicariously misplaced.

Re: Project Glasswing: An Initial Update

#107

Post by cpard »

My understanding so far is that that Mythos (and any model in general) can produce candidate reasoning but you really need a system around that reasoning that is capable of producing auditable security findings.

So, success is coming not just from the model but also from the harnesses they built around it. The Cloudflare post was more detailed on that front and I wish the rest would share more about it.

The Cisco spec is interesting too, it pretty much describes an architecture of a harness: https://github.com/CiscoDevNet/foundry-security-spec

Re: Project Glasswing: An Initial Update

#109

Post by NitpickLawyer »

gck1 wrote: post #100
nikcub wrote: post #50

There has been a lot of cynicism around mythos, that it's just the usual public models without guardrails, etc. etc. but this: > 1,752 of those high- or critical-rated vulnerabilities have now been carefully assessed by one of six independent security research firms, or in a small number of cases by ourselves. Of these, 90.6% (1,587) have proved to be valid true positives, and 62.4% (1,094) were confirmed as either h…

> This is why I believe mythos will remain private for the foreseeable future. There's such a large surface that needs to be secured and so much to triage, fix, deploy. sigh I remember the GPT-2 days - when it was the first time OpenAI restricted access to the models citing "humanity is not ready for it". The model was good at writing poetry or something. Since then, I don't remember a single model announcement from…

Writing marketing 10 times doesn't invalidate the (many) claims from many respectable sources that the model is a step change in cybersec. There's also the report [1] from the Brits that track cyber capabilities since '22 or '23 and they've also confirmed it's a step change (together with 5.5 cyber or whatever they call it).

Marketing is like propaganda. It doesn't need to be based on false facts. Of course they're gonna milk it, keep it private and so on. But that doesn't mean the model is bad. Or that others are as good (apparently they're not there yet).

[1] - https://www.aisi.gov.uk/blog/our-evaluation-of-openais-gpt-5...

Post reply on HN