Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Glasswing: An Initial Update

anthropic.com

Post reply on HN
Print view View original

81–90 of 345 posts

Re: Project Glasswing: An Initial Update

#81

Post by ospray »

OsrsNeedsf2P wrote: post #2

The vulnerabilities found continues to impress, and make legacy media, Twitter and Youtube go nuts. But we still have no data to prove this wasn't doable with the same initiative backed by Opus 4.7, and there is no GA for Mythos access.

This report is far more positive with a far lower false positive rate than I was expecting based on reports from the curl team and a few others. I guess I have just been hearing about the ten percent misses. Can anyone not employed by Anthropic who has used it vouch that it is equal to general human testers and do you need xbow to make it that way.

Re: Project Glasswing: An Initial Update

#82

Post by sp527 »

morpheos137 wrote: post #68
jimmar wrote: post #61

People predict that in 50 years, no human will be driving a car, and people will be shocked that we let humans drive cars manually. Coding may be the same. So many vulnerabilities in code written by very competent programmers. Manually building large, complex systems without major bugs or security vulnerabilities seems to be a nearly impossible challenge.

there is little evidence for this prediction.

Oh there's plenty of evidence. Because a lot of these people have been committing to repos in public for over a decade. Wouldn't take much to show the world just how fallible human coders really are.

Re: Project Glasswing: An Initial Update

#83

Post by unethical_ban »

winstonwinston wrote: post #54
mdeeks wrote: post #34

You can get a taste of this today yourself with Codex Security. I turned it on just as an experiment and in less than a week it has now become essential to all of us. I was shocked how accurate it is, how many security issues it found in existing code, how it continually finds them as we commit, and how NO ONE is immune from making these mistakes. I'd say it is about 90% accurate for us. Often even the "Low" findings…

> I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO. So, how is that supposed to work? Claude Code generates security bugs, then Claude Security finds them, then Claude Code generate fix, spend tokens, profit?

How is this supposed to work? Humans generate security bugs, then humans find them, then humans generate the fix, profit?

Yeah. Presumably as AI code generation gets better, the output gets better. As smaller portions of code are stitched together, human/AI systems analyze it holistically to make sure all its integrations are secure and bug free.

In 2026, different models are better at different things. Cheap models can plan and do small/medium code projects well, more expensive models are even better at architecture and exploit discovery.

Re: Project Glasswing: An Initial Update

#84

Post by jstummbillig »

winstonwinston wrote: post #54
mdeeks wrote: post #34

You can get a taste of this today yourself with Codex Security. I turned it on just as an experiment and in less than a week it has now become essential to all of us. I was shocked how accurate it is, how many security issues it found in existing code, how it continually finds them as we commit, and how NO ONE is immune from making these mistakes. I'd say it is about 90% accurate for us. Often even the "Low" findings…

> I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO. So, how is that supposed to work? Claude Code generates security bugs, then Claude Security finds them, then Claude Code generate fix, spend tokens, profit?

Ngl, watching folks getting irritated about normal employer-employee absurdities from the employer perspective through usage of agents and having to pay for tokens has been a little therapeutic for me.

Re: Project Glasswing: An Initial Update

#85

Post by unethical_ban »

InsideOutSanta wrote: post #25

I wonder if it coincidentally becomes safe to release when compute capacity bought from SpaceX will provide enough headroom to let a lot more people run it.

Total speculation: As the software world shakes out the many hidden vulns in their software, big AI will try to limit the access while it gets ironed out. Once the big projects/systems are reasonably patched after being vetted by SOTA models, the models will be released to the public. I don't think there's a scenario where Mythos-level or better models stay closed permanently.

Re: Project Glasswing: An Initial Update

#86

Post by demorro »

If you're not already applying static analysis and linters to your codebase (and I know many of you aren't), ask yourself why you would bother to apply an expensive LLM tool?

Not to say these things won't catch vulnerabilities static tools cannot, I think they can, it's just we already have the capability to automatically catch a large surface area of common vulns, and have chosen not to, often for expense reasons.

If you're a team that does already apply several layers of analysis and linting, and wants to add this on top, all power to you.

Re: Project Glasswing: An Initial Update

#87

Post by sandeepkd »

ygjb wrote: post #67
winstonwinston wrote: post #54

Earlier quoted context omitted.

> I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO. So, how is that supposed to work? Claude Code generates security bugs, then Claude Security finds them, then Claude Code generate fix, spend tokens, profit?

Yeah, with a budget assigned. This is actually just software development and security right? Developers create software, which has bugs. Users (including bad guys, pen testers, QA folks, automated scans etc, etc, etc) find bugs, including security bugs, Developers fix bugs and maybe make more. It's an OODA loop, and continues until the developers decide to stop supporting the software. Whether that fits into the busi…

Somehow this reminded me of the historical efforts of some government bounty collections for mouse tails which were discontinued due to fraud (such as hunters breeding mice to collect the reward). There is a reason why/how devs and QA keep each other in check. Guess in case of LLM writing code, one has to use different models for dev and security checks.

On other hand, in real world, the developers learn from mistakes and avoid them in the future. However there is no feedback loop with enterprises using LLM with the agreement that the LLM would not use the enterprise code for training purposes

Re: Project Glasswing: An Initial Update

#88

Post by brightbeige »

jimmar wrote: post #61

People predict that in 50 years, no human will be driving a car, and people will be shocked that we let humans drive cars manually. Coding may be the same. So many vulnerabilities in code written by very competent programmers. Manually building large, complex systems without major bugs or security vulnerabilities seems to be a nearly impossible challenge.

And to consider AI agents are still mostly entirely limited to generating code in token-heavy programming languages designed to be written, tested and debugged by humans.

Here are two experimental exceptions:

https://github.com/vercel-labs/zerolang

https://github.com/sbhooley/ainativelang

Re: Project Glasswing: An Initial Update

#89

Post by sobellian »

demorro wrote: post #86

If you're not already applying static analysis and linters to your codebase (and I know many of you aren't), ask yourself why you would bother to apply an expensive LLM tool? Not to say these things won't catch vulnerabilities static tools cannot, I think they can, it's just we already have the capability to automatically catch a large surface area of common vulns, and have chosen not to, often for expense reasons. I…

Static analysis often shows many false positives. A more intelligent tool can help not to waste limited engineering time.

Re: Project Glasswing: An Initial Update

#90

Post by vb-8448 »

The report on findings is very interesting: 1451 acknowledged findings out of 23k candidates(~6%, not high but neither low).

But I didn't find the most important information (or maybe I missed it): how much did it cost to find 1451 security bugs?

Post reply on HN