Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Glasswing: An Initial Update

anthropic.com

Post reply on HN
Print view View original

201–210 of 345 posts

Re: Project Glasswing: An Initial Update

#201

Post by mncharity »

ygjb wrote: post #101
sandeepkd wrote: post #87

Earlier quoted context omitted.

Somehow this reminded me of the historical efforts of some government bounty collections for mouse tails which were discontinued due to fraud (such as hunters breeding mice to collect the reward). There is a reason why/how devs and QA keep each other in check. Guess in case of LLM writing code, one has to use different models for dev and security checks. On other hand, in real world, the developers learn from mistake…

> the developers learn from mistakes and avoid them in the future No. Humans learn from mistakes and try to avoid them in the future, but there is a whole pile of other stuff in the bag of neurons between our ears that prevent us from avoiding repetition of errors. I have seen extremely talented engineers write trivial to avoid memory corruption bugs because they were thinking about the problem they were trying to so…

> write trivial to avoid memory corruption bugs because they were thinking about [something else] [...] defects [...] written by people that know better, but the bug introduced was orthogonal to [their focus]

For whatever reason, hadn't associated the inattentional blindness of bug writing with the invisible gorilla experiment and car crashes - selective attention fails. People looking right at the gorilla strolling into production while chest thumping, but not seeing it, for a focus on passing basketballs. That's quite an image. Tnx.

Re: Project Glasswing: An Initial Update

#202

Post by ofjcihen »

lateral_cloud wrote: post #169
mdeeks wrote: post #34

You can get a taste of this today yourself with Codex Security. I turned it on just as an experiment and in less than a week it has now become essential to all of us. I was shocked how accurate it is, how many security issues it found in existing code, how it continually finds them as we commit, and how NO ONE is immune from making these mistakes. I'd say it is about 90% accurate for us. Often even the "Low" findings…

Did you need to do anything special to get access to Codex Security?

Not sure what the threshold is but I sent them all of my bug bounty profiles and papers I’ve authored.

I don’t think you need all of that though. I know a whole mess of people that have gotten it for much less. Should just give it a try.

Re: Project Glasswing: An Initial Update

#203

Post by sfink »

scotty79 wrote: post #173
morpheos137 wrote: post #68

Earlier quoted context omitted.

there is little evidence for this prediction.

What evidence would you expect to see if that was the case?

Some numbers, however shaky, that AI-written code is secure.

It could become that way, but thus far no evidence has been presented for it. The best we have right now is that you can spend $20 in tokens to write a patch and then $20K to find a vulnerability in it. First, that's not measuring the same thing. Second, it's not very impressive.

50 years is a long, long time, so I wouldn't bet against it. But I agree that we don't have evidence for it yet.

Re: Project Glasswing: An Initial Update

#204

Post by Forgeties79 »

ygjb wrote: post #101
sandeepkd wrote: post #87

Earlier quoted context omitted.

Somehow this reminded me of the historical efforts of some government bounty collections for mouse tails which were discontinued due to fraud (such as hunters breeding mice to collect the reward). There is a reason why/how devs and QA keep each other in check. Guess in case of LLM writing code, one has to use different models for dev and security checks. On other hand, in real world, the developers learn from mistake…

> the developers learn from mistakes and avoid them in the future No. Humans learn from mistakes and try to avoid them in the future, but there is a whole pile of other stuff in the bag of neurons between our ears that prevent us from avoiding repetition of errors. I have seen extremely talented engineers write trivial to avoid memory corruption bugs because they were thinking about the problem they were trying to so…

The problem is you as a person are not incentivized to introduce bugs in your code. If I am a company that provide provides an LLM/agent, and I know that the more bugs you have the more money I’m going to make, then I am not exactly incentivized to make my LLM/Agent better at preventing bugs. I don’t even have to explicitly make it introduce them. The incentive structure is simply out of whack.

Re: Project Glasswing: An Initial Update

#205

Post by mindcrime »

pjmlp wrote: post #69
genghisjahn wrote: post #63

Earlier quoted context omitted.

I thought we were all doing that already?

The idea is to take the human out of the loop.

    > But in 30 days we could put in electronic relays. Get the men out of the loop.

    > Gentlemen...
    > I wouldn't trust this overgrown pile of microchips further than I could throw it. I don't know if 
    > you wanna trust the safety of our country to some... silicon diode...

Re: Project Glasswing: An Initial Update

#206

Post by tptacek »

mukmuk wrote: post #122

I’m not sure how to reconcile anthropic’s update / some of the exuberant comments here with recent feedback like the following from curl maintainer Daniel Steinberg: “I see no evidence that this setup [Mythos] finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to…

It's a weird accident of fate that curl has somehow become the reference target for LLM bugfinding. Curl is not an especially interesting project. What seems to have happened is that Stenberg made waves (legitimately) complaining about LLM slop submissions, then more waves when LLM bug reports got good, and so now everyone seems to think a good measure of a vuln researcher is how many curl findings they generate. No. Curl is a straightforward CLI HTTP client.

The Linux kernel is the right reference target, if you need one.

Re: Project Glasswing: An Initial Update

#207

Post by BoorishBears »

blueboo wrote: post #183
nikcub wrote: post #50

There has been a lot of cynicism around mythos, that it's just the usual public models without guardrails, etc. etc. but this: > 1,752 of those high- or critical-rated vulnerabilities have now been carefully assessed by one of six independent security research firms, or in a small number of cases by ourselves. Of these, 90.6% (1,587) have proved to be valid true positives, and 62.4% (1,094) were confirmed as either h…

> That may suit Anthropic as private models can't be distilled They can be distilled internally… expect great things from Sonnet 4.8

We already have "Opus" 4.7 from the same base

Re: Project Glasswing: An Initial Update

#208

Post by BoorishBears »

demorro wrote: post #86

If you're not already applying static analysis and linters to your codebase (and I know many of you aren't), ask yourself why you would bother to apply an expensive LLM tool? Not to say these things won't catch vulnerabilities static tools cannot, I think they can, it's just we already have the capability to automatically catch a large surface area of common vulns, and have chosen not to, often for expense reasons. I…

I quite like that the most honest answer for the majority of devs was downvoted then flagged to death.

Most people doing this now didn't use static analysis tools because they were seen as an unnecessary extra.

Re: Project Glasswing: An Initial Update

#210

Post by mordymoop »

winstonwinston wrote: post #54
mdeeks wrote: post #34

You can get a taste of this today yourself with Codex Security. I turned it on just as an experiment and in less than a week it has now become essential to all of us. I was shocked how accurate it is, how many security issues it found in existing code, how it continually finds them as we commit, and how NO ONE is immune from making these mistakes. I'd say it is about 90% accurate for us. Often even the "Low" findings…

> I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO. So, how is that supposed to work? Claude Code generates security bugs, then Claude Security finds them, then Claude Code generate fix, spend tokens, profit?

This also describes the work of software engineers.
Post reply on HN