I think this is a largely inflated PR stunt. Opus 4.6 was already capable of finding 0days and chaining together vulns to create exploits. See [0] and [1]. [0] https://www.csoonline.com/article/4153288/vim-and-gnu-emacs-... [1] https://xbow.com/blog/top-1-how-xbow-did-it
Project Glasswing: Securing critical software for the AI era
551–560 of 921 posts
Re: Project Glasswing: Securing critical software for the AI era
#552Re: Project Glasswing: Securing critical software for the AI era
#553I’m sure the new model is a step above the old one but I can’t be the only person who’s getting tired of hearing about how every new iteration is going to spell doom/be a paradigm shift/change the entire tech industry etc. I would honestly go so far as to say the overhype is detrimental to actual measured adoption.
> I would honestly go so far as to say the overhype is detrimental to actual measured adoption. I think you are a bit dishonest about how objectively you are measuring. From where I'm sitting, I don't know a lot of developers that still artisanally code like they did a few years ago. The question is no longer if they are using AI for coding but how much they are still coding manually. I myself barely use IDEs at this…
You don't know a lot of developers then.
Re: Project Glasswing: Securing critical software for the AI era
#554Earlier quoted context omitted.
Because questions like this force us to hold up a very uncomfortable mirror to ourselves. It’s much easier to just dismiss.
I’m pretty close to the point of saying that human intelligence is not special.
So far no evidence has been detected in space or on earth, for all of history, of anything being intelligent in the way humans are.
One certain outcome of the Fermi Paradox: humans are outstandingly unique, according to all available evidence, which is the only measure that matters.
Re: Project Glasswing: Securing critical software for the AI era
#555I buy the rationale for this. There's been a notable uptick over the past couple of weeks of credible security experts unrelated to Anthropic calling the alarm on the recent influx of actually valuable AI-assisted vulnerability reports. From Willy Tarreau, lead developer of HA Proxy: https://lwn.net/Articles/1065620/ > On the kernel security list we've seen a huge bump of reports. We were between 2 and 3 per week may…
The reason I ask is because I’ve been using them to snag bounties to great effect for quite a while and while other models have of course improved they’ve been useful for this kind of work before now.
Re: Project Glasswing: Securing critical software for the AI era
#556OpenAI initially claimed that GPT-2 was too dangerous to release in 2019. How many times will labs repeat the same absurd propaganda?
Re: Project Glasswing: Securing critical software for the AI era
#557Re: Project Glasswing: Securing critical software for the AI era
#558To be clear, we don’t know that this tool is better at finding bugs than fuzzing. We just know that it’s finding bugs that fuzzing missed. It’s possible fuzzing also finds bugs that this AI would miss.
This is obviously just cope (there's a long, strong-form argument for why LLM-agent vulnerability research is plausibly much more potent than fuzzing, but we don't have to reach it because you can dispose of the whole argument by noting that agents can build and drive fuzzers and triage their outputs), but what I'd really like to understand better is why? What's the impetus to come up with these weird rationalization…
An LLM can guide all of this work, but current models tend to slowly go off the rails if you don't keep a hand on the wheel. I suspect this new model will be the same. I've had Opus4.6 write custom fuzzing tools from scratch, and I've gotten good results from that. But you just know people will prompt this new model by saying "make this software secure". And it'll forget fuzzing exists at all.
Re: Project Glasswing: Securing critical software for the AI era
#559Earlier quoted context omitted.
There is plenty of overhyping, no one denies that. But the antidote is not to dismiss everything. Ignore the words and look at the data. In this case, I see a pretty strong case that this will significantly change computer security. They provide plenty of evidence that the models can create exploits autonomously, meaning that the cost of finding valuable security breaches will plummet once they're widely available.
Which sounds like a great thing. Less undiscovered security vulnerabilities
Re: Project Glasswing: Securing critical software for the AI era
#560Earlier quoted context omitted.
Good lord, why such a virulent response to something that seems like we should be considering? As someone in cybersecurity for 10+ years my immediate assumption is why not both? I don’t think considering that they could both have their uses is “cope”.
Again: LLM agents already are both . But it's also remarkable and worth digging into the fact that LLM agents haven't needed fuzzers to produce many (any? in Anthropic Red's case?) of the vulnerabilities they're discussing.
A few weeks ago someone talked about their method for finding bugs in linux. They prompted claude with "Find the security bug in this program. Hint: It is probably in file X.". And they did that for every file in the repo.