Live data from Hacker News

Project Glasswing: Securing critical software for the AI era

anthropic.com

551–560 of 921 posts

Re: Project Glasswing: Securing critical software for the AI era

#551

I think this is a largely inflated PR stunt. Opus 4.6 was already capable of finding 0days and chaining together vulns to create exploits. See [0] and [1]. [0] https://www.csoonline.com/article/4153288/vim-and-gnu-emacs-... [1] https://xbow.com/blog/top-1-how-xbow-did-it

I’m in the same boat as you. I believe the model is an improvement of course but I’ve been successfully bug finding 0 day hunting and red teaming with models for the last two years and while that’s impressive I have a feeling that this doomsaying/overhype is mostly marketing being that’s being amplified by non-security folks.

Re: Project Glasswing: Securing critical software for the AI era

#553

I’m sure the new model is a step above the old one but I can’t be the only person who’s getting tired of hearing about how every new iteration is going to spell doom/be a paradigm shift/change the entire tech industry etc. I would honestly go so far as to say the overhype is detrimental to actual measured adoption.

> I would honestly go so far as to say the overhype is detrimental to actual measured adoption. I think you are a bit dishonest about how objectively you are measuring. From where I'm sitting, I don't know a lot of developers that still artisanally code like they did a few years ago. The question is no longer if they are using AI for coding but how much they are still coding manually. I myself barely use IDEs at this…

> From where I'm sitting, I don't know a lot of developers that still artisanally code like they did a few years ago.

You don't know a lot of developers then.

Re: Project Glasswing: Securing critical software for the AI era

#554

Earlier quoted context omitted.

Because questions like this force us to hold up a very uncomfortable mirror to ourselves. It’s much easier to just dismiss.

I’m pretty close to the point of saying that human intelligence is not special.

Despite the stupendous amount of evidence to the contrary?

So far no evidence has been detected in space or on earth, for all of history, of anything being intelligent in the way humans are.

One certain outcome of the Fermi Paradox: humans are outstandingly unique, according to all available evidence, which is the only measure that matters.

Re: Project Glasswing: Securing critical software for the AI era

#555
post #220

I buy the rationale for this. There's been a notable uptick over the past couple of weeks of credible security experts unrelated to Anthropic calling the alarm on the recent influx of actually valuable AI-assisted vulnerability reports. From Willy Tarreau, lead developer of HA Proxy: https://lwn.net/Articles/1065620/ > On the kernel security list we've seen a huge bump of reports. We were between 2 and 3 per week may…

Could this potentially be because more researches are becoming accustomed to the tools/adding them in their pipelines?

The reason I ask is because I’ve been using them to snag bounties to great effect for quite a while and while other models have of course improved they’ve been useful for this kind of work before now.

Re: Project Glasswing: Securing critical software for the AI era

#556

OpenAI initially claimed that GPT-2 was too dangerous to release in 2019. How many times will labs repeat the same absurd propaganda?

OpenAI did not make the strong specific claims about GPT2's abilities that Anthropic is making about Claude Mythos.

Re: Project Glasswing: Securing critical software for the AI era

#557
post #525

Earlier quoted context omitted.

It's very good but it's also recycled Ayn Rand, the Fountainhead.

I've not read it. Could you either link to a section or generally describe the reference?

I have, and it’s not.

Re: Project Glasswing: Securing critical software for the AI era

#558

To be clear, we don’t know that this tool is better at finding bugs than fuzzing. We just know that it’s finding bugs that fuzzing missed. It’s possible fuzzing also finds bugs that this AI would miss.

This is obviously just cope (there's a long, strong-form argument for why LLM-agent vulnerability research is plausibly much more potent than fuzzing, but we don't have to reach it because you can dispose of the whole argument by noting that agents can build and drive fuzzers and triage their outputs), but what I'd really like to understand better is why? What's the impetus to come up with these weird rationalization…

I don't have an anti-AI stance. Maybe I should have spelled that out more clearly in my comment above. I'm as excited and terrified by this technology as everyone else. I think we're all in vicious agreement that we need defense-in-depth - including LLMs and fuzzing (and static analysis and so on).

An LLM can guide all of this work, but current models tend to slowly go off the rails if you don't keep a hand on the wheel. I suspect this new model will be the same. I've had Opus4.6 write custom fuzzing tools from scratch, and I've gotten good results from that. But you just know people will prompt this new model by saying "make this software secure". And it'll forget fuzzing exists at all.

Re: Project Glasswing: Securing critical software for the AI era

#559

Earlier quoted context omitted.

There is plenty of overhyping, no one denies that. But the antidote is not to dismiss everything. Ignore the words and look at the data. In this case, I see a pretty strong case that this will significantly change computer security. They provide plenty of evidence that the models can create exploits autonomously, meaning that the cost of finding valuable security breaches will plummet once they're widely available.

Which sounds like a great thing. Less undiscovered security vulnerabilities

The only people panicking are probably those state level actors who were using these for their own benefit.

Re: Project Glasswing: Securing critical software for the AI era

#560

Earlier quoted context omitted.

Good lord, why such a virulent response to something that seems like we should be considering? As someone in cybersecurity for 10+ years my immediate assumption is why not both? I don’t think considering that they could both have their uses is “cope”.

Again: LLM agents already are both . But it's also remarkable and worth digging into the fact that LLM agents haven't needed fuzzers to produce many (any? in Anthropic Red's case?) of the vulnerabilities they're discussing.

Do we know that? I'd love to see some of the ways security researchers are using LLMs. We have no idea if claude was using fuzzing here, or just reading the files and spotting bugs directly in the source code.

A few weeks ago someone talked about their method for finding bugs in linux. They prompted claude with "Find the security bug in this program. Hint: It is probably in file X.". And they did that for every file in the repo.

Post reply on HN