Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Glasswing: Securing critical software for the AI era

anthropic.com

Post reply on HN
Print view View original

511–520 of 921 posts

Re: Project Glasswing: Securing critical software for the AI era

#511

Post by SpicyLemonZest »

tptacek wrote: post #500
LiamPowell wrote: post #485

Earlier quoted context omitted.

I agree they found at least some real vulnerabilities. What I think is nonsense is the claim of finding thousands of real critical vulnerabilities and claims that they've found other Linux vulnerabilities that they simply can't exploit. There are notably no SHA-3 sums for all their out-of-bound write Linux vulnerabilities, which would be the most interesting ones.

Why is that nonsense? Do you think they exhausted all their compute finding just the few big vulnerabilities they've already discussed, and don't have a budget to just keep cranking the machine to generate more? They're not publishing SHAs for things that aren't confirmed vulnerabilities. They're doing exactly the thing you'd want them to do: they claim to have vulnerabilities when they have actual vulnerabilities.

If I understand Anthropic's statements correctly, they've been cranking for a while, and what they have now is the results of Mythos-enabled vulnerability scans on every important piece of software they could find. (I do want to acknowledge how crazy it is that "vulnerability scan all important software repos in the world" is even an operation that can be performed.)

Re: Project Glasswing: Securing critical software for the AI era

#512

Post by solenoid0937 »

rvz wrote: post #499
gck1 wrote: post #426

I chuckle every time says something in line of "the model is so good that we won't release it to general public, ekhm, because safety". Because the exact same thing has been said on every single upcoming model since GPT 3.5. At this point, this must be an inside joke to do this just because.

This how Anthropic is marketing their AI releases and the reality is, they are terrified of local AI models competing against them. Almost everyone on this thread is falling for the same trick they are pulling and not asking why are their benchmarks and research after training new models not independently verified but always internal to the company. So it is just marketing wrapped around creating fear to get local AI…

The disbelief in this thread is wild. Most of yall are cooked if you think this is actually the case.

Re: Project Glasswing: Securing critical software for the AI era

#513

Post by qnleigh »

pilgrim0 wrote: post #400
woeirua wrote: post #76

Earlier quoted context omitted.

You should watch this talk by Nicholas Carlini (security researcher at Anthropic). Everything in the talk was done with Opus 4.6: https://www.youtube.com/watch?v=1sd26pWhfmg

Just a thought: The fact that the found kernel vulnerability went decades without a fix says nothing about the sophistication needed to find it. Just that nobody was looking. So it says nothing about the model’s capability. That LLMs can find vulnerabilities is a given and expected, considering they are trained on code. What worries me is the public buying the idea that it could in any way be a comprehensive security…

Regardless of how impressive you find the vulnerabilities themselves, the fact that the model is able make exploits without human guidance will enable vastly more people to create them. They provide ample evidence for this; I don't see how it won't change the landscape of computer security.

Re: Project Glasswing: Securing critical software for the AI era

#514

Post by ofjcihen »

tptacek wrote: post #510
ofjcihen wrote: post #506

Earlier quoted context omitted.

Good lord, why such a virulent response to something that seems like we should be considering? As someone in cybersecurity for 10+ years my immediate assumption is why not both? I don’t think considering that they could both have their uses is “cope”.

Again: LLM agents already are both . But it's also remarkable and worth digging into the fact that LLM agents haven't needed fuzzers to produce many (any? in Anthropic Red's case?) of the vulnerabilities they're discussing.

Are you saying that LLMs can use fuzzers or are you saying that they work like fuzzers? Because one of those is less…deterministic? Then the other.

Regardless and in the spirit of my original response my answer would be to give the LLM access to a fuzzer (plus other tools etc) but also have fuzzers in the pipeline. Partially because that increases the determinism in the mix and partially because why not? Layering is almost always better than not.

But again more than anything I’m focusing on the accusations of cope. People SHOULD have measured reactions to claims about any product. People SHOULD be asking questions like this. I know that the LLM debate is often “spicy” but man let’s just try to lower the temperature a bit yeah?

Re: Project Glasswing: Securing critical software for the AI era

#515

Post by vasco »

_pdp_ wrote: post #142
redfloatplane wrote: post #13

The system card for Claude Mythos (PDF): https://www-cdn.anthropic.com/53566bf5440a10affd749724787c89... Interesting to see that they will not be releasing Mythos generally. [edit: Mythos Preview generally - fair to say they may release a similar model but not this exact one] I'm still reading the system card but here's a little highlight: > Early indications in the training of Claude Mythos Preview suggested that th…

If it is that dangerous as they make it appear to be, 24h does not seem sufficient time. I cannot accept this as a serious attempt.

Time doesn't mean much, what is important is what they did in this 24h. If all they did was talk about it then it could be 1000 years and it wouldn't matter. What are the safety checks in place?

Do they have a honey pot infrastructure to launch the model in first and then wait to see if it destroys it? What they did in the 24h matters.

Re: Project Glasswing: Securing critical software for the AI era

#516

Post by solenoid0937 »

skejeke wrote: post #309
SpicyLemonZest wrote: post #249

Earlier quoted context omitted.

They were right, it's hit 100% at a number of large tech companies. (They missed their initial prediction of 90% 6 months ago, because the models then available publicly weren't capable enough.)

So why aren’t they laying people off and pumping the extra money towards research efforts associated with Llm’s? Lmao. They should all cut down their labour input right now if what you claim is true.

Have you considered that some companies want to grow instead of laying people off? No one at Anthropic writes code, they manage 20 Claude Code SWEs.

Re: Project Glasswing: Securing critical software for the AI era

#517

Post by cute_boi »

bwfan123 wrote: post #405
trevorm4 wrote: post #288

Earlier quoted context omitted.

Anthropic has done plenty of cheap marketing tricks as of late, see their recent non-functional C compiler that relied on a harness using gcc's entire test suite

Not surprising given that they dont even know why claude-code works as before or doesnt work [1] ie, there is no known theory of operation. Explains why they are afraid of it. [1] https://news.ycombinator.com/item?id=47660925

I think Boris will come and say there is no issue with claude code.

Re: Project Glasswing: Securing critical software for the AI era

#518

Post by tptacek »

SpicyLemonZest wrote: post #511
tptacek wrote: post #500

Earlier quoted context omitted.

Why is that nonsense? Do you think they exhausted all their compute finding just the few big vulnerabilities they've already discussed, and don't have a budget to just keep cranking the machine to generate more? They're not publishing SHAs for things that aren't confirmed vulnerabilities. They're doing exactly the thing you'd want them to do: they claim to have vulnerabilities when they have actual vulnerabilities.

If I understand Anthropic's statements correctly, they've been cranking for a while, and what they have now is the results of Mythos-enabled vulnerability scans on every important piece of software they could find. (I do want to acknowledge how crazy it is that "vulnerability scan all important software repos in the world" is even an operation that can be performed.)

We talked to Nicholas Carlini on SCW and did not at all get the impression that they've hit everything they can possibly hit. They're still proving the concept one target at a time, last I heard.

Re: Project Glasswing: Securing critical software for the AI era

#519

Post by slopinthebag »

rvz wrote: post #499
gck1 wrote: post #426

I chuckle every time says something in line of "the model is so good that we won't release it to general public, ekhm, because safety". Because the exact same thing has been said on every single upcoming model since GPT 3.5. At this point, this must be an inside joke to do this just because.

This how Anthropic is marketing their AI releases and the reality is, they are terrified of local AI models competing against them. Almost everyone on this thread is falling for the same trick they are pulling and not asking why are their benchmarks and research after training new models not independently verified but always internal to the company. So it is just marketing wrapped around creating fear to get local AI…

Yep, this is exactly it. Open source models and especially ones that run locally are catching up and it's literally an existential threat to these companies. Local models are now quite useful (Qwen, Gemma) and open weight models running on cheaper clouds are perfectly sufficient for use by responsible software engineers to use for building software. You can take your pick of Kimi 2.5, GLM 5.1, and the soon to be released Deepseek 4 which might end up above Opus levels as it stands for a fifth of the cost. Anthropic is particularly vulnerable here, since their entire marketshare rests on the developer market. There is a reason why Google for example, is not so concerned with this and is perfectly happy releasing open models which cut into their own marketshare, and to a lesser extend, same with OpenAI. Anthropic has bet the house on software development which is why we see increasing desperation to both lobby for regulation on open/local models and to wall off their coding harness and subscription plans.

Re: Project Glasswing: Securing critical software for the AI era

#520

Post by tptacek »

ofjcihen wrote: post #514
tptacek wrote: post #510

Earlier quoted context omitted.

Again: LLM agents already are both . But it's also remarkable and worth digging into the fact that LLM agents haven't needed fuzzers to produce many (any? in Anthropic Red's case?) of the vulnerabilities they're discussing.

Are you saying that LLMs can use fuzzers or are you saying that they work like fuzzers? Because one of those is less…deterministic? Then the other. Regardless and in the spirit of my original response my answer would be to give the LLM access to a fuzzer (plus other tools etc) but also have fuzzers in the pipeline. Partially because that increases the determinism in the mix and partially because why not? Layering is…

LLMs can use fuzzers and also LLMs can explore the semantic space of a program in ways fuzzers can't.
Post reply on HN