Live data from Hacker News

Project Glasswing: Securing critical software for the AI era

anthropic.com

461–470 of 921 posts

Re: Project Glasswing: Securing critical software for the AI era

#461

Mythos Preview has already found thousands of high-severity vulnerabilities, including some in every major operating system and web browser. Scary but also cool

Or more likely, its just an exaggeration or lie.

What evidence makes you say that? Do you have insider info?

Re: Project Glasswing: Securing critical software for the AI era

#462

This is pretty insane. A model so powerful they felt that releasing it would create a netsec tsunami if released publicly. AGI isn't here yet, but we don't need to get there for massive societal effects. How long will they hold off, especially as competitors are getting closer to their releases of equally powerful models?

OpenAI did the same thing with GPT3 trying to scare people into thinking it would end the internet. OpenAI even reached out to someone who reproduced a weaker version of GPT3 and convinced him to change his mind about releasing it publicly due to how much "harm" it would cause. These claims of how much harm the models will cause is always overblown.

Sure, but the GPT3 thing was mostly hype without stuff to back it up. On the other hand - the reported numbers on specific benchmarks here are insane, I don't doubt that it will have a major impact if it actually is that much more powerful than Opus, and I'd doubt they'd outright lie about benchmark results.

Re: Project Glasswing: Securing critical software for the AI era

#463

Earlier quoted context omitted.

Just reading this, the inevitable scaremongering about biological weapons comes up. Since most of us here are devs, we understand that software engineering capabilities can be used for good or bad - mostly good, in practice. I think this should not be different for biology. I would like to reach out and talk to biologists - do you find these models to be useful and capable? Can it save you time the way a highly capab…

It is not scaremongering.

Equating the ability to make weapons as something to be scared about it scaremongering.

Re: Project Glasswing: Securing critical software for the AI era

#464

Earlier quoted context omitted.

Isn't this the premise of Garfield's Ex Machina?

Hmm, it's been a long time since I watched it. I was thinking more about first contact sci-fi mostly, but Ex Machina is certainly quite prescient. It's also Blade Runner I guess. In general I was wondering about what I would have thought seeing Claude today side-by-side with the original ChatGPT, and then going back further to GPT-2 or BERT (which I used to generate stochastic 'poetry' back in 2019). And then… what a…

> Hmm, it's been a long time since I watched it. I was thinking more about first contact sci-fi mostly, but Ex Machina is certainly quite prescient. It's also Blade Runner I guess.

I kind of felt the opposite - rewatching Ex Machina today in a post-ChatGPT world felt very different from watching it when it came out. The parts of the differences between humans and robots that seemed important then don't seem important now.

Re: Project Glasswing: Securing critical software for the AI era

#465
post #454

Earlier quoted context omitted.

Its not, if you dont trust Anthropic, I hope you trust Daniel Steinberg of curl, who has said AI has gotten really good at detecting bugs and vulnerabilities. Here is his LinkedIN post https://www.linkedin.com/posts/danielstenberg_hackerone-acti...

Didn’t they ban issues generated by ai?

No, they stopped paying bounties.

Re: Project Glasswing: Securing critical software for the AI era

#466

> Mythos Preview identified a number of Linux kernel vulnerabilities that allow an adversary to write out-of-bounds (e.g., through a buffer overflow, use-after-free, or double-free vulnerability.) Many of these were remotely-triggerable. However, even after several thousand scans over the repository, because of the Linux kernel’s defense in depth measures Mythos Preview was unable to successfully exploit any of these…

I think the point they were trying to make here was “Claude did better than a fuzzer because it found a bunch of OOB writes and was able to tell us they weren’t RCE,” not “Claude is awesome because it found a bunch of unreachable OOB writes.”

Re: Project Glasswing: Securing critical software for the AI era

#467

> On the global stage, state-sponsored attacks from actors like China, Iran, North Korea, and Russia have threatened to compromise the infrastructure that underpins both civilian life and military readiness. AITA for thinking that PRISM was probably the state sponsored program affecting civilian life the most? And that one state is missing from the list here?

I can think of two I’d add to the list. One was recently publicly denied access to Anthropics models and the other was busy exploding pagers.

Not clear how an LLM is going to prevent a bomb from being put in a custom-built pager, or why Anthropic should object to Israel waging war against a militia whose goal it is to destroy that country.

Re: Project Glasswing: Securing critical software for the AI era

#469
post #142

The system card for Claude Mythos (PDF): https://www-cdn.anthropic.com/53566bf5440a10affd749724787c89... Interesting to see that they will not be releasing Mythos generally. [edit: Mythos Preview generally - fair to say they may release a similar model but not this exact one] I'm still reading the system card but here's a little highlight: > Early indications in the training of Claude Mythos Preview suggested that th…

If it is that dangerous as they make it appear to be, 24h does not seem sufficient time. I cannot accept this as a serious attempt.

24 h before general internal access seems fine. They don’t have general external access.

Re: Project Glasswing: Securing critical software for the AI era

#470

> Mythos Preview identified a number of Linux kernel vulnerabilities that allow an adversary to write out-of-bounds (e.g., through a buffer overflow, use-after-free, or double-free vulnerability.) Many of these were remotely-triggerable. However, even after several thousand scans over the repository, because of the Linux kernel’s defense in depth measures Mythos Preview was unable to successfully exploit any of these…

We've very quickly reached the point where AI models are now too dangerous to publicly release, and HN users are still trying to trivialize the situation.
Post reply on HN