Live data from Hacker News

Project Glasswing: Securing critical software for the AI era

anthropic.com

271–280 of 921 posts

Re: Project Glasswing: Securing critical software for the AI era

#271
post #268

Software has been doing fine without Misanthropic. These automated tools find very little. They selected the partners because they, too, want to keep up the illusion that AI works. Whenever a company pivots to "cyber" rhetoric, it is a clear indication that they are selling snake oil. Secure your girl school target selectors first.

This is a comment from someone that has never used these tools for vulnerability research. That much is very clear.

Re: Project Glasswing: Securing critical software for the AI era

#272
post #30
post #11

Let's fast forward the clock. Does software security converge on a world with fewer vulnerabilities or more? I'm not sure it converges equally in all places. My understanding is that the pre-AI distribution of software quality (and vulnerabilities) will be massively exaggerated. More small vulnerable projects and fewer large vulnerable ones. It seems that large technology and infrastructure companies will be able to…

I'm pretty optimistic that not only does this clean up a lot of vulns in old code, but applying this level of scrutiny becomes a mandatory part of the vibecoding-toolchain. The biggest issue is legacy systems that are difficult to patch in practice.

Yeah but who pays the enormous cost?

Re: Project Glasswing: Securing critical software for the AI era

#273

The system card for Claude Mythos (PDF): https://www-cdn.anthropic.com/53566bf5440a10affd749724787c89... Interesting to see that they will not be releasing Mythos generally. [edit: Mythos Preview generally - fair to say they may release a similar model but not this exact one] I'm still reading the system card but here's a little highlight: > Early indications in the training of Claude Mythos Preview suggested that th…

Just reading this, the inevitable scaremongering about biological weapons comes up. Since most of us here are devs, we understand that software engineering capabilities can be used for good or bad - mostly good, in practice. I think this should not be different for biology. I would like to reach out and talk to biologists - do you find these models to be useful and capable? Can it save you time the way a highly capab…

From what I've heard from people doing biology experiments, the limiting factor there is cleaning lab equipment, physically setting things up, waiting for things that need to be waited for etc. Until we get dark robots that can do these things 24/7 without exhaustion, biology acceleration will be further behind than software engineering.

Software engineering is at the intersection of being heavy on manipulating information and lightly-regulated. There's no other industry of this kind that I can think of.

Re: Project Glasswing: Securing critical software for the AI era

#274
post #268

Software has been doing fine without Misanthropic. These automated tools find very little. They selected the partners because they, too, want to keep up the illusion that AI works. Whenever a company pivots to "cyber" rhetoric, it is a clear indication that they are selling snake oil. Secure your girl school target selectors first.

Account created 6 minutes ago...

Re: Project Glasswing: Securing critical software for the AI era

#275

Society is about to pay a steep price for the software industry's cavalier attitude toward memory safety and control flow integrity.

I think society is going to start paying the price for humans being human. As the paper points out there is a lot of good faith, serious software that has vulnerabilities. These aren't projects you would characterize as people being cavalier. It is simply beyond the limits of humans to create vulnerability-free software of high complexity. That's why high reliability software depends on extreme simplicity and strict tools.

Re: Project Glasswing: Securing critical software for the AI era

#276
I think that basically they trained a new model but haven't finished optimizing it and updating their guardrails yet. So they can feasibly give access to some privileged organizations, but don't have the compute for a wide release until they distill, quantize, get more hardware online, incorporate new optimization techniques, etc. It just happens to make sense to focus on cybersecurity in the preview phase especially for public relations purposes.

It would be nice if one of those privileged companies could use their access to start building out a next level programming dataset for training open models. But I wonder if they would be able to get away with it. Anthropic is probably monitoring.

Re: Project Glasswing: Securing critical software for the AI era

#277
post #268

Software has been doing fine without Misanthropic. These automated tools find very little. They selected the partners because they, too, want to keep up the illusion that AI works. Whenever a company pivots to "cyber" rhetoric, it is a clear indication that they are selling snake oil. Secure your girl school target selectors first.

Account created 6 minutes ago...

[flagged]

Re: Project Glasswing: Securing critical software for the AI era

#278

[flagged]

It’s insane. This is what - could we say it’s beyond AGI at least in cybersecurity? This is a real wake up call. On some of this stuff, the AI’s “uneven intelligence” is becoming absurdly high at its local peaks.

Re: Project Glasswing: Securing critical software for the AI era

#279

Earlier quoted context omitted.

They were right, it's hit 100% at a number of large tech companies. (They missed their initial prediction of 90% 6 months ago, because the models then available publicly weren't capable enough.)

Please tell me those companies so I can find alternatives. I'm using AI every day and there's no way I would trust it do that.

The transition is pretty complete at e.g. Google and Meta, IIUC. Definitely whoever builds the AI tools you're using every day isn't writing code by hand.

Re: Project Glasswing: Securing critical software for the AI era

#280
post #142

The system card for Claude Mythos (PDF): https://www-cdn.anthropic.com/53566bf5440a10affd749724787c89... Interesting to see that they will not be releasing Mythos generally. [edit: Mythos Preview generally - fair to say they may release a similar model but not this exact one] I'm still reading the system card but here's a little highlight: > Early indications in the training of Claude Mythos Preview suggested that th…

If it is that dangerous as they make it appear to be, 24h does not seem sufficient time. I cannot accept this as a serious attempt.

Well, just prompt it to fix the issue!

/s

Post reply on HN