Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Glasswing: Securing critical software for the AI era

anthropic.com

Post reply on HN
Print view View original

201–210 of 921 posts

Re: Project Glasswing: Securing critical software for the AI era

#201

Post by baq »

pizlonator wrote: post #186
baq wrote: post #181

Earlier quoted context omitted.

> It's messed up that Anthropic simultaneously claims to be a public benefit copro and is also picking who gets to benefit from their newly enhanced cybersecurity capabilities. It means that the economic benefit is going to the existing industry heavyweights. It's messed up that the US Government simultaneously claims to be a public benefit and is also picking who gets to benefit from their newly enhanced nuclear cap…

I mean it was messed up, which is why the other world powers raced to develop their own capabilities. And it remains messed up to this day - some countries get to be under their own nuclear umbrella, while others don't. This kind of selective distribution of superpowers doesn't lead to great outcomes

in that case in particular it led to 80 years of relatively calm geopolitics kinetically, all things considered. I'm not sure I want to live through an AI cold war, but it sure seems I don't get to choose.

Re: Project Glasswing: Securing critical software for the AI era

#202

Post by SheinhardtWigCo »

Miraste wrote: post #27

>We plan to launch new safeguards with an upcoming Claude Opus model, allowing us to improve and refine them with a model that does not pose the same level of risk as Mythos Preview2. This seems like the real news. Are they saying they're going to release an intentionally degraded model as the next Opus? Big opportunity for the other labs, if that's true.

The other labs already censor their models. Everyone is trying to find the sweet spot where performance and ‘alignment’ are both maximized. This seems no different

Re: Project Glasswing: Securing critical software for the AI era

#203

Post by conradkay »

conradkay wrote: post #129
justincormack wrote: post #73

Earlier quoted context omitted.

Software security heavily favours the attacker (ex. its much easier to find a single vulnerability than to patch every vulnerability). Thus with better tools and ample time to reach steady-state, we would expect software to remain insecure.

That generally makes sense to me, but I wonder if it's different when the attacker and defender are using the same tool (Mythos in this case) Maybe you just spend more on tokens by some factor than the attackers do combined, and end up mostly okay. Put another way, if there's 20 vulnerabilities that Mythos is capable of finding, maybe it's reasonable to find all of them?

From the red team post https://red.anthropic.com/2026/mythos-preview/

"Most security tooling has historically benefitted defenders more than attackers. When the first software fuzzers were deployed at large scale, there were concerns they might enable attackers to identify vulnerabilities at an increased rate. And they did. But modern fuzzers like AFL are now a critical component of the security ecosystem: projects like OSS-Fuzz dedicate significant resources to help secure key open source software.

We believe the same will hold true here too—eventually. Once the security landscape has reached a new equilibrium, we believe that powerful language models will benefit defenders more than attackers, increasing the overall security of the software ecosystem. The advantage will belong to the side that can get the most out of these tools. In the short term, this could be attackers, if frontier labs aren’t careful about how they release these models. In the long term, we expect it will be defenders who will more efficiently direct resources and use these models to fix bugs before new code ever ships. "

Re: Project Glasswing: Securing critical software for the AI era

#204

Post by aurareturn »

dang wrote: post #165

Related ongoing threads: System Card: Claude Mythos Preview [pdf] - https://news.ycombinator.com/item?id=47679258 Assessing Claude Mythos Preview's cybersecurity capabilities - https://news.ycombinator.com/item?id=47679155 I can't tell which of the 3 current threads should be merged - they all seem significant. Anyone?

Let them all live. This is going to blow up one thread if you merge them.

Re: Project Glasswing: Securing critical software for the AI era

#205

Post by impulser_ »

dreis_sw wrote: post #43
impulser_ wrote: post #18

So they are only giving access to their smartest model to corporations. You think these AI companies are really going to give AGI access to everyone. Think again. We better fucking hope open source wins, because we aren't getting access if it doesn't.

It also took many years to put capable computers in the hands of the general public, but it eventually happened. I believe the same will happen here, we're just in the Mainframe era of AI.

Yeah, but computers don't replace you. They are building AI to replace you. You think if these companies eventually achieve AGI that you are going to give you access to it? They are already gatekeeping an LLM because they don't trust you with it.

Re: Project Glasswing: Securing critical software for the AI era

#206

Post by _1100 »

pizlonator wrote: post #182
hector_vasquez wrote: post #178

Earlier quoted context omitted.

Releasing the model to bad actors at the same time as the major OS, browser, and security companies would be one idea. But some might consider that "messed up" too, whatever you mean by that. But in terms of acting in the public benefit, it seems consistent to work with companies that can make significant impact on users' security. The stated goal of Project Glasswing is to "secure the world's most critical software,…

I don't trust a corpo to choose what is "most critical". That's what's messed up about it.

That is a fine stance to hold but some facts are still true regardless of your view on large businesses.

For example, it will benefit more people to secure Microsoft or Amazon services than it would be to secure a smaller, less corporate player in those same service ecosystems.

You could go on to argue that the second order effects of improving one service provider over another chooses who gets to play, but that is true whether you choose small or large businesses, so this argument devolves into “who are we to choose on behalf of others”.

Which then comes back to “we should secure what the market has chosen in order to provide the greatest benefit.”

Re: Project Glasswing: Securing critical software for the AI era

#208

Post by qingcharles »

georgemcbay wrote: post #106
9cb14c1ec0 wrote: post #52

Now, its very possible that this is Anthropic marketing puffery, but even if it is half true it still represents an incredible advancement in hunting vulnerabilities. It will be interesting to see where this goes. If its actually this good, and Apple and Google apply it to their mobile OS codebases, it could wipe out the commercial spyware industry, forcing them to rely more on hacking humans rather than hacking mobi…

> It will be interesting to see where this goes. If its actually this good, and Apple and Google apply it to their mobile OS codebases, it could wipe out the commercial spyware industry, forcing them to rely more on hacking humans rather than hacking mobile OSes. It will likely cause some interesting tensions with government as well. eg. Apple's official stance per their 2016 customer letter is no backdoors: https://…

I assume that right now some of the biggest spenders on tokens at Anthropic are state intelligence communities who are burning up GPU cycles on Android, Chromium, WebKit code bases etc trying to find exploits.

Re: Project Glasswing: Securing critical software for the AI era

#209

Post by dievskiy »

impulser_ wrote: post #18

So they are only giving access to their smartest model to corporations. You think these AI companies are really going to give AGI access to everyone. Think again. We better fucking hope open source wins, because we aren't getting access if it doesn't.

Would you hope that it would be released today so that evil actors could invest few millions to search for 0days across popular open-source repos?

Re: Project Glasswing: Securing critical software for the AI era

#210

Post by bink »

conradkay wrote: post #98
endunless wrote: post #15

Another Anthropic PR release based on Anthropic’s own research, uncorroborated by any outside source, where the underlying, unquestioned fact is that their model can do something incredible. > AI models have reached a level of coding capability where they can surpass all but the most skilled humans at finding and exploiting software vulnerabilities I like Anthropic, but these are becoming increasingly transparent att…

I would've basically agreed with you until I'd seen this talk: https://www.youtube.com/watch?v=1sd26pWhfmg Maybe a bad example since Nicholas works at Anthropic, but they're very accomplished and I doubt they're being misleading or even overly grandiose here See the slide 13 minutes in, which makes it look to be quite a sudden change

I watched the talk as well and it's very interesting. But isn't this just a buffer overflow in the NFS client code? The way the LLM diagnosed the flaw, demonstrated the bug, and wrote an exploit is cool and all, but doesn't this still come down to the fact that the NFS client wasn't checking bounds before copying a bunch of data into a fixed length buffer? I'm not sure why this couldn't have been detected with static analysis.
Post reply on HN