Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Expanding Project Glasswing

anthropic.com

Post reply on HN
Print view View original

211–220 of 261 posts

Re: Expanding Project Glasswing

#211

Post by jen20 »

airstrike wrote: post #126

I'll share the first-hand account I recently got from someone else. > We've used it at work > it is... not as hype as everyone is concerned about > I'd argue the framework around it for security scanning is the arguably more useful side of the tool, definitely doesnt take a huge model to get all the issues it flagged on our systems > For us, it absolutely flooded us with noise > I mean hundreds if not thousands of fa…

> The biggest issue it created was the execs treated every issue it produced like it was a drop everything and fix the issue type deal

While this is definitely not the ideal end of the spectrum either, execs treating security issues as something serious instead of annoyances that should only be addressed if revenue can be tied to doing so is a welcome improvement.

Re: Expanding Project Glasswing

#212

Post by strictnein »

Work in a top tier security org at a Fortune 50. We still can't get access to this stuff, even though we've reached out repeatedly.

I mention this because if you're frustrated that you can't access it, you're not alone. Even with our company's heft and a security org that is very well known in the industry we're getting nowhere.

Re: Expanding Project Glasswing

#213

Post by 15155 »

overfeed wrote: post #137
ianm218 wrote: post #33

In case the topic of memory safety is interesting to anyone I've been experimenting with using AI agents to port common web infra projects to safe/ performant Rust. Somewhat inspired by the Bun port - was thinking that at some point memory safety might be such a big deal that people just need drop in replacements. - Valkey/ Redis port here https://github.com/ianm199/valdr (passes ~99% of single node test suite, real…

Are you preserving the original software licenses, or AI-laundering the code in the manner[1] of https://malus.sh 1. AI-rewrites are not clean room implementations.

Do you have any specific case citations to substantiate this idea?

Re: Expanding Project Glasswing

#214

Post by protocolture »

airstrike wrote: post #126

I'll share the first-hand account I recently got from someone else. > We've used it at work > it is... not as hype as everyone is concerned about > I'd argue the framework around it for security scanning is the arguably more useful side of the tool, definitely doesnt take a huge model to get all the issues it flagged on our systems > For us, it absolutely flooded us with noise > I mean hundreds if not thousands of fa…

Seems like there might be a market for a product that just prefixes "The AI Said" on emails to executives about security vulns.

Re: Expanding Project Glasswing

#215

Post by protocolture »

mofeien wrote: post #167
mekpro wrote: post #40

It’s clear that Anthropic has run out of the compute capacity needed to serve Mythos publicly. They’re using security concerns to mask their inability to deliver the model at scale, while still trying to maintain their lead over OpenAI. As a result, they’ve chosen to release it privately under the banner of an “ethical” rollout.

Jack Clark, co-founder of Anthropic said the following at an Oxford lecture last week ([0], at around 10 and 12 mins): "It's a technology that we do not fully understand because it's more grown than made. And it is a technology that you can concoct plausible scenarios where it could kill every single person on the planet. So to think building this technology is without risk would be an act of hubris or insanity. [...…

>you can concoct plausible scenarios where it could kill every single person on the planet.

Idiots can scary black box their way to that concern. Plausible? Not so much.

Re: Expanding Project Glasswing

#216

Post by protocolture »

mentalgear wrote: post #5

Here's my big fear: Even IF (and that's a BIG if) we get all critical vulnerabilities fixed in tech (before adversarial/state-actors turn up with open attack models) - we still have (in at least a year) models that will be so good in social engineering that they can still (given enough tokens) gain access to whatever system they want. If society can't trust banks and other institutions to safely control their data, w…

>Here's my big fear: Even IF (and that's a BIG if) we get all critical vulnerabilities fixed in tech (before adversarial/state-actors turn up with open attack models) - we still have (in at least a year) models that will be so good in social engineering that they can still (given enough tokens) gain access to whatever system they want.

I was working at the fruit company when they just hard stopped people from recovering their fruitcloud accounts via phone support due to social engineering.

Social Engineering risk just increases the burden on the consumer/internal support services. The risk is that not everyone has pulled up stumps to protect these services. After a few high profile fuck ups they will. The herd loses 2 beasts and the rest wander away from that water hole.

Its much like how after bitlocker we dont have user access to backup server disks anymore. The lesson was learned and we moved on. Lots of high profile fuckups but we dont get those anymore. CTO's were forced, basically at gunpoint, to adapt or die.

Re: Expanding Project Glasswing

#218

Post by insanitybit »

MostlyStable wrote: post #187
insanitybit wrote: post #174

Earlier quoted context omitted.

Also impossible imo so it's moot.

Because you think that current models can, in a practical sense, find an infinite number of vulnerabilities, or you think that they can find so many that it isn't possible to fix them? In other words: do you think that the impossibility lines in exhausting the number finds or does the impossibility lie in fixing them? In either case, do you think that this was also true pre-AI? That is to say: it was not possible to,…

I think that there are, practically, infinite vulnerabilities in common and critical software - browsers, operating systems, etc. So discovering all of them is not tractable, and even if we 100x our rate of discovery it won't matter.

> In either case, do you think that this was also true pre-AI? That is to say: it was not possible to, given some set of practical resource constraints, find and fix all the vulnerabilities that a similarly-resourced group would find?

Yes.

> If so, then would you say that you just fundamentally don't believe in secure software and the only defense is lack of attention?

I believe in security software, few people are building it though and the majority of relevant attack surface is dogshit for security.

Squashing vulns via discovery is irrelevant to security. If we want safer software it has to be built to be safer.

Re: Expanding Project Glasswing

#219

Post by graerg »

protocolture wrote: post #215
mofeien wrote: post #167

Earlier quoted context omitted.

Jack Clark, co-founder of Anthropic said the following at an Oxford lecture last week ([0], at around 10 and 12 mins): "It's a technology that we do not fully understand because it's more grown than made. And it is a technology that you can concoct plausible scenarios where it could kill every single person on the planet. So to think building this technology is without risk would be an act of hubris or insanity. [...…

>you can concoct plausible scenarios where it could kill every single person on the planet. Idiots can scary black box their way to that concern. Plausible? Not so much.

It is already very plausible (and has been since the 1950s) without the advent of LLMs. This is just another layer on top of the preexisting and very plausible existential threats we already face.

Re: Expanding Project Glasswing

#220

Post by huflungdung »

airstrike wrote: post #126

I'll share the first-hand account I recently got from someone else. > We've used it at work > it is... not as hype as everyone is concerned about > I'd argue the framework around it for security scanning is the arguably more useful side of the tool, definitely doesnt take a huge model to get all the issues it flagged on our systems > For us, it absolutely flooded us with noise > I mean hundreds if not thousands of fa…

[dead]
Post reply on HN