Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Expanding Project Glasswing

anthropic.com

Post reply on HN
Print view View original

191–200 of 261 posts

Re: Expanding Project Glasswing

#191

Post by JacobAsmuth »

rfgplk wrote: post #114
yanis_t wrote: post #20

Is there any evidence Mythos is qualitatively better than the Opus 4.x? I'm afraid that the usual mantra that "we just need more scale" that worked well for attracting investments, is not working anymore - bigger models provide marginal improvements while naturally get much more expensive to run. Is this why both Anthropic and OpenAI are rushing for IPOs this year?

It probably isn't, at least in terms of security or memory safety. The current models can already sniff out all memory vulnerabilities with relative ease, you can't really beat that.

Have you read firefoxes findings? They found it to be qualitatively improved over Opus, and have published several of the resulting CVEs as well as more detailed numbers.

Re: Expanding Project Glasswing

#192

Post by JumpCrisscross »

bgilroy26 wrote: post #147
airstrike wrote: post #126

I'll share the first-hand account I recently got from someone else. > We've used it at work > it is... not as hype as everyone is concerned about > I'd argue the framework around it for security scanning is the arguably more useful side of the tool, definitely doesnt take a huge model to get all the issues it flagged on our systems > For us, it absolutely flooded us with noise > I mean hundreds if not thousands of fa…

It seems like there is a genuine communication breakdown between management and engineering. Engineers know that there are vulnerabilities all over the place and that there have been for ages and that where the rubber hits the road every vulnerability does not represent a successful exploit by some nefarious actor. Management can often treat cybersecurity like a black box that represents millions upon millions in lia…

Yeah, I’m getting the sense that Mythos is for cybersecurity what blockchain was for back-end finance. A bit useful. But mostly good for bringing attention to upgrading neglected systems.

Re: Expanding Project Glasswing

#193

Post by JumpCrisscross »

NiloCK wrote: post #78
mekpro wrote: post #40

It’s clear that Anthropic has run out of the compute capacity needed to serve Mythos publicly. They’re using security concerns to mask their inability to deliver the model at scale, while still trying to maintain their lead over OpenAI. As a result, they’ve chosen to release it privately under the banner of an “ethical” rollout.

I find this line of reasoning highly dubious. Yes, Anthropic is compute constrained, even after the SpaceX Colossus deal. But supply constraints are the normal operating mode of any market. Anthropic could choose to serve whatever models it pleases at whatever price points it chooses and let the market decide where the value is. If Mythos at $X overwhelms their capacity, they could just charge $X+1 . If still overwhe…

> Mythos at $X overwhelms their capacity, they could just charge $X+1

This may not be as valuable in the long term as getting committed customers hooked at a sustainable price.

Re: Expanding Project Glasswing

#194

Post by yalogin »

We are in the early stages of monetizing the AI stack/service and Anthropic is set to take it in. Not sure if it’s cost effective for them or not but they are clear winner here. They have created this awareness among executives about the value and need for AI and that is what matters, it will be budgeted accordingly. They are positioning it as a must have not just for productivity but also beyond that as a Swiss army tool , pretty smart

Re: Expanding Project Glasswing

#195

Post by bcrosby95 »

JacobAsmuth wrote: post #191
rfgplk wrote: post #114

Earlier quoted context omitted.

It probably isn't, at least in terms of security or memory safety. The current models can already sniff out all memory vulnerabilities with relative ease, you can't really beat that.

Have you read firefoxes findings? They found it to be qualitatively improved over Opus, and have published several of the resulting CVEs as well as more detailed numbers.

They also seem to point to it being more the harness than the model itself.

Re: Expanding Project Glasswing

#196

Post by cassianoleal »

dymk wrote: post #128
cassianoleal wrote: post #122

In the meantime, not everyone with actual access to the model are all that impressed. https://cyberplace.social/@GossiTheDog/116679693992983945

“Cybersecurity weather person and award winning shitposter.” why are they someone we should pay attention to the opinion of?

There you go, judge it yourself.

https://www.linkedin.com/in/kevin-beaumont-security/

Re: Expanding Project Glasswing

#197

Post by sbayg »

airstrike wrote: post #126

I'll share the first-hand account I recently got from someone else. > We've used it at work > it is... not as hype as everyone is concerned about > I'd argue the framework around it for security scanning is the arguably more useful side of the tool, definitely doesnt take a huge model to get all the issues it flagged on our systems > For us, it absolutely flooded us with noise > I mean hundreds if not thousands of fa…

In other words it creates work. In other words Jevons paradox.

I can’t wait for the first court case where an LLM surfaces a vuln, lazy devs ignore it, and someone later sues the company into oblivion for liability.

Re: Expanding Project Glasswing

#198

Post by clbrmbr »

mofeien wrote: post #167
mekpro wrote: post #40

It’s clear that Anthropic has run out of the compute capacity needed to serve Mythos publicly. They’re using security concerns to mask their inability to deliver the model at scale, while still trying to maintain their lead over OpenAI. As a result, they’ve chosen to release it privately under the banner of an “ethical” rollout.

Jack Clark, co-founder of Anthropic said the following at an Oxford lecture last week ([0], at around 10 and 12 mins): "It's a technology that we do not fully understand because it's more grown than made. And it is a technology that you can concoct plausible scenarios where it could kill every single person on the planet. So to think building this technology is without risk would be an act of hubris or insanity. [...…

Was a good watch, tho would have liked to be there in person. Props to Brenden & his Cosmos team for really setting the bar.

Re: Expanding Project Glasswing

#199

Post by ianm218 »

0x000xca0xfe wrote: post #184
ianm218 wrote: post #173

Earlier quoted context omitted.

Hmm I view open source as purely positive sum. Valkey was forked from Redis in the first place. But this is more about memory safety - you can have immense respect for the giants who built these tools but also be worried that memory safety might become an even bigger deal. If someone found a memory zero day in nginx or openSSL for example that is a very big deal! I think this is one strategy we should look into, hope…

And what if someone gets pwned by a bog standard logic or input validation bug in your slopped together "nginx" that is not present in the original?

If someone is running projects with a big "alpha" tag in production, exposed to the web they very well might get pwned haha!

Re: Expanding Project Glasswing

#200

Post by SpicyLemonZest »

0123456789ABCDE wrote: post #186
airstrike wrote: post #126

I'll share the first-hand account I recently got from someone else. > We've used it at work > it is... not as hype as everyone is concerned about > I'd argue the framework around it for security scanning is the arguably more useful side of the tool, definitely doesnt take a huge model to get all the issues it flagged on our systems > For us, it absolutely flooded us with noise > I mean hundreds if not thousands of fa…

why are folks looking at the output of the first pass? my understanding, and experience, is that you 1. run a bunch of sessions with small permutations to create variety , 2. run more sessions dedupe reports into a smaller collections of potential vulns, 3. run a handful of agents at max effort to write PoCs + write-ups, 4. rank findings, 5. finally look at what, if anything that, was found. maybe ask questions, try…

What Project Glasswing claimed at launch is that Mythos can "surpass all but the most skilled humans at finding and exploiting software vulnerabilities". What you're describing sounds more like making skilled humans more effective at penetration testing. That's cool, but it's not clear how much it matters, because most security teams were not previously bottlenecked on penetration testing capacity.
Post reply on HN