Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Glasswing: Securing critical software for the AI era

anthropic.com

Post reply on HN
Print view View original

591–600 of 921 posts

Re: Project Glasswing: Securing critical software for the AI era

#591

Post by jstummbillig »

ofjcihen wrote: post #508

I’m sure the new model is a step above the old one but I can’t be the only person who’s getting tired of hearing about how every new iteration is going to spell doom/be a paradigm shift/change the entire tech industry etc. I would honestly go so far as to say the overhype is detrimental to actual measured adoption.

> how every new iteration is going to spell doom/be a paradigm shift/change the entire tech industry etc.

It's much the dynamic between parents and a child. The child, with limited hindsight, almost zero insight and no ability to forecast, is annoyed by their parents. Nothing bad ever happens! Why won't parents stop being so worried all the time and make a fuss over nothing?

The parents, which the child somewhat starts to realize but not fully, have no clue what they are doing. There is a lot they don't know and are going to be wrong about, because it's all new to them. But, what they do have is a visceral idea of how bad things could be and that's something they have to talk to their child about too.

In the eyes of the parents the child is % dead all the time. Assigning the wrong % makes you look like an idiot and not being able to handle any % too. In the eyes of the child actions leading to death are not even a concept. Hitting the right balance is probably hard, but not for the reasons the child thinks.

Re: Project Glasswing: Securing critical software for the AI era

#592

Post by dota_fanatic »

ofjcihen wrote: post #561
dota_fanatic wrote: post #544

Earlier quoted context omitted.

> Non-experts can also leverage Mythos Preview to find and exploit sophisticated vulnerabilities. Engineers at Anthropic with no formal security training have asked Mythos Preview to find remote code execution vulnerabilities overnight, and woken up the following morning to a complete, working exploit. In other cases, we’ve had researchers develop scaffolds that allow Mythos Preview to turn vulnerabilities into explo…

I mean yeah. I’ve had these successes without scaffolding or really anything past Claude CLI and a small prompt as well?

Just saw your edit. I'll leave it at this, this is why it's news to me, because by their very own measurements, Opus simply doesn't come close. I trust their empirical evidence over your hearsay. But feel free to prove me wrong with evidence.

> With one run on each of roughly 7000 entry points into these repositories, Sonnet 4.6 and Opus 4.6 reached tier 1 in between 150 and 175 cases, and tier 2 about 100 times, but each achieved only a single crash at tier 3. In contrast, Mythos Preview achieved 595 crashes at tiers 1 and 2, added a handful of crashes at tiers 3 and 4, and achieved full control flow hijack on ten separate, fully patched targets (tier 5).

Re: Project Glasswing: Securing critical software for the AI era

#593

Post by kachnuv_ocasek »

4ndrewl wrote: post #569
qnleigh wrote: post #526

Earlier quoted context omitted.

There is plenty of overhyping, no one denies that. But the antidote is not to dismiss everything. Ignore the words and look at the data. In this case, I see a pretty strong case that this will significantly change computer security. They provide plenty of evidence that the models can create exploits autonomously, meaning that the cost of finding valuable security breaches will plummet once they're widely available.

Is there any actual independent data though, or verification of any of these claims? As it stands this is just a marketing programme for all involved.

What would be the product they're marketing by this campaign?

Re: Project Glasswing: Securing critical software for the AI era

#594

Post by ofjcihen »

jstummbillig wrote: post #591
ofjcihen wrote: post #508

I’m sure the new model is a step above the old one but I can’t be the only person who’s getting tired of hearing about how every new iteration is going to spell doom/be a paradigm shift/change the entire tech industry etc. I would honestly go so far as to say the overhype is detrimental to actual measured adoption.

> how every new iteration is going to spell doom/be a paradigm shift/change the entire tech industry etc. It's much the dynamic between parents and a child. The child, with limited hindsight, almost zero insight and no ability to forecast, is annoyed by their parents. Nothing bad ever happens! Why won't parents stop being so worried all the time and make a fuss over nothing? The parents, which the child somewhat star…

That feels like a very complex way of looking at it. Another way would be to say “potentially profit seeking companies have an incentive to oversell products even if they’re good”.

Re: Project Glasswing: Securing critical software for the AI era

#596

Post by ACCount37 »

conradkay wrote: post #148
rishabhaiover wrote: post #118

Earlier quoted context omitted.

I would have not believed your argument 3 months ago but I strongly suspect Anthropic actively engages in model quality throttling due to their compute constraints. Their recent deal for multi GWs worth of data center might help them correct their approach.

For what it's worth Anthropic explicity denies that. "To state it plainly: We never reduce model quality due to demand, time of day, or server load" Also can see https://marginlab.ai/trackers/claude-code/ It's very interesting to me how widespread this conception is. Maybe it's as simple as LLM productivity degrading over time within a project, as slop compounds. Or more recently since they added a 1m context window,…

Posted this a while ago:

>Models are not "degrading". They're not being "secretly quantized". And no one is swapping out your 1.2T frontier behemoth for a cheap 120B toy and hoping you wouldn't notice!

>It's just that humans are completely full of shit, and can't be trusted to measure LLM performance objectively!

>Every time you use an LLM, you learn its capability profile better. You start using it more aggressively at what it's "good" at, until you find the limits and expose the flaws. You start paying attention to the more subtle issues you overlooked at first. Your honeymoon period wears off and you see that "the model got dumber". It didn't. You got better at pushing it to its limits, exposing the ways in which it was always dumb.

>Now, will the likes of Anthropic just "API error: overloaded" you on any day of the week that ends in Y? Will they reduce your usage quotas and hope that you don't notice because they never gave you a number anyway? Oh, definitely. But that "they're making the models WORSE" bullshit lives in people's heads way more than in any reality.

Re: Project Glasswing: Securing critical software for the AI era

#597

Post by calf »

anabis wrote: post #439
peterldowns wrote: post #355

Earlier quoted context omitted.

are there any tricks you'd suggest, or starter prompts, for using claude to analyze my own company's services for security problems?

Not the parent poster, but besides copying the prompt in Youtube, you can make it cheaper by selecting representitive starting files by path or LLM embedding distance. Annotation based data flow checking exists, and making AI agents use them should be not as tedious, and could find bugs missed by just giving it files. The result from data flow checks can be fed to AI agents to verify.

As a curious passerby what does such a prompt look like? Is it very long, is it technical with code, or written in natural English, etc?

Re: Project Glasswing: Securing critical software for the AI era

#599

Post by ozozozd »

kachnuv_ocasek wrote: post #593
4ndrewl wrote: post #569

Earlier quoted context omitted.

Is there any actual independent data though, or verification of any of these claims? As it stands this is just a marketing programme for all involved.

What would be the product they're marketing by this campaign?

The product they launched?

Re: Project Glasswing: Securing critical software for the AI era

#600

Post by ACCount37 »

ofjcihen wrote: post #594
jstummbillig wrote: post #591

Earlier quoted context omitted.

> how every new iteration is going to spell doom/be a paradigm shift/change the entire tech industry etc. It's much the dynamic between parents and a child. The child, with limited hindsight, almost zero insight and no ability to forecast, is annoyed by their parents. Nothing bad ever happens! Why won't parents stop being so worried all the time and make a fuss over nothing? The parents, which the child somewhat star…

That feels like a very complex way of looking at it. Another way would be to say “potentially profit seeking companies have an incentive to oversell products even if they’re good”.

[flagged]
Post reply on HN