Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Glasswing: Securing critical software for the AI era

anthropic.com

Post reply on HN
Print view View original

531–540 of 921 posts

Re: Project Glasswing: Securing critical software for the AI era

#531

Post by orbital-decay »

bottlepalm wrote: post #470
LiamPowell wrote: post #399

> Mythos Preview identified a number of Linux kernel vulnerabilities that allow an adversary to write out-of-bounds (e.g., through a buffer overflow, use-after-free, or double-free vulnerability.) Many of these were remotely-triggerable. However, even after several thousand scans over the repository, because of the Linux kernel’s defense in depth measures Mythos Preview was unable to successfully exploit any of these…

We've very quickly reached the point where AI models are now too dangerous to publicly release, and HN users are still trying to trivialize the situation.

GPT-2 was already too dangerous to publicly release according to OpenAI, however they still did. If something is not dangerous, it's also not useful.

Re: Project Glasswing: Securing critical software for the AI era

#532

Post by solenoid0937 »

slopinthebag wrote: post #523
solenoid0937 wrote: post #512

Earlier quoted context omitted.

The disbelief in this thread is wild. Most of yall are cooked if you think this is actually the case.

The only people who are "cooked" are those who rely on SOTA models to function in their jobs, and companies who are desperate to regulate open / local models to maintain their marketshare.

If you aren't relying on a SOTA model to do your job, you aren't doing your job right (and are cooked.)

Re: Project Glasswing: Securing critical software for the AI era

#533

Post by tex0 »

9cb14c1ec0 wrote: post #52

Now, its very possible that this is Anthropic marketing puffery, but even if it is half true it still represents an incredible advancement in hunting vulnerabilities. It will be interesting to see where this goes. If its actually this good, and Apple and Google apply it to their mobile OS codebases, it could wipe out the commercial spyware industry, forcing them to rely more on hacking humans rather than hacking mobi…

The interesting selling point about this, if the claims are substantial, is that nobody will be able to produce secure software without access to one of these models. Good for them $$$ ^^

Re: Project Glasswing: Securing critical software for the AI era

#535

Post by ofjcihen »

qnleigh wrote: post #526
ofjcihen wrote: post #508

I’m sure the new model is a step above the old one but I can’t be the only person who’s getting tired of hearing about how every new iteration is going to spell doom/be a paradigm shift/change the entire tech industry etc. I would honestly go so far as to say the overhype is detrimental to actual measured adoption.

There is plenty of overhyping, no one denies that. But the antidote is not to dismiss everything. Ignore the words and look at the data. In this case, I see a pretty strong case that this will significantly change computer security. They provide plenty of evidence that the models can create exploits autonomously, meaning that the cost of finding valuable security breaches will plummet once they're widely available.

With the right prompting (mostly creating a narrative that justifies the subject matter as okay to perform) other models have already been doing this for me though. That’s another confusing bit for me about how this is portrayed and I refuse to believe I’m a revolutionary user right?

I mean I’m sitting on $10k worth of bug payouts right now partially because that was already a thing.

Re: Project Glasswing: Securing critical software for the AI era

#536

Post by qnleigh »

rootkea wrote: post #459
LiamPowell wrote: post #399

> Mythos Preview identified a number of Linux kernel vulnerabilities that allow an adversary to write out-of-bounds (e.g., through a buffer overflow, use-after-free, or double-free vulnerability.) Many of these were remotely-triggerable. However, even after several thousand scans over the repository, because of the Linux kernel’s defense in depth measures Mythos Preview was unable to successfully exploit any of these…

> The model autonomously found and chained together several vulnerabilities in the Linux kernel—the software that runs most of the world’s servers—to allow an attacker to escalate from ordinary user access to complete control of the machine.

I'm confused on this point. The text you quote implies that they were able to build an exploit, but the text quoted in the parent comment implies that they were not.

What were they actually able to do and not do? I got confused by this when reading the article as well.

Re: Project Glasswing: Securing critical software for the AI era

#537

Post by Ms-J »

Anthropic and ClosedAI are some of the biggest bullshitters in the industry.

The is no moat, no special "capability" and when the time comes when we can run these models on our own, they will be cheap SaaS gimmicks marketed to corporate and making more slop pictures for social media.

Re: Project Glasswing: Securing critical software for the AI era

#538

Post by slopinthebag »

solenoid0937 wrote: post #532
slopinthebag wrote: post #523

Earlier quoted context omitted.

The only people who are "cooked" are those who rely on SOTA models to function in their jobs, and companies who are desperate to regulate open / local models to maintain their marketshare.

If you aren't relying on a SOTA model to do your job, you aren't doing your job right (and are cooked.)

Whatever helps you sleep at night kiddo

Re: Project Glasswing: Securing critical software for the AI era

#540

Post by rossjudson »

Security by obscurity is over. The security vs usability balance is about to get a hard reset.

I think a number of black swan events are imminent, and it will substantially change the financial calculus that decides to put security behind revenue.

Any hole will be found, and any hole will be exploited. Plug as many holes as you can, and make lateral movement as painful as possible.

Post reply on HN