Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Glasswing: Securing critical software for the AI era

anthropic.com

Post reply on HN
Print view View original

241–250 of 921 posts

Re: Project Glasswing: Securing critical software for the AI era

#241

Post by SubiculumCode »

torginus wrote: post #53
redfloatplane wrote: post #13

The system card for Claude Mythos (PDF): https://www-cdn.anthropic.com/53566bf5440a10affd749724787c89... Interesting to see that they will not be releasing Mythos generally. [edit: Mythos Preview generally - fair to say they may release a similar model but not this exact one] I'm still reading the system card but here's a little highlight: > Early indications in the training of Claude Mythos Preview suggested that th…

Just reading this, the inevitable scaremongering about biological weapons comes up. Since most of us here are devs, we understand that software engineering capabilities can be used for good or bad - mostly good, in practice. I think this should not be different for biology. I would like to reach out and talk to biologists - do you find these models to be useful and capable? Can it save you time the way a highly capab…

It is not scaremongering.

Re: Project Glasswing: Securing critical software for the AI era

#242

Post by lanternfish »

pizlonator wrote: post #218
thorncorona wrote: post #200

Earlier quoted context omitted.

Let's let the California HSR committee do it instead!

I'm too much of an anarchist for that. I believe what I said: > I think it would be net better for the public if they just made Mythos available to everyone.

10 Axios's within 5 days.

Re: Project Glasswing: Securing critical software for the AI era

#243

Post by Gigachad »

9cb14c1ec0 wrote: post #52

Now, its very possible that this is Anthropic marketing puffery, but even if it is half true it still represents an incredible advancement in hunting vulnerabilities. It will be interesting to see where this goes. If its actually this good, and Apple and Google apply it to their mobile OS codebases, it could wipe out the commercial spyware industry, forcing them to rely more on hacking humans rather than hacking mobi…

Apple has already largely crushed hacking with memory tagging on the iPhone 17 and lockdown mode. Architectural changes, safer languages, and sandboxing have done more for security than just fixing bugs when you find them.

Re: Project Glasswing: Securing critical software for the AI era

#244

Post by kenjackson »

pizlonator wrote: post #168
SheinhardtWigCo wrote: post #163

Earlier quoted context omitted.

> picking who gets to benefit from their newly enhanced cybersecurity capabilities You could say this about coordinated disclosure of any widespread 0-day or new bug class, though

That's a really good point! But: - Coordinated disclosure is ethically sketchy. I know why we do it, and I'm not saying we shouldn't. But it's not great. - This isn't a single disclosure. This is a new technology that dramatically increases capability. So, even if we thought that coordinated disclosure was unambiguously good, then I think we'd still need to have a new conversation about Mythos

So private companies shouldn’t get to determine who they provide services to? Assuming no extremely malicious intent, I’d be fine if they said it was only going to McDonalds because the founders like Big Macs.

Re: Project Glasswing: Securing critical software for the AI era

#245

Post by SubiculumCode »

nonameiguess wrote: post #119
torginus wrote: post #53

Earlier quoted context omitted.

Just reading this, the inevitable scaremongering about biological weapons comes up. Since most of us here are devs, we understand that software engineering capabilities can be used for good or bad - mostly good, in practice. I think this should not be different for biology. I would like to reach out and talk to biologists - do you find these models to be useful and capable? Can it save you time the way a highly capab…

Surely more than 10% of the time consumed by going to market with a cancer treatment is giving it to living organisms and waiting to see what happens, which can't be made any faster with software. That's not to say speedups can't happen, but 90% can't happen. Not that that justifies doom and gloom, but there is a pretty inescapable assymetry here between weaponry and medicine. You can manufacture and blast every conc…

It is easier to destroy than it is to protect or fix, as a general rule of the universe. I would not feel so confident about the speed of the testing loop keeping things in check.

Re: Project Glasswing: Securing critical software for the AI era

#246

Post by Gigachad »

tdaltonc wrote: post #213
jryio wrote: post #11

Let's fast forward the clock. Does software security converge on a world with fewer vulnerabilities or more? I'm not sure it converges equally in all places. My understanding is that the pre-AI distribution of software quality (and vulnerabilities) will be massively exaggerated. More small vulnerable projects and fewer large vulnerable ones. It seems that large technology and infrastructure companies will be able to…

Depends - do you think people are good at keeping their fridge firmware up-to-date?

I’m good at keeping my fridge off the internet.

Re: Project Glasswing: Securing critical software for the AI era

#247

Post by SubiculumCode »

temp123789246 wrote: post #207

OpenAI initially claimed that GPT-2 was too dangerous to release in 2019. How many times will labs repeat the same absurd propaganda?

Anthropic and OpenAI have very different cultures and ethos. Point to other times where anthropic has gone the way of cheap marketing tricks. Now look at openAI. Not even close.

Re: Project Glasswing: Securing critical software for the AI era

#248

Post by torginus »

bonsai_spool wrote: post #188
torginus wrote: post #105

Earlier quoted context omitted.

Yes that is correct. I would like a large body of experience and consenus to rely on as opposed to the regular 'trust the experts' argument, which has been shown for decades that is a deeply flawed and easy to manipulate argument.

> Yes that is correct. I would like a large body of experience and consenus to rely on as opposed to the regular 'trust the experts' argument, which has been shown for decades that is a deeply flawed and easy to manipulate argument. Yes, it is far inferior to the 'Trust torginus and his ability to understand the large body of experience that other actual subject-matter-experts have somehow not understood' strategy

It's not my credibility I want to measure against Anthropic's. I just said to apply the same logic to biology you would apply for software development.

The parallels here are quite remarkable imo, but defer to your own judgement on what you make of them.

Re: Project Glasswing: Securing critical software for the AI era

#249

Post by SpicyLemonZest »

Gigachad wrote: post #240
SubiculumCode wrote: post #227

Earlier quoted context omitted.

Anthropic has behaved the least like this of the AI companies.

They made a claim that 100% of code would be AI generated in a year, over a year ago.

They were right, it's hit 100% at a number of large tech companies. (They missed their initial prediction of 90% 6 months ago, because the models then available publicly weren't capable enough.)

Re: Project Glasswing: Securing critical software for the AI era

#250

Post by [deleted] »

pizlonator wrote: post #218
thorncorona wrote: post #200

Earlier quoted context omitted.

Let's let the California HSR committee do it instead!

I'm too much of an anarchist for that. I believe what I said: > I think it would be net better for the public if they just made Mythos available to everyone.

[deleted]
Post reply on HN