Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Glasswing: Securing critical software for the AI era

anthropic.com

Post reply on HN
Print view View original

391–400 of 921 posts

Re: Project Glasswing: Securing critical software for the AI era

#391

Post by pizlonator »

kenjackson wrote: post #244
pizlonator wrote: post #168

Earlier quoted context omitted.

That's a really good point! But: - Coordinated disclosure is ethically sketchy. I know why we do it, and I'm not saying we shouldn't. But it's not great. - This isn't a single disclosure. This is a new technology that dramatically increases capability. So, even if we thought that coordinated disclosure was unambiguously good, then I think we'd still need to have a new conversation about Mythos

So private companies shouldn’t get to determine who they provide services to? Assuming no extremely malicious intent, I’d be fine if they said it was only going to McDonalds because the founders like Big Macs.

McDonalds isn't a public benefit corporation.

Re: Project Glasswing: Securing critical software for the AI era

#392

Post by dwd »

TheAtomic wrote: post #131
redfloatplane wrote: post #77

Earlier quoted context omitted.

A thought experiment: It's April, 1991. Magically, some interface to Claude materialises in London. Do you think most people would think it was a sentient life form? How much do you think the interface matters - what if it looks like an android, or like a horse, or like a large bug, or a keyboard on wheels? I don't come down particularly hard on either side of the model sapience discussion, but I don't think dismissi…

Isn't this the premise of Garfield's Ex Machina?

The premise in Ex Machina was to see if Caleb developed an emotional attachment to Ava. We already see people getting an attachment, but no one is seriously thinking they have any rights.

I think the real moment is when we cross that uncanny valley, and the AI is able to elicit a response that it might receive if it was human. When the human questions whether they themselves could be an android.

Re: Project Glasswing: Securing critical software for the AI era

#393

Post by pizlonator »

baq wrote: post #236
pizlonator wrote: post #222

Earlier quoted context omitted.

> relatively calm geopolitics kinetically Relative to what? There's this trend in history that every hundred years there's a giant blow up, lots of violence, followed by peace. It's likely that we would have had 80 years of relative calm due to that cycle even if nukes hadn't happened

> Relative to what? to WW1 and WW2.

History tells us that we would have had calm after WW2 even without nukes

Re: Project Glasswing: Securing critical software for the AI era

#394

Post by pizlonator »

Flere-Imsaho wrote: post #192
pizlonator wrote: post #154

It's messed up that Anthropic simultaneously claims to be a public benefit copro and is also picking who gets to benefit from their newly enhanced cybersecurity capabilities. It means that the economic benefit is going to the existing industry heavyweights. (And no, the Linux Foundation being in the list doesn't imply broad benefit to OSS. Linux Foundation has an agenda and will pick who benefits according to what is…

If you're a maintainer, you can apply here: https://claude.com/contact-sales/claude-for-oss ... As mentioned in the article.

So a corpo gets to pick favorites.

Great.

Re: Project Glasswing: Securing critical software for the AI era

#395

Post by uselessTA »

temp123789246 wrote: post #207

OpenAI initially claimed that GPT-2 was too dangerous to release in 2019. How many times will labs repeat the same absurd propaganda?

The claim I remember was that releasing it would start an arms race for AGI, which I think it clearly did

Re: Project Glasswing: Securing critical software for the AI era

#396

Post by [deleted] »

redfloatplane wrote: post #156
copx wrote: post #123

Earlier quoted context omitted.

Interesting thought experiment. I would say, if you put Claude in an android body with voice recognition and TTS, people in 1991 would think they are interacting with a sentinent machine from outer space.

Thanks, I find it very interesting as well. I think very many people would assume they must be interacting with another person, and I don't think there's really a way to _prove_ it's not that, just through conversation. But we do have a lot of mechanisms for understanding how others think through conversation only, and so I think the approach of having a clinical psychiatrist interact with the model make sense.

[deleted]

Re: Project Glasswing: Securing critical software for the AI era

#397

Post by solenoid0937 »

meander_water wrote: post #305

I think this is a largely inflated PR stunt. Opus 4.6 was already capable of finding 0days and chaining together vulns to create exploits. See [0] and [1]. [0] https://www.csoonline.com/article/4153288/vim-and-gnu-emacs-... [1] https://xbow.com/blog/top-1-how-xbow-did-it

Absolutely not a PR stunt, talk to one of your friends working at partner companies with access to the model

Re: Project Glasswing: Securing critical software for the AI era

#398

Post by elboru »

redfloatplane wrote: post #156
copx wrote: post #123

Earlier quoted context omitted.

Interesting thought experiment. I would say, if you put Claude in an android body with voice recognition and TTS, people in 1991 would think they are interacting with a sentinent machine from outer space.

Thanks, I find it very interesting as well. I think very many people would assume they must be interacting with another person, and I don't think there's really a way to _prove_ it's not that, just through conversation. But we do have a lot of mechanisms for understanding how others think through conversation only, and so I think the approach of having a clinical psychiatrist interact with the model make sense.

There’s definitely a way to prove it, ask it to spell out a moderately complex program.

Re: Project Glasswing: Securing critical software for the AI era

#399

Post by LiamPowell »

> Mythos Preview identified a number of Linux kernel vulnerabilities that allow an adversary to write out-of-bounds (e.g., through a buffer overflow, use-after-free, or double-free vulnerability.) Many of these were remotely-triggerable. However, even after several thousand scans over the repository, because of the Linux kernel’s defense in depth measures Mythos Preview was unable to successfully exploit any of these.

Do they really need to include this garbage which is seemingly just designed for people to take the first sentence out of context? If there's no way to trigger a vulnerability then how is it a vulnerability? Is the following code vulnerable according to Mythos?

    if (x != null) {
        y = *x; // Vulnerability! X could be null!
    }
Is it really so difficult for them to talk about what they've actually achieved without smearing a layer of nonsense over every single blog post?

Edit: See my reply below for why I think Claude is likely to have generated nonsensical bug reports here: https://news.ycombinator.com/item?id=47683336

Re: Project Glasswing: Securing critical software for the AI era

#400

Post by pilgrim0 »

woeirua wrote: post #76
9cb14c1ec0 wrote: post #52

Now, its very possible that this is Anthropic marketing puffery, but even if it is half true it still represents an incredible advancement in hunting vulnerabilities. It will be interesting to see where this goes. If its actually this good, and Apple and Google apply it to their mobile OS codebases, it could wipe out the commercial spyware industry, forcing them to rely more on hacking humans rather than hacking mobi…

You should watch this talk by Nicholas Carlini (security researcher at Anthropic). Everything in the talk was done with Opus 4.6: https://www.youtube.com/watch?v=1sd26pWhfmg

Just a thought: The fact that the found kernel vulnerability went decades without a fix says nothing about the sophistication needed to find it. Just that nobody was looking. So it says nothing about the model’s capability. That LLMs can find vulnerabilities is a given and expected, considering they are trained on code. What worries me is the public buying the idea that it could in any way be a comprehensive security solution. Most likely outcome is that they’re as good at hacking as they’re at development: mediocre on average; untrustworthy at scale.
Post reply on HN