Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Glasswing: Securing critical software for the AI era

anthropic.com

Post reply on HN
Print view View original

561–570 of 921 posts

Re: Project Glasswing: Securing critical software for the AI era

#561

Post by ofjcihen »

dota_fanatic wrote: post #544
ofjcihen wrote: post #535

Earlier quoted context omitted.

With the right prompting (mostly creating a narrative that justifies the subject matter as okay to perform) other models have already been doing this for me though. That’s another confusing bit for me about how this is portrayed and I refuse to believe I’m a revolutionary user right? I mean I’m sitting on $10k worth of bug payouts right now partially because that was already a thing.

> Non-experts can also leverage Mythos Preview to find and exploit sophisticated vulnerabilities. Engineers at Anthropic with no formal security training have asked Mythos Preview to find remote code execution vulnerabilities overnight, and woken up the following morning to a complete, working exploit. In other cases, we’ve had researchers develop scaffolds that allow Mythos Preview to turn vulnerabilities into explo…

I mean yeah. I’ve had these successes without scaffolding or really anything past Claude CLI and a small prompt as well?

Re: Project Glasswing: Securing critical software for the AI era

#562

Post by literalAardvark »

junon wrote: post #553
jillesvangurp wrote: post #549

Earlier quoted context omitted.

> I would honestly go so far as to say the overhype is detrimental to actual measured adoption. I think you are a bit dishonest about how objectively you are measuring. From where I'm sitting, I don't know a lot of developers that still artisanally code like they did a few years ago. The question is no longer if they are using AI for coding but how much they are still coding manually. I myself barely use IDEs at this…

> From where I'm sitting, I don't know a lot of developers that still artisanally code like they did a few years ago. You don't know a lot of developers then.

I do. The good ones use AI.

Re: Project Glasswing: Securing critical software for the AI era

#563

Post by sam0x17 »

It's all just really genius marketing. In 6 months Mythos will be nothing special, but right now everyone is being manipulated into fearing its release, as a marketing ploy.

This is the same reason AI founders perennially worry in public that they have created AGI...

Re: Project Glasswing: Securing critical software for the AI era

#564

Post by baq »

Gigachad wrote: post #333
SpicyLemonZest wrote: post #279

Earlier quoted context omitted.

The transition is pretty complete at e.g. Google and Meta, IIUC. Definitely whoever builds the AI tools you're using every day isn't writing code by hand.

I really just don't believe it. I have not met anyone in tech who writes zero code now. The idea that no one at Google writes any code is such a huge claim it requires extraordinary evidence. Which none ever gets presented.

Anecdotally me and my colleagues haven’t written a substantial line of code since January and this isn’t a mag7; I would be very surprised if mag7 were writing anything by hand unless it’s a custom DSL.

Re: Project Glasswing: Securing critical software for the AI era

#565

Post by nearbuy »

vasco wrote: post #525
stevenhuang wrote: post #345

Earlier quoted context omitted.

Oh I enjoyed the Sign Painter short story it wrote. --- Teodor painted signs for forty years in the same shop on Vell Street, and for thirty-nine of them he was angry about it. Not at the work. He loved the work — the long pull of a brush loaded just right, the way a good black sat on primed board like it had always been there. What made him angry was the customers. They had no eye. A man would come in wanting COFFEE…

It's very good but it's also recycled Ayn Rand, the Fountainhead.

There is a similar theme in both of an artistic person not wanting to compromise their vision to suit common tastes. But this goes in a completely different direction than Rand.

Re: Project Glasswing: Securing critical software for the AI era

#566

Post by ofjcihen »

jillesvangurp wrote: post #549
ofjcihen wrote: post #508

I’m sure the new model is a step above the old one but I can’t be the only person who’s getting tired of hearing about how every new iteration is going to spell doom/be a paradigm shift/change the entire tech industry etc. I would honestly go so far as to say the overhype is detrimental to actual measured adoption.

> I would honestly go so far as to say the overhype is detrimental to actual measured adoption. I think you are a bit dishonest about how objectively you are measuring. From where I'm sitting, I don't know a lot of developers that still artisanally code like they did a few years ago. The question is no longer if they are using AI for coding but how much they are still coding manually. I myself barely use IDEs at this…

> I think you are a bit dishonest about how objectively you are measuring

As someone who has made a sizable amount of money in security research while using Claude you might be right but not in the way you think.

Re: Project Glasswing: Securing critical software for the AI era

#568

Post by array_key_first »

userbinator wrote: post #498
QuiEgo wrote: post #450

Earlier quoted context omitted.

Well yeah, it’s a toy example to illustrate a point in an HN discussion :). Imagine “silly mistake” is a parameter, and rename it “error_code” (pass by reference), put a label named “cleanup” right before the if statement, and throw in a ton of “goto cleanup” statements to the point the control flow of the function is hard to follow if you want it to model real code ever so slightly more. It will be interesting to se…

That's what I'm saying; a static analyser will be able to determine whether the code and/or state is reachable without any AI, and it will be completely deterministic in its output.

You cannot tell if code is actually reachable if it depends on runtime input.

Those really evil bugs are the ones that exist in code paths that only trigger 0.001% of the time.

Often, the code path is not triggerable at all with regular input. But with malicious input, it is, so you can only find it through fuzzing or human analysis.

Re: Project Glasswing: Securing critical software for the AI era

#569

Post by 4ndrewl »

qnleigh wrote: post #526
ofjcihen wrote: post #508

I’m sure the new model is a step above the old one but I can’t be the only person who’s getting tired of hearing about how every new iteration is going to spell doom/be a paradigm shift/change the entire tech industry etc. I would honestly go so far as to say the overhype is detrimental to actual measured adoption.

There is plenty of overhyping, no one denies that. But the antidote is not to dismiss everything. Ignore the words and look at the data. In this case, I see a pretty strong case that this will significantly change computer security. They provide plenty of evidence that the models can create exploits autonomously, meaning that the cost of finding valuable security breaches will plummet once they're widely available.

Is there any actual independent data though, or verification of any of these claims?

As it stands this is just a marketing programme for all involved.

Re: Project Glasswing: Securing critical software for the AI era

#570

Post by NickJLange »

qingcharles wrote: post #217
ollin wrote: post #166

Earlier quoted context omitted.

- The OpenBSD one is 'TCP packets with invalid SACK options could crash the kernel' https://cdn.openbsd.org/pub/OpenBSD/patches/7.8/common/025_s... - One (patched) Linux kernel bug is 'UaF when sys_futex_requeue() is used with different flags' https://github.com/torvalds/linux/commit/e2f78c7ec1655fedd94... These links are from the more-detailed 'Assessing Claude Mythos Preview’s cybersecurity capabilities' post relea…

That OpenBSD one is exactly the kind of bug that easily slips past a human. Especially as the code worked perfectly under regular circumstances. Looks like they've been approaching folks with their findings for at least a few weeks before this article.

While not entirely unrelated, Linux also had a remote SACK issue ~ 6 years back.

So if this Mythos is just an expensive combination of better RL and the original source material, that should hopefully point out where we might see an uptick in work ( as opposed to a novel class of attack vectors).

Post reply on HN