Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Glasswing: An Initial Update

anthropic.com

Post reply on HN
Print view View original

281–290 of 345 posts

Re: Project Glasswing: An Initial Update

#281

Post by frumiousirc »

_heimdall wrote: post #276
ZrArm wrote: post #147

Earlier quoted context omitted.

> Maybe curl is significantly better hardened than most projects? Meanwhile from [1]: "Not even half-way through this #curl release cycle we are already at 11 confirmed vulnerabilities - and there are three left in the queue to assess and new reports keep arriving at a pace of more than one/day." "The simple reason is: the (AI powered) tools are this good now. And people use these tools against curl source code.They…

Based on the article here, and Firefox's mythos article, they had found bugs with Opus 4.6 as well but mythos is finding more that it missed. That would align with the curl feedback you linked, they aren't using mythos but are finding bugs with other models. Presumably the expectation would be that with mythos they'd find more that were missed by other models already used.

> Based on the article here, and Firefox's mythos article, they had found bugs with Opus 4.6 as well but mythos is finding more that it missed.

It's not quite apples-to-apples. It was Opus on Firefox 148, Mythos on 150. A better test of Mythos vs Opus would have been to apply Mythos to Firefox 148. Or also re-apply Opus to Firefox 150.

Do we know all the Opus+Firefox 148 bugs are fixed in Firefox 150? Do we know the number of new bugs introduced per Firefox release?

Re: Project Glasswing: An Initial Update

#283

Post by junon »

rco8786 wrote: post #275
winstonwinston wrote: post #54

Earlier quoted context omitted.

> I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO. So, how is that supposed to work? Claude Code generates security bugs, then Claude Security finds them, then Claude Code generate fix, spend tokens, profit?

Software engineers generate security bugs, Software engineers find them, then Software engineers generate fix, collect salary, profit?

Those are individual revenue streams, distributed at a very granular level across the world.

LLMs are currently relegated to individual for-profit companies. They collect that money. There's no other choice to use them and to provide them that money.

Re: Project Glasswing: An Initial Update

#284

Post by junon »

nullbio wrote: post #246
mdeeks wrote: post #230

Earlier quoted context omitted.

Two years? That exists right now. You only have to point Codex Security at an open source repo. There are a lot of tools and companies that are spinning up today that do autonomous pentesting. I'm not even sure a specialized model is needed here. It probably just needs the right harness around existing ones. I expect the next two years to be absolutely brutal for hacks. Attackers have supercharged tools in their hand…

> The major short term downside is that open source or personal projects won't be able to afford things like Codex Security. Realistically, all open-source projects should be forced to have automated scans of this nature before their releases can be shipped. This is something the package managers and github need to figure out. It'd stop the supply chain attacks too.

> all open-source projects should be forced

That's a great way to kill OSS. This is only bootlicking the idea of corporations profiting off of unpaid labor.

Re: Project Glasswing: An Initial Update

#285

Post by junon »

hqt124 wrote: post #212
tptacek wrote: post #197

Earlier quoted context omitted.

You are linking to a Wikipedia page in which I am literally cited (I presented a hypervisor malware detection scheme at the Black Hat conference where Joanna Rutkowska presented this; it was a whole thing). I'm telling you that the term makes no sense in this thread. I think you meant to use a different term.

[flagged]

Stop abusing the system with new accounts. You're not cool like that.

Re: Project Glasswing: An Initial Update

#286

Post by nikcub »

criemen wrote: post #156
nikcub wrote: post #50

There has been a lot of cynicism around mythos, that it's just the usual public models without guardrails, etc. etc. but this: > 1,752 of those high- or critical-rated vulnerabilities have now been carefully assessed by one of six independent security research firms, or in a small number of cases by ourselves. Of these, 90.6% (1,587) have proved to be valid true positives, and 62.4% (1,094) were confirmed as either h…

> There's also a runaway effect of model improvement from the discovery, triage and fix data. This is likely already the most potent corpus of curated offensive data ever assembled and will only get better. But that corpus of data is accessible to all competitors, American or not. I don't believe that this can't be replicated. I'd posit that there's enough annotated data out there (CVE+patch), only increasing thanks…

the CVE + patch data has been built into models for a few generations now. I actually thought the bug bounty companies were well positioned here, but they've been overtaken.

Mythos is a better hacker than we ever were

Re: Project Glasswing: An Initial Update

#287

Post by designerarvid »

winstonwinston wrote: post #54
mdeeks wrote: post #34

You can get a taste of this today yourself with Codex Security. I turned it on just as an experiment and in less than a week it has now become essential to all of us. I was shocked how accurate it is, how many security issues it found in existing code, how it continually finds them as we commit, and how NO ONE is immune from making these mistakes. I'd say it is about 90% accurate for us. Often even the "Low" findings…

> I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO. So, how is that supposed to work? Claude Code generates security bugs, then Claude Security finds them, then Claude Code generate fix, spend tokens, profit?

Just refactor and rebrand all of it as Claude Code and see it as one process.

Re: Project Glasswing: An Initial Update

#288

Post by qerahb »

nullbio wrote: post #246
mdeeks wrote: post #230

Earlier quoted context omitted.

Two years? That exists right now. You only have to point Codex Security at an open source repo. There are a lot of tools and companies that are spinning up today that do autonomous pentesting. I'm not even sure a specialized model is needed here. It probably just needs the right harness around existing ones. I expect the next two years to be absolutely brutal for hacks. Attackers have supercharged tools in their hand…

> The major short term downside is that open source or personal projects won't be able to afford things like Codex Security. Realistically, all open-source projects should be forced to have automated scans of this nature before their releases can be shipped. This is something the package managers and github need to figure out. It'd stop the supply chain attacks too.

So first they steal all code and launder it without attribution. Then they release a tool that doesn't find anything in hardened projects and is marketed through secrecy and modern equivalents of Netcraft like this British AI institute.

Then open source projects need a McKinsey-like stamp of approval to even be released.

Sounds like there are many parasites in this process.

You know that open source users are free to scan everything if they want to?

Re: Project Glasswing: An Initial Update

#289

Post by kingforaday »

cassianoleal wrote: post #176
tomjakubowski wrote: post #130

Earlier quoted context omitted.

I wonder this too. I prompted Opus 4.7 to generate some Python threading code for me. The code to run the sub-thread looked like this: def run(): with contextlib.suppress(SystemExit): do_thread_thing() threading.Thread(target=run, daemon=True).start() Suppressing SystemExit was surprising, and made me curious. I followed up and asked the model: what's the purpose of that? The model's response: "Honestly? Cargo-cultin…

I had some shell scripts littered with `|| true`, which was obviously obscuring real errors everywhere. When I challenged the model, it gave me the same "cargo-culting" answer.

From your statement and the parent comment, just learned that "cargo cult" is a thing, but cargo-culting as a compound is something AI has made up? [1].

As I was educating myself, I found Richard Feynman's Commencement Speech at Caltech in '74 [2] that might have coined this for our industry? If you would rather listen than read [3]. Posting this for others curious on the term.

1. https://trends.google.com/trends/explore?q=Cargo-culting&hl=...

2. https://calteches.library.caltech.edu/51/2/CargoCult.htm

3. https://www.youtube.com/watch?v=yvfAtIJbatg

Re: Project Glasswing: An Initial Update

#290

Post by TeMPOraL »

ashdksnndck wrote: post #196
amazingamazing wrote: post #188

Earlier quoted context omitted.

What are you talking about?

I’ll try to steelman this comment. Anyone who uses coding tools knows that the output is heavily affected by details of the task you give it. The same model can give you garbage code or genius code for the same problem with slightly different framing. So it’s not necessarily a limitation in the model’s training that causes it to output security bugs. The model might be great at writing secure code, but you need a dif…

Counter-counter-argument: for LLMs, tokens are units of thinking. And token use is, on the margin, directly proportional to costs of inference. So while the details of the harness, and how you prompt the model, and nature of the code and docs you put in context, etc. all matter to the quality of output you get from LLM coding tool, ultimately, there's always a ceiling to how much you're willing to spend on solving a problem - say, no more than 30 minutes, or $10, on refactoring a target module or implementing a small feature - and that puts a limit on how much thinking the model can put into it.

Thing is, writing secure and efficient and readable and simple code is in many cases fundamentally over that limit. It's possible, but you can't afford (or rationally just don't want) to spend as much on it as it's required for superhuman quality on all these aspects. Also most of the time, you don't want to operate at a limit - you probably expected that feature to take 30 seconds and less than $1 to implement. So you choose, both what the model optimizes for, and how much.

Because of that, no matter how good the model and the harness and the prompting are, $10 spent on coding is still bound to leave behind some security vulnerabilities that subsequent $10 spent on security review will find (especially with a model post-trained for that, at expense of general performance).

Post reply on HN