Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's open-source framework for AI-powered vulnerability discovery

github.com

Post reply on HN
Print view View original

71–80 of 177 posts

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#71

Post by chrisweekly »

tptacek wrote: post #54
bflesch wrote: post #16

Earlier quoted context omitted.

It's weird because why can't they train the AI to simply output secure code? The basic security flaws with regards to input validation and overflows should never ever be output by an AI. For "security flaws due to bad design" I'll cut them slack until AGI is achieved.

What's destabilizing the industry right now isn't vulnerabilities AI introduces into new code; it's a flood of sev:hi vulnerabilities in existing code, not introduced by AI but discovered by it.

Agreed -- and, compounding the challenge, the flood of _reported_ high-sev CVEs is itself a kind of DDoS attack on maintainers.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#72

Post by beering »

sofixa wrote: post #62
xerxes249 wrote: post #9

Earlier quoted context omitted.

In the Mythos blogpost they revealed to run the model like a 1000 times on the same code-base maybe with slightly different prompt or temperature. That suggests it will just be pay to win. If the 'attacker' spends more money/tokens than the 'defender' you will eventually be outclassed.

It's even worse, it's loot box style. Not pay to win, but pay to have the chance to win. The result will always be non-deterministic, so for some cases it can give you what you're looking for from the first time, or it can take 1000 tries.

It’s never not been “loot box style”. None of your past hired security audits were guaranteed to catch all issues?

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#74

Post by zuzululu »

Retr0id wrote: post #64
zuzululu wrote: post #57

Earlier quoted context omitted.

No idea why people are so upset I genuinely thought his references using analogy was a typical AI slop comment that I'm used to seeing from chatgpt

Believe it or not, people have been making analogies since before AI

I believe you and you can also believe AI is pretty good at it too.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#75

Post by DrewADesign »

energy123 wrote: post #22
DrewADesign wrote: post #20

Earlier quoted context omitted.

> They can only do that if they're a monopoly, which they're not Why do you say that? I reckon lots and lots of companies sell software that aren’t monopolies. Having competition, even stiff competition, isn’t anathema to running a business.

You said "They wouldn't be selling tokens directly ... They'd hoard them" But they can't do that because they aren't monopolies.

> You said

Just to clarify, I’m not the person you initially replied to.

> "They wouldn't be selling tokens directly ... They'd hoard them" But they can't do that because they aren't monopolies.

Hoarding them— not selling any of them, but instead using them internally and selling the products created by them — doesn’t at all seem like it would require a monopoly.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#76

Post by jaxn »

colmmacc wrote: post #66
redfloatplane wrote: post #46

Earlier quoted context omitted.

"Shop jigs" is a great way to put it. I think a lot of software has gone from being made for general use to extremely individualised use. Before the Age of AI, it took so much human effort to write something that solved your problem that you might often go the extra mile so that others could re-use it. Now, it takes almost no effort, so the software stays ungeneralised. Some of the incentive has changed, I think. Mos…

Unless it is very specific to a proprietary product, craftspeople take their jigs with them from job to job, building up a personal library over a career. As a software developer I've always had a well-tuned IDE and shell config in a safe place. Something I think about a lot is what is the equivalent for the software builders of today using AI tools? how do make these harnesses exportable and portable? You might thin…

i have been thinking about this from a different direction: how do we make these shared within a company in a way that increases the productivity floor of the team/department/company. Sure, they can still be extended/enhanced by individuals, but we don’t need everyone configuring mcps, building institutional memory, etc.

for me, it’s not about the cost to leave, it’s about lowering the cost of onboarding and change.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#78

Post by morpheos137 »

hyperpape wrote: post #38
richardbarosky wrote: post #8

To be sure, security is an amazing AI/LLM use case. A huge swath of the work is pattern matching known security issues against stuff that's very precise to analyze -- programming language text. Something that stands out is that for the strongest use cases, AI companies will prefer to sell the technique as a service rather than its raw output. For use cases where the output is less valuable, tokens are sold. If AI tok…

> If AI tokens were so magical in creating new value in developing software applications generally, they wouldn't be selling tokens directly. They'd hoard the tokens are use them to dominate SaaS software in any industry they want. This doesn't follow at all. Anthropic's revenue is growing 10x year over year selling tokens. Their tokens can be super magical, let them enter established industries and displace incumben…

kind of funny tokens don't prompt and steer themselves. it almost as if the value still lies with the human holding the tool.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#79

Post by lateral_cloud »

majicDave wrote: post #73

It will always be easier to find a single hole than it will be to seal every one. The hackers have all the same tools, so this is an arms race that cannot be won.

Defenders have context that attackers don't though.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#80

Post by worldsayshi »

colmmacc wrote: post #66
redfloatplane wrote: post #46

Earlier quoted context omitted.

"Shop jigs" is a great way to put it. I think a lot of software has gone from being made for general use to extremely individualised use. Before the Age of AI, it took so much human effort to write something that solved your problem that you might often go the extra mile so that others could re-use it. Now, it takes almost no effort, so the software stays ungeneralised. Some of the incentive has changed, I think. Mos…

Unless it is very specific to a proprietary product, craftspeople take their jigs with them from job to job, building up a personal library over a career. As a software developer I've always had a well-tuned IDE and shell config in a safe place. Something I think about a lot is what is the equivalent for the software builders of today using AI tools? how do make these harnesses exportable and portable? You might thin…

> craftspeople take their jigs with them from job to job

Except for software gigs the software typically belongs to the customer so you'd need to rewrite it every time...

Post reply on HN