Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's open-source framework for AI-powered vulnerability discovery

github.com

Post reply on HN
Print view View original

91–100 of 177 posts

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#91

Post by tptacek »

colmmacc wrote: post #66
redfloatplane wrote: post #46

Earlier quoted context omitted.

"Shop jigs" is a great way to put it. I think a lot of software has gone from being made for general use to extremely individualised use. Before the Age of AI, it took so much human effort to write something that solved your problem that you might often go the extra mile so that others could re-use it. Now, it takes almost no effort, so the software stays ungeneralised. Some of the incentive has changed, I think. Mos…

Unless it is very specific to a proprietary product, craftspeople take their jigs with them from job to job, building up a personal library over a career. As a software developer I've always had a well-tuned IDE and shell config in a safe place. Something I think about a lot is what is the equivalent for the software builders of today using AI tools? how do make these harnesses exportable and portable? You might thin…

Just nerding out here, not rebutting, but when you say "craftspeople take their jigs with them from job to job" --- sort of. Sometimes. I think if you put a woodworker in a position where they obliged to build a new miter sled or assembly table, they might actually be thrilled. You make a tool, you use it for awhile, you build up a mental list of things you'd like to improve about it, that you'd do differently if you got a do-over; now you have an excuse to do it.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#92

Post by therealdrag0 »

richardbarosky wrote: post #8

To be sure, security is an amazing AI/LLM use case. A huge swath of the work is pattern matching known security issues against stuff that's very precise to analyze -- programming language text. Something that stands out is that for the strongest use cases, AI companies will prefer to sell the technique as a service rather than its raw output. For use cases where the output is less valuable, tokens are sold. If AI tok…

Isn’t this analogous to saying if farming equipment is so productive why doesn’t John Deer hoard all the tractors and do the farming themselves?

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#93

Post by cpard »

It’s clear that Anthropic is building harnesses for specific use cases now and turns them into products.

This is the equivalent of Claude Design but for security.

Different harness, different packaging and obviously different distribution because the persona is different.

It’s funny because from all the posts I’ve read from companies reporting on Mythos, everyone is building their own harness for it.

Cisco even published a specification for one.

But Anthropic is the one who has figured out how to package and distribute this. Great GTM!

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#95

Post by hsaliak »

tptacek wrote: post #18

The thing about things like this is that they're shop jigs. You can buy a crosscut sled if you really want to, but most woodworkers just make their own. It was a different situation 2 years ago, when there was significant cost to building your own harness (but then: you probably weren't doing AI vuln research 2 years ago). Today, I think your best bet is to look at something like this for ideas, and then just ask for…

I’ve been looking for a way to articulate this shift, and your analogy nails it. The value of libraries and infrastructure components in software engineering is eroding fast.

I am sure that in many organizations, teams responsible for this sort of work have less and less users coming to them.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#96

Post by eranation »

simonw wrote: post #5

I wonder how much this thing costs to run. https://github.com/anthropics/defending-code-reference-harne... says: > As a rough guideline, expect ~10K uncached input tokens/min and ~2K output tokens/min per agent. You can scale parallelism up to your account's ITPM limit (roughly 10 agents per 100K ITPM). My guess would be hundreds of dollars with Opus and thousands of dollars with Mythos.

We actually created a calculator to estimate scanning costs (including whether you do this continuously or not) https://ai-cost-calculator.arnica.io

It's an estimate, so it might be wrong, but it gives the ballpark based on our experience. Happy to hear everyone's feedback.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#97

Post by niros_valtos »

simonw wrote: post #5

I wonder how much this thing costs to run. https://github.com/anthropics/defending-code-reference-harne... says: > As a rough guideline, expect ~10K uncached input tokens/min and ~2K output tokens/min per agent. You can scale parallelism up to your account's ITPM limit (roughly 10 agents per 100K ITPM). My guess would be hundreds of dollars with Opus and thousands of dollars with Mythos.

I think that the cost of Opus is already prohibitively expensive, so not sure how that would compare to Mythos. Check this calculator- it shows that a company with 100 devs can hit ~2.5M cost on tokens annually, which is wild! https://ai-cost-calculator.arnica.io

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#98

Post by eranation »

If anyone wonders how much it can cost to run scans like this on your entire codebase with SOTA models: https://ai-cost-calculator.arnica.io

tl;dr - not that it's surprising, but it's not cheap, especially if you want to do this continuously.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#99

Post by happyopossum »

tptacek wrote: post #91
colmmacc wrote: post #66

Earlier quoted context omitted.

Unless it is very specific to a proprietary product, craftspeople take their jigs with them from job to job, building up a personal library over a career. As a software developer I've always had a well-tuned IDE and shell config in a safe place. Something I think about a lot is what is the equivalent for the software builders of today using AI tools? how do make these harnesses exportable and portable? You might thin…

Just nerding out here, not rebutting, but when you say "craftspeople take their jigs with them from job to job" --- sort of. Sometimes. I think if you put a woodworker in a position where they obliged to build a new miter sled or assembly table, they might actually be thrilled. You make a tool, you use it for awhile, you build up a mental list of things you'd like to improve about it, that you'd do differently if you…

This, for like 37 things in my workshop right now.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#100

Post by DrewADesign »

napoleond wrote: post #81
majicDave wrote: post #73

It will always be easier to find a single hole than it will be to seal every one. The hackers have all the same tools, so this is an arms race that cannot be won.

It seems clear that LLMs significantly change threat model math, but this observation alone does not explain how or why; the asymmetry that you’re describing is a property of pre-LLM software as well.

Same ratio of imbalance, just with matching multipliers distributed to each side, and everybody is probably worse off because of it: I cite post-LLM-ATS hiring/job hunting.
Post reply on HN