Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's open-source framework for AI-powered vulnerability discovery

github.com

Post reply on HN
Print view View original

121–130 of 177 posts

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#121

Post by flir »

tptacek wrote: post #111
hsaliak wrote: post #95

Earlier quoted context omitted.

I’ve been looking for a way to articulate this shift, and your analogy nails it. The value of libraries and infrastructure components in software engineering is eroding fast. I am sure that in many organizations, teams responsible for this sort of work have less and less users coming to them.

Maybe for developer tooling, but on the consumer app side I think it's the opposite: MusicKit is much more valuable than Music.app now, because Claude can one-shot most reasonable things you could ask it to do. I think there's actually more value in ambitious libraries than there was 5 years ago, when any serious use of a library entailed a minimum 5-figure investment of time.

I had a pleasant experience one-shotting a dashboard on top of a library designed for building dashboards. Because everything was abstracted away, the chatbot had relatively few places it could get into the weeds. If I'd asked for the same thing from scratch, I think the result would have been more inconsistent, and would have had more bugs.

So I can definitely see the value in a library for constraining the chatbot to some well-worn paths.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#124

Post by Zetaphor »

ElijahLynn wrote: post #101
cpard wrote: post #93

It’s clear that Anthropic is building harnesses for specific use cases now and turns them into products. This is the equivalent of Claude Design but for security. Different harness, different packaging and obviously different distribution because the persona is different. It’s funny because from all the posts I’ve read from companies reporting on Mythos, everyone is building their own harness for it. Cisco even publi…

This post is misleading and so is the GitHub org. Anthropics vs Anthropic.

That is their actual account. We have this discussion every time they post something sadly

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#125

Post by [deleted] »

simonw wrote: post #29
bflesch wrote: post #16

Earlier quoted context omitted.

It's weird because why can't they train the AI to simply output secure code? The basic security flaws with regards to input validation and overflows should never ever be output by an AI. For "security flaws due to bad design" I'll cut them slack until AGI is achieved.

> It's weird because why can't they train the AI to simply output secure code? The most interesting security bugs have causes that are spread across large codebases, or networks of dependencies. Training the AI to "output secure code" won't work if it doesn't also have access to the source code of every dependency that it's using... and even then, given current model speeds and prices most developers won't want to wa…

[deleted]

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#126

Post by Npovview »

ashdksnndck wrote: post #116
tptacek wrote: post #18

The thing about things like this is that they're shop jigs. You can buy a crosscut sled if you really want to, but most woodworkers just make their own. It was a different situation 2 years ago, when there was significant cost to building your own harness (but then: you probably weren't doing AI vuln research 2 years ago). Today, I think your best bet is to look at something like this for ideas, and then just ask for…

Sure it’s possible for anyone to build a harness if they had the inclination, but most people don’t have the inclination to do that. And even if you did… I spent months refining AI workflows that were just obsoleted by ultracode.

Just as Python is batteries included language, we similarly need batteries included harnesses as well. This is what I don't like minimalism setups like Pi.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#128

Post by dgellow »

kenjackson wrote: post #118
dgellow wrote: post #14

Earlier quoted context omitted.

> The same way as someone selling an expensive course in the stock market is signaling that they have more to gain by selling the course rather than Or they want to diversify > If AI tokens were so magical in creating new value in developing software applications generally, they wouldn't be selling tokens directly. That requires to build and sell a whole product they have little experience with, competing with their…

What market is hotter than AI models? Do you think their energy would be better making games or image editing software?

No, I’m saying the opposite

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#129

Post by andai »

nikcub wrote: post #7
simonw wrote: post #5

I wonder how much this thing costs to run. https://github.com/anthropics/defending-code-reference-harne... says: > As a rough guideline, expect ~10K uncached input tokens/min and ~2K output tokens/min per agent. You can scale parallelism up to your account's ITPM limit (roughly 10 agents per 100K ITPM). My guess would be hundreds of dollars with Opus and thousands of dollars with Mythos.

It's becoming apparent that it requires more tokens to secure code than it does to write it May even be an order of magnitude more

There's a parallel between looking for bugs and mining. As models get smarter, they'll find "deeper bugs".

I expect at some point formal verification will become more economical than red teaming. Writing it correctly is more expensive, but it may be cheaper than trying to secure incorrect software.

(Or rather, as hacking incorrect software becomes vastly cheaper, the amount of software worth writing properly will increase.)

I've been thinking, by Dijkstra's standards we have already been vibe coding for almost a century :)

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#130

Post by beezlewax »

redfloatplane wrote: post #46
tptacek wrote: post #18

The thing about things like this is that they're shop jigs. You can buy a crosscut sled if you really want to, but most woodworkers just make their own. It was a different situation 2 years ago, when there was significant cost to building your own harness (but then: you probably weren't doing AI vuln research 2 years ago). Today, I think your best bet is to look at something like this for ideas, and then just ask for…

"Shop jigs" is a great way to put it. I think a lot of software has gone from being made for general use to extremely individualised use. Before the Age of AI, it took so much human effort to write something that solved your problem that you might often go the extra mile so that others could re-use it. Now, it takes almost no effort, so the software stays ungeneralised. Some of the incentive has changed, I think. Mos…

No effort? You are really drinking the AI marketing soup with that one.

"It takes less effort for some parts of the software development life cycle" would be more correct.

Post reply on HN