Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's open-source framework for AI-powered vulnerability discovery

github.com

Post reply on HN
Print view View original

131–140 of 177 posts

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#131

Post by iammrpayments »

tptacek wrote: post #54
bflesch wrote: post #16

Earlier quoted context omitted.

It's weird because why can't they train the AI to simply output secure code? The basic security flaws with regards to input validation and overflows should never ever be output by an AI. For "security flaws due to bad design" I'll cut them slack until AGI is achieved.

What's destabilizing the industry right now isn't vulnerabilities AI introduces into new code; it's a flood of sev:hi vulnerabilities in existing code, not introduced by AI but discovered by it.

[flagged]

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#132

Post by nbardy »

tptacek wrote: post #18

The thing about things like this is that they're shop jigs. You can buy a crosscut sled if you really want to, but most woodworkers just make their own. It was a different situation 2 years ago, when there was significant cost to building your own harness (but then: you probably weren't doing AI vuln research 2 years ago). Today, I think your best bet is to look at something like this for ideas, and then just ask for…

In general this is the way I see open source going.

We won't reuse open source libraries as libraries we import, but as design inspiration for the bespoke tools we make.

It's too cheap to make your own stuff and too expensive to be stuck with someone else primitives.

But grounding AI Coding in existing tools is incredibly powerful.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#133

Post by kolesnikov-arch »

simonw wrote: post #5

I wonder how much this thing costs to run. https://github.com/anthropics/defending-code-reference-harne... says: > As a rough guideline, expect ~10K uncached input tokens/min and ~2K output tokens/min per agent. You can scale parallelism up to your account's ITPM limit (roughly 10 agents per 100K ITPM). My guess would be hundreds of dollars with Opus and thousands of dollars with Mythos.

[flagged]

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#134

Post by ClikeX »

djfergus wrote: post #119
aquajet wrote: post #83

Earlier quoted context omitted.

Using something like pi helps. I've made my own dotfiles for skills/extensions I like and can install them just like my normal dotfiles https://github.com/anishthite/agent-dotfiles

"Humor When you finish a job — completing a task, answering a question, fixing a bug, shipping a feature — end your final message with one short funny line. A quip, a dad joke, a wry observation, a playful self-roast. One line. No emoji spam. Make it land, then shut up." whats the purpose of this? just fun or does it cause some desired behaviour?

> does it cause some desired behaviour?

Fun is desirable.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#135

Post by ClikeX »

worldsayshi wrote: post #80
colmmacc wrote: post #66

Earlier quoted context omitted.

Unless it is very specific to a proprietary product, craftspeople take their jigs with them from job to job, building up a personal library over a career. As a software developer I've always had a well-tuned IDE and shell config in a safe place. Something I think about a lot is what is the equivalent for the software builders of today using AI tools? how do make these harnesses exportable and portable? You might thin…

> craftspeople take their jigs with them from job to job Except for software gigs the software typically belongs to the customer so you'd need to rewrite it every time...

Depends. With all the web agencies I've made, the only code that belonged to customers was the actual website part. Any of the "jigs" that we made for our workflow was not part of that.

And contractually, any code I made was my employer's if I made it during office hours. Some even made a claim for code I would've written that during my employ that would be "competitive". Luckily, there was a massive difference in what I would do in my own time versus what they did.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#136

Post by sam-cop-vimes »

nikcub wrote: post #7
simonw wrote: post #5

I wonder how much this thing costs to run. https://github.com/anthropics/defending-code-reference-harne... says: > As a rough guideline, expect ~10K uncached input tokens/min and ~2K output tokens/min per agent. You can scale parallelism up to your account's ITPM limit (roughly 10 agents per 100K ITPM). My guess would be hundreds of dollars with Opus and thousands of dollars with Mythos.

It's becoming apparent that it requires more tokens to secure code than it does to write it May even be an order of magnitude more

Are AI firms going to charge us to write code, and then charge us even more to secure it?!

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#137

Post by smt88 »

sam-cop-vimes wrote: post #136
nikcub wrote: post #7

Earlier quoted context omitted.

It's becoming apparent that it requires more tokens to secure code than it does to write it May even be an order of magnitude more

Are AI firms going to charge us to write code, and then charge us even more to secure it?!

Yes, obviously. Infosec has always been plagued by this. How many services make you pay for SSO?

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#139

Post by sumedh »

edot wrote: post #109
ElijahLynn wrote: post #102

Anthropics vs Anthropic. That repo is Anthropics. This post title should clarify that it is not Anthropic (no "s").

Anthropic, no s, is owned by some Australian guy.

I wonder if he is using Anthropic's claude code to work on his Anthropic Github account.

Re: Anthropic's open-source framework for AI-powered vulnerability discovery

#140

Post by bflesch »

tptacek wrote: post #54
bflesch wrote: post #16

Earlier quoted context omitted.

It's weird because why can't they train the AI to simply output secure code? The basic security flaws with regards to input validation and overflows should never ever be output by an AI. For "security flaws due to bad design" I'll cut them slack until AGI is achieved.

What's destabilizing the industry right now isn't vulnerabilities AI introduces into new code; it's a flood of sev:hi vulnerabilities in existing code, not introduced by AI but discovered by it.

Even before that everybody was getting drowned in shitty reports from automated tools.

The goal of AI-generated code should not be that one needs a AI-based security review tool on top of it, but that the AI-generated code in itself is reasonably secure.

Post reply on HN