Earlier quoted context omitted.
This post is misleading and so is the GitHub org. Anthropics vs Anthropic.
That is their actual account. We have this discussion every time they post something sadly
Oh, bummer. That is really confusing.
Anthropic's open-source framework for AI-powered vulnerability discovery
161–170 of 177 posts
Re: Anthropic's open-source framework for AI-powered vulnerability discovery
#162Re: Anthropic's open-source framework for AI-powered vulnerability discovery
#163I wonder how much this thing costs to run. https://github.com/anthropics/defending-code-reference-harne... says: > As a rough guideline, expect ~10K uncached input tokens/min and ~2K output tokens/min per agent. You can scale parallelism up to your account's ITPM limit (roughly 10 agents per 100K ITPM). My guess would be hundreds of dollars with Opus and thousands of dollars with Mythos.
It's becoming apparent that it requires more tokens to secure code than it does to write it May even be an order of magnitude more
[deleted]
Re: Anthropic's open-source framework for AI-powered vulnerability discovery
#164They seem to be using this to advertise their "Claude Security" product which promises to find vulnerabilities in your software. This makes for a somewhat amusing set of product offerings given that according to Dario 90% of all software is being AI generated. Maybe next they can sell something to find the bugs in the security scanner ?
> Maybe next they can sell something to find the bugs in the security scanner ?
So, tokens are used to produce sloppy code, and then this thing uses more tokens to fix vulnerabilities in the slop ? Whats not to like in this business model ? Similar to microsoft's. Create an OS which is vulnerable, and then enable business models for anti-virus software. Everyone wins.
More seriously, linters are turned off in ci because the amount of time spent chasing false-positives is prohibitive.
Re: Anthropic's open-source framework for AI-powered vulnerability discovery
#165Earlier quoted context omitted.
Again: it depends. It is all about how the contract is written.
I never seen any other kind of contract, on my 50ys.
I'm curious how does it work, you handover the tools you wrote, .bashrc/.zshrc, etc?
When I'm hired in a company (not contract), they wipe the harddrive when I leave (well, I also do it before I hand it over sometimes). So they don't get the tools (I take them with myself, it would be a waste to loose them)
Re: Anthropic's open-source framework for AI-powered vulnerability discovery
#166The thing about things like this is that they're shop jigs. You can buy a crosscut sled if you really want to, but most woodworkers just make their own. It was a different situation 2 years ago, when there was significant cost to building your own harness (but then: you probably weren't doing AI vuln research 2 years ago). Today, I think your best bet is to look at something like this for ideas, and then just ask for…
100% concur and if you dig into any of these tools they are all frameworks and wrappers with prompt injections
Re: Anthropic's open-source framework for AI-powered vulnerability discovery
#167Anthropic realized security and safety are their main value prop compared to the competition. Either mythos or anything else since seem purpose built to streamline the messaging. It’s good, am not complaining, but i wonder how much this is intended to showcase what Claude can do over using it as is
Re: Anthropic's open-source framework for AI-powered vulnerability discovery
#168Ran this last night and it correctly identified a sql injection that could allow cross tenant data access via snowflake. It burnt A LOT of tokens to get there.
Like others I suspect this is exactly what they are going to paywall with product features going forward.
Re: Anthropic's open-source framework for AI-powered vulnerability discovery
#169I wonder how much this thing costs to run. https://github.com/anthropics/defending-code-reference-harne... says: > As a rough guideline, expect ~10K uncached input tokens/min and ~2K output tokens/min per agent. You can scale parallelism up to your account's ITPM limit (roughly 10 agents per 100K ITPM). My guess would be hundreds of dollars with Opus and thousands of dollars with Mythos.
I think that the cost of Opus is already prohibitively expensive, so not sure how that would compare to Mythos. Check this calculator- it shows that a company with 100 devs can hit ~2.5M cost on tokens annually, which is wild! https://ai-cost-calculator.arnica.io
A 100 dev team is going to cost on the order of $25m a year (keep in mind cost is not just their salary, but also the HR/Management org to support a team of that size, benefits, office space, hardware/software). So if you think you get a 10% boost in productivity out of Opus, its not prohibitive at all.
Re: Anthropic's open-source framework for AI-powered vulnerability discovery
#170[dead]