Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Glasswing: An Initial Update

anthropic.com

Post reply on HN
Print view View original

331–340 of 345 posts

Re: Project Glasswing: An Initial Update

#331

Post by Stubbs »

sandeepkd wrote: post #87
ygjb wrote: post #67

Earlier quoted context omitted.

Yeah, with a budget assigned. This is actually just software development and security right? Developers create software, which has bugs. Users (including bad guys, pen testers, QA folks, automated scans etc, etc, etc) find bugs, including security bugs, Developers fix bugs and maybe make more. It's an OODA loop, and continues until the developers decide to stop supporting the software. Whether that fits into the busi…

Somehow this reminded me of the historical efforts of some government bounty collections for mouse tails which were discontinued due to fraud (such as hunters breeding mice to collect the reward). There is a reason why/how devs and QA keep each other in check. Guess in case of LLM writing code, one has to use different models for dev and security checks. On other hand, in real world, the developers learn from mistake…

Reminds me of the contracts we sign with off-shore development companies to write the software at one rate and then fix bugs at a higher rate. Won’t be long till tokens spent on security review agents cost more than the tokens to create the bugs in the first place.

Re: Project Glasswing: An Initial Update

#332

Post by scotty79 »

sfink wrote: post #323
scotty79 wrote: post #268

Earlier quoted context omitted.

What are the numbers on how secure is human written code? We should have something to compare AI numbers to. It seems more likely to me that you could spend $20 to find a vulnerability in a piece of software that costed you $20k in human labor.

> What are the numbers on how secure is human written code? We should have something to compare AI numbers to. That's kind of what the article is about? Mythos is finding lots of security bugs in lots of human-written code. They can now compute some sort of baseline estimate of security bugs per N lines of human-written code or whatever. (Restricted to security bugs that the AI is currently capable of finding, but wh…

> > It seems more likely to me that you could spend $20 to find a vulnerability in a piece of software that costed you $20k in human labor.

> The topic is whether or not we have data suggesting that AI-written code is or can be secure

I think my point is related because if the AI is great at finding vulnerabilities then it should be possible just tell AI to write the code and another AI to look for vulnerabilities and secure them. All for $20 + $20 instead of 20k.

Unless AI is somehow uncharacteristically weak in finding vulnerabilities in AI produced code. Which can probably be tested.

Re: Project Glasswing: An Initial Update

#333

Post by nullbio »

qerahb wrote: post #288
nullbio wrote: post #246

Earlier quoted context omitted.

> The major short term downside is that open source or personal projects won't be able to afford things like Codex Security. Realistically, all open-source projects should be forced to have automated scans of this nature before their releases can be shipped. This is something the package managers and github need to figure out. It'd stop the supply chain attacks too.

So first they steal all code and launder it without attribution. Then they release a tool that doesn't find anything in hardened projects and is marketed through secrecy and modern equivalents of Netcraft like this British AI institute. Then open source projects need a McKinsey-like stamp of approval to even be released. Sounds like there are many parasites in this process. You know that open source users are free to…

I don't like it either. But what alternative is there when a spaghetti graph of open-source dependencies serves as the backbone for the entire worlds software? I haven't seen or heard of any novel solutions to this problem yet.

Re: Project Glasswing: An Initial Update

#334

Post by nullbio »

junon wrote: post #284
nullbio wrote: post #246

Earlier quoted context omitted.

> The major short term downside is that open source or personal projects won't be able to afford things like Codex Security. Realistically, all open-source projects should be forced to have automated scans of this nature before their releases can be shipped. This is something the package managers and github need to figure out. It'd stop the supply chain attacks too.

> all open-source projects should be forced That's a great way to kill OSS. This is only bootlicking the idea of corporations profiting off of unpaid labor.

Well something needs to be done urgently, before hospitals and critical infrastructure start getting ransomware infected on a daily basis. This isn't an unlikely scenario either, all it will take is one well resourced attacker to spin up thousands of decensored agents and have them pumping out attacks 24/7. I'm actually kind of surprised it hasn't already happened. TeamPCP is just the beginning. We're lucky they're not using ransomware, otherwise the carnage would be 100-fold.

Re: Project Glasswing: An Initial Update

#335

Post by firesteelrain »

dyauspitr wrote: post #330
firesteelrain wrote: post #295

Earlier quoted context omitted.

How is it nonsense? We already have these tools for years and magically Mythos comes around and makes old new again.

A guy working with those tools for 6 months might find some disparate vulnerabilities. You don’t see value in writing a prompt and having everything done for you in an evening? It’s not just that either, it chains various vulnerabilities, crafts an attack to exploit it and then verifies its real. You don’t see any value in this over having some guy plod through it for months? Also security is hard, most devs won’t fi…

That sounds like marketing.

Security tooling already automates scanning and exploit chaining. The hard part is context, accuracy, prioritization, and avoiding garbage results.

Re: Project Glasswing: An Initial Update

#336

Post by junon »

nullbio wrote: post #334
junon wrote: post #284

Earlier quoted context omitted.

> all open-source projects should be forced That's a great way to kill OSS. This is only bootlicking the idea of corporations profiting off of unpaid labor.

Well something needs to be done urgently, before hospitals and critical infrastructure start getting ransomware infected on a daily basis. This isn't an unlikely scenario either, all it will take is one well resourced attacker to spin up thousands of decensored agents and have them pumping out attacks 24/7. I'm actually kind of surprised it hasn't already happened. TeamPCP is just the beginning. We're lucky they're n…

Then the corporations, medical system, etc needs to help support the people who make OSS software if they want the immediate, urgent change you're suggesting.

Re: Project Glasswing: An Initial Update

#338

Post by akoboldfrying »

dwaltrip wrote: post #321
akoboldfrying wrote: post #296

Earlier quoted context omitted.

I didn't mean to imply that automation through programming is always bad. Like with any technology that increases productivity, there are many obvious benefits. We have all benefited enormously on the consumer side of the economy, for example. But I think it's recently become a lot clearer to many in the tech industry that automation can have downsides too, and those downsides are not evenly distributed. This truth w…

Programming is just another form of tool building, no? So anyone who builds things that humans use to solve problems is a job eliminator.

I agree with you that there's no obvious way to separate tool creation from job elimination. This argument holds going right back to the wheel.

I think the strongest argument that can be made against this is that the supply of human labour has varied by time and place, so that in times where labour was short, new tools were no doubt welcomed by all, whereas when labour was plentiful, new tools that eliminated some of that demand for labour were opposed by those whose livelihoods were threatened. But this is not very satisfying because the types of labour available at any time are highly contingent on the current culture and technology, i.e., highly path-dependent.

(And I think there are different kinds of labour, and that not everyone can do every kind, contra the usual capitalist assumption.)

Re: Project Glasswing: An Initial Update

#339

Post by Forgeties79 »

ben_w wrote: post #270
Forgeties79 wrote: post #204

Earlier quoted context omitted.

The problem is you as a person are not incentivized to introduce bugs in your code. If I am a company that provide provides an LLM/agent, and I know that the more bugs you have the more money I’m going to make, then I am not exactly incentivized to make my LLM/Agent better at preventing bugs. I don’t even have to explicitly make it introduce them. The incentive structure is simply out of whack.

Depends on how the billing works. For users on fixed monthly pay accounts they'll be incentivised to do the exact opposite, as their income is fixed and the cost goes up for more tokens. If the available evidence (third-party cloud pricing of open models) is correct and they make a profit on tokens but lose it on training, they will be incentivised for as many tokens as possible on pay-as-you-go API calls. If it isn'…

> but lose it on training

Call me when this stops being a meaningful cost I suppose. I just don’t think there’s much point to ignoring a massive cost burden that, even if it shifts in intensity, shows no signs of going away. Anthropocic is talking IPO and shows no signs of stopping their training. Google and Microsoft sure aren’t going to stop, they have deep as hell pockets. It’s a non-stop arm’s race currently. Frankly that’s a massive “if” at this stage. An “if” costing countless billions annually.

Re: Project Glasswing: An Initial Update

#340

Post by ben_w »

Forgeties79 wrote: post #339
ben_w wrote: post #270

Earlier quoted context omitted.

Depends on how the billing works. For users on fixed monthly pay accounts they'll be incentivised to do the exact opposite, as their income is fixed and the cost goes up for more tokens. If the available evidence (third-party cloud pricing of open models) is correct and they make a profit on tokens but lose it on training, they will be incentivised for as many tokens as possible on pay-as-you-go API calls. If it isn'…

> but lose it on training Call me when this stops being a meaningful cost I suppose. I just don’t think there’s much point to ignoring a massive cost burden that, even if it shifts in intensity, shows no signs of going away. Anthropocic is talking IPO and shows no signs of stopping their training. Google and Microsoft sure aren’t going to stop, they have deep as hell pockets. It’s a non-stop arm’s race currently. Fra…

While that matters for investors, it's completely unrelated to the question "do these companies want users consuming more tokens, e.g. by first creating buggy code and then by selling the tool to fix it?"

When the answer is "yes", it is "yes regardless of how much training costs", when it is "no", it is "no regardless of how much training costs".

Post reply on HN