Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vulnerability research is cooked

sockpuppet.org

Post reply on HN
Print view View original

171–180 of 181 posts

Re: Vulnerability research is cooked

#171

Post by strogonoff »

pron wrote: post #170
strogonoff wrote: post #154

Earlier quoted context omitted.

> > exploiting software is someone’s full-time job, whereas the engineers already have one—building it. > But the attackers needs to spread their attack over many products, while the engineers only need to defend one. Are you assuming every piece of software has a dedicated defender team? Strikes me as unlikely. Realistically, you have people whose job or passion is to develop software, who often work not on one but…

> Are you assuming every piece of software has a dedicated defender team? Strikes me as unlikely. No, I'm assuming it has maintainers (they play the role of defenders). > engineers who work on software are simply not that great and dedicated about finding vulnerabilities in it. Yes, but LLMs help them more than they help the attackers, because the attackers are already security experts. In other words, the LLMs reduc…

> I'm assuming it has maintainers (they play the role of defenders).

A maintainer has a full-time job: to develop software. A maintainer who is also a defender has two full-time jobs, and as we all know in such a case one of these jobs will have to be done poorly, and we all know which one that is.

On the other side there’s an attacker with a singular job and a strong incentive to do it well.

> LLMs help them more than they help the attackers, because the attackers are already security experts.

The supposed logic is that an LLM multiplies your skill. If the multiplier is 5, and your attacking skill is 1 before the multiplication, then you get 5 after; if your attacking skill is alreaady at 10, you get 50. You could argue that LLMs are not good enough to act as multipliers, and then my math won’t work.

Re: Vulnerability research is cooked

#172

Post by cushychicken »

tptacek wrote: post #165
cushychicken wrote: post #157

Earlier quoted context omitted.

Repeatability and/or an actual negative effect. POC generally means “you can demonstrate unintentional behavior”. “Exploit” means you can gain access or do something malicious. It’s a fine line. Author’s point is that the LLM was able to demonstrate some malfeasance, not just unintended consequence. That’s a big deal considering that actual malicious intent generally requires more knowhow than raw POC.

Specifically: the exploit extracted the admin's credentials from the database. A blind SQLI POC would simply demonstrate the existence of a timing channel based on a pathological input.

One other commenter asked a decent question - does going lighter (Zig) or harder on memory safety (Rust) confer any meaningful advantages against the phenomenon you describe?

Re: Vulnerability research is cooked

#173

Post by pron »

strogonoff wrote: post #171
pron wrote: post #170

Earlier quoted context omitted.

> Are you assuming every piece of software has a dedicated defender team? Strikes me as unlikely. No, I'm assuming it has maintainers (they play the role of defenders). > engineers who work on software are simply not that great and dedicated about finding vulnerabilities in it. Yes, but LLMs help them more than they help the attackers, because the attackers are already security experts. In other words, the LLMs reduc…

> I'm assuming it has maintainers (they play the role of defenders). A maintainer has a full-time job: to develop software. A maintainer who is also a defender has two full-time jobs, and as we all know in such a case one of these jobs will have to be done poorly, and we all know which one that is. On the other side there’s an attacker with a singular job and a strong incentive to do it well. > LLMs help them more th…

> A maintainer has a full-time job: to develop software. A maintainer who is also a defender has two full-time jobs,

I don't think so. This is already the situation. Maintainers already fix vulnerabilities when they know about them.

> On the other side there’s an attacker with a singular job and a strong incentive to do it well.

If the situation is that the attacker is focusing on a single project, the attacker will win, as they do already. But the attackers usually need to split their attention over lots of projects.

> The supposed logic is that an LLM multiplies your skill

I don't agree with that logic. Agents bring knowledge with them. That's not a multiplier. Compare how well a 12 year old can do compared to a Roman history professor on questions about Roman history when they both can use an LLM or when they both can't. The LLM will shrink the gap, not increase it.

Re: Vulnerability research is cooked

#174

Post by acdha »

weavie wrote: post #146
stavros wrote: post #8

I don't understand why the takeaway here is (unless I'm missing something), more or less "everything is going to get exploited all the time". If LLMs can really find a ton of vulnerabilities in my software, why would I not run them and just patch all the vulnerabilities, leading to perfectly secure software (or, at the very least, software for which LLMs can no longer find any new vulnerabilities)?

I've worked at companies before where they have balked at spending $300 to buy me a second hand thinkpad because I really wanted to work on a Linux machine rather than a mac. I don't see them throwing $unlimited at tokens to find vulnerabilities, at least until after it's too late.

I think you’re right that they’re going to skimp as much as regulators & the market let them, but that Thinkpad would cost a lot more than $300: a new platform is an ongoing cost for maintenance, security, and interoperability – not crushing, but those factors quickly outweigh the hardware.

Re: Vulnerability research is cooked

#175

Post by strogonoff »

pron wrote: post #173
strogonoff wrote: post #171

Earlier quoted context omitted.

> I'm assuming it has maintainers (they play the role of defenders). A maintainer has a full-time job: to develop software. A maintainer who is also a defender has two full-time jobs, and as we all know in such a case one of these jobs will have to be done poorly, and we all know which one that is. On the other side there’s an attacker with a singular job and a strong incentive to do it well. > LLMs help them more th…

> A maintainer has a full-time job: to develop software. A maintainer who is also a defender has two full-time jobs, I don't think so. This is already the situation. Maintainers already fix vulnerabilities when they know about them. > On the other side there’s an attacker with a singular job and a strong incentive to do it well. If the situation is that the attacker is focusing on a single project, the attacker will…

> I don't think so. This is already the situation. Maintainers already fix vulnerabilities when they know about them.

This is already the situation and it is a problem and that is why we are talking about it.

> If the situation is that the attacker is focusing on a single project, the attacker will win, as they do already. But the attackers usually need to split their attention over lots of projects.

Just like that, the developers split their attention over N projects, the activities of developing and finding vulnerabilities, etc. Unlike the attackers, they live in free countries without figurative guns to their heads. Unlike the attackers, they do not have government-funded datacenters churning on finding vulnerabilities. So it more than cancels out, and you are repeating yourself.

> I don't agree with that logic

Sure, knock yourself out.

> The LLM will shrink the gap, not increase it.

I’m not going to argue with you on behalf of all the different posters here who claim how LLM help more if you are already knowledgeable and don’t help as much if you are a beginner and don’t actually know what you are doing compared to the pro. I think you are a minority in your opinion.

Re: Vulnerability research is cooked

#176

Post by HackerThemAll »

If we can't get zero-days discovered and fixed in due time to protect our privacy, I certainly do hope one day AI will discover most of it and suggest how to fix it.

I do hope it's going to be capable enough to be plugged into CI/CD to discover that the top-talent today made another obvious XSS, SQLi or other trivial issue that just created a 0-day. Even a few of those cyber-models, so they verify each other. I do hope it's going to be trained on all prior issues, like the one with xz, or Axios, and be vigilant against these things.

Re: Vulnerability research is cooked

#177

Post by saagarjha »

nickpsecurity wrote: post #164
saagarjha wrote: post #124

Earlier quoted context omitted.

Silo’d, academic codebases are not under the kind of attacks that commodity software is

The silo'd codebases I was referring to are verification tools they produce. They're used to prevent attacks. Each tool has one or more capabilities others lack. If combined, they'd catch many problems. Examples: KLEE test generator; combinatorial or path-bases testing; CPAChecker; race detectors for concurrency; SIF information flow control; symbolic execution; Why3 verifier which commercial tools already build on.

"Each lacks capabilities" is not a strong sell for "together they can catch most problems".

Re: Vulnerability research is cooked

#178

Post by benlivengood »

somesortofthing wrote: post #81

Am I wrong in thinking that an "exploits are free" environment massively favors the defender? Given that real-world exploits usually chain 0days, the attacker has to find the whole chain while the defender only needs to fix the weakest link. The defender also gets to make the first move by just putting a "run an agent to find vulns" step in their CI pipeline. If LLMs truly make finding exploits free, almost no LLM-fi…

The economics favor attackers. Who sells 0-days for quite a lot of money (or directly exploits them for ransomware), vs. who has to pay engineers quite a lot of money to review bug reports and patch code and publish new releases?

The validation/verification balance also favors attackers. "Yes, I now have a remote root shell on this VM with a default install of X" vs. "My test suite is not dependable enough to turn an agent loose fixing security bug reports, not to mention the extra QA work that live humans would have to do where there isn't coverage".

Re: Vulnerability research is cooked

#179

Post by rybosworld »

jen729w wrote: post #145
rybosworld wrote: post #80

Earlier quoted context omitted.

Shouldn't be down voted for saying this. There are active repo's this is happening in. "BuT ThE LlM iS pRoBaBlY iNtRoDuCiNg MoRe BuGs ThAn It FiXeS" This is an absurd take.

I wonder whether people like you have actually used Claude for any length of time. I use it all day. I consider it a near-miracle. Yet I correct it multiple times daily.

> I wonder whether people like you have actually used Claude for any length of time.

I stated the LLMs are actively being used in repo's today, to chew through backlog items, and your response is to wonder if I've ever used Claude.

To me it's surprising that someone like you, who appears to have a reading comprehension deficiency, is able to use Claude.

Re: Vulnerability research is cooked

#180

Post by threethirtytwo »

habinero wrote: post #91
lifty wrote: post #29

Earlier quoted context omitted.

Perhaps a meta evolution, they become experts at writing harnesses and prompts for discovering and patching vulnerabilities in existing code and software. My main interest is, now that we have LLMs, will the software industry move to adopting techniques like formal verification and other perhaps more lax approaches that massively increase the quality of software.

Testing exists. > formal verification Outside of limited specific circumstances, formal verification gives you nothing that tests don't give you, and it makes development slow and iteration a chore. People know about it, and it's not used for lot of reasons.

This statement shows an intense lack of technical knowledge. You’re probably one of those ignorant managerial types.

First type checking is a form of formal verification and it’s used everywhere. Second have you heard of rust? Do you know why it’s becoming an alternative to C++ or C? Entirely because of its type checker or aka formal verification. It is the literal main reason why rust was created.

Have you heard of typescript? It’s essentially a formal verification layer over JavaScript. Everyone uses it now for the front end.

You don’t know what you’re talking about. I recommend you do some research before saying anything on this site.

Post reply on HN