Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vulnerability research is cooked

sockpuppet.org

Post reply on HN
Print view View original

61–70 of 181 posts

Re: Vulnerability research is cooked

#61

Post by IsTom »

Legend2440 wrote: post #34
spr-alex wrote: post #10

I interned for the author at 18. I assumed security testing worked like this: 1. Static analysis catches nearly all bugs with near-total code coverage 2. Private tooling extends that coverage further with better static analysis and dynamic analysis, and that edge is what makes contractors valuable 3. Humans focus on design flaws and weird hardware bugs like cryptographic side-channels from electromagnetic emanations…

Catching all bugs with static analysis would involve solving the halting problem, so it's never going to happen.

A lot of software doing useful work halts pretty trivialy, consuming inputs and doing bounded computation on each of them. You're not going to recurse much in click handlers or keep making larger requests to handle the current one.

Re: Vulnerability research is cooked

#62

Post by gdulli »

samuelknight wrote: post #60

LLMs are expert hackers because: 1) They are expert coders, including a decently comprehensive CVE knowledge 2) They know every programming language/framework/stack 3) They know every human language They already have super human breadth and attention. And their depth is either super human or getting there. The state of the security industry through 2025 was expensive appsec human reviewers or primitive scanners. Now…

So much of the current internet is posts that read as a superposition of sincere and parody, and until that's resolved how do you know how to respond?

Re: Vulnerability research is cooked

#63

Post by Kinrany »

lll-o-lll wrote: post #58
batshit_beaver wrote: post #52

Earlier quoted context omitted.

You literally cannot, since ANY changes to code tend to introduce unintended (or at least not explicitly requested) new behaviors.

Eventual convergence? Assuming each defect fix has a 30% chance of introducing a new defect, we keep cycling until done?

Why would it converge?

Re: Vulnerability research is cooked

#64

Post by charcircuit »

joatmon-snoo wrote: post #48
stavros wrote: post #8

I don't understand why the takeaway here is (unless I'm missing something), more or less "everything is going to get exploited all the time". If LLMs can really find a ton of vulnerabilities in my software, why would I not run them and just patch all the vulnerabilities, leading to perfectly secure software (or, at the very least, software for which LLMs can no longer find any new vulnerabilities)?

Breaking something is easier than fixing it.

A proper fix maybe. But LLMs can easily make it no longer exploitable in most cases.

Re: Vulnerability research is cooked

#65

Post by underdeserver »

tptacek wrote: post #12
stavros wrote: post #8

I don't understand why the takeaway here is (unless I'm missing something), more or less "everything is going to get exploited all the time". If LLMs can really find a ton of vulnerabilities in my software, why would I not run them and just patch all the vulnerabilities, leading to perfectly secure software (or, at the very least, software for which LLMs can no longer find any new vulnerabilities)?

That might be one outcome, especially for large, expertly-staffed vendors who are already on top of this stuff. My real interest in what happens to the field for vulnerability researchers.

Just wanted to point out that tptacek is the blog post's author (and a veteran security researcher).

Re: Vulnerability research is cooked

#66

Post by vibe42 »

If everyone is running the same models, does this not favour white hat / defense?

Since many exploits consists of several vulnerabilities used in a chain, if a LLM finds one in the middle and it's fixed, that can change a zero day to something of more moderate severity?

E.g. someone finds a zero day that's using three vulns through different layers. The first and third are super hard to find, but the second is of moderate difficulty.

Automated checks by not even SOTA models could very well find the moderate difficulty vuln in the middle, breaking the chain.

Re: Vulnerability research is cooked

#67

Post by anematode »

Ya, I tend to believe that (most) human VR will be obsoleted well before human software engineering. Software engineering is a lot more squishy and has many more opportunities to go off the rails. Once a goal is established, the output of VR agents is verifiable.

Re: Vulnerability research is cooked

#68

Post by saintfire »

lll-o-lll wrote: post #58
batshit_beaver wrote: post #52

Earlier quoted context omitted.

You literally cannot, since ANY changes to code tend to introduce unintended (or at least not explicitly requested) new behaviors.

Eventual convergence? Assuming each defect fix has a 30% chance of introducing a new defect, we keep cycling until done?

Assuming you can catch every new bug it introduces.

Both assumptions being unlikely.

You also end up with a code base you let an AI agent trample until it is satisfied; ballooned in complexity and redudant brittle code.

Re: Vulnerability research is cooked

#69

Post by saltcured »

gdulli wrote: post #62
samuelknight wrote: post #60

LLMs are expert hackers because: 1) They are expert coders, including a decently comprehensive CVE knowledge 2) They know every programming language/framework/stack 3) They know every human language They already have super human breadth and attention. And their depth is either super human or getting there. The state of the security industry through 2025 was expensive appsec human reviewers or primitive scanners. Now…

So much of the current internet is posts that read as a superposition of sincere and parody, and until that's resolved how do you know how to respond?

Just wait until you see the same showing up in compliance realms...

Edit: to be slightly less implicit, consider the cargo cult madness that erupts from people thinking they can address risk management and compliance by auto-generating documentation and avoid really doing the legwork.

Re: Vulnerability research is cooked

#70

Post by love2read »

thadt wrote: post #44

So the intersting question: are we long term safer with "simpler" closer to hardware memory unsafe(ish) environments like Zig, or is the memory safe but more abstract feature set of languages like Rust still the winning direction? If a hypothetical build step is "look over this program and carfully examine the bounds of safety using your deep knowledge of the OS, hardware, language and all the tools that come along w…

What about this article raises this question? If anything, this article makes it pretty clear that memory safe languages are a win. It seems like a serious disadvantage to require a nondeterministic program to evaluate your code's safety.
Post reply on HN