Vulnerability research is cooked • Hacker News Board

Re: Vulnerability research is cooked

#51

Post by tptacek » Mon, Mar 30, 2026, 9:22 PM UTC

joatmon-snoo wrote: post #48

stavros wrote: post #8

I don't understand why the takeaway here is (unless I'm missing something), more or less "everything is going to get exploited all the time". If LLMs can really find a ton of vulnerabilities in my software, why would I not run them and just patch all the vulnerabilities, leading to perfectly secure software (or, at the very least, software for which LLMs can no longer find any new vulnerabilities)?

Breaking something is easier than fixing it.

People have said that for decades and it wasn't true until recently.

Re: Vulnerability research is cooked

#52

Post by batshit_beaver » Mon, Mar 30, 2026, 9:27 PM UTC

reactordev wrote: post #46

Veserv wrote: post #19

Earlier quoted context omitted.

When did we enter the twilight zone where bug trackers are consistently empty? The limiting factor of bug reduction is remediation, not discovery. Even developer smoke testing usually surfaces bugs at a rate far faster than they can be fixed let alone actual QA. To be fair, the limiting factor in remediation is usually finding a reproducible test case which a vulnerability is by necessity. But, I would still bet most…

That might have been true pre LLMs but you can literally point an agent at the queue until it’s empty now.

You literally cannot, since ANY changes to code tend to introduce unintended (or at least not explicitly requested) new behaviors.

Re: Vulnerability research is cooked

#53

Post by [deleted] » Mon, Mar 30, 2026, 9:28 PM UTC

thadt wrote: post #44

So the intersting question: are we long term safer with "simpler" closer to hardware memory unsafe(ish) environments like Zig, or is the memory safe but more abstract feature set of languages like Rust still the winning direction? If a hypothetical build step is "look over this program and carfully examine the bounds of safety using your deep knowledge of the OS, hardware, language and all the tools that come along w…

[deleted]

Re: Vulnerability research is cooked

#54

Post by rkrbaccord94f » Mon, Mar 30, 2026, 9:30 PM UTC

The pipewire-libs package local address function refers to alsa_output.pci

Driver benchmarking the pipewire script calls three local ports:

local.source.port = 10001

local.repair.port = 10002

local.control.port = 10003

Re: Vulnerability research is cooked

#55

Post by reactordev » Mon, Mar 30, 2026, 9:31 PM UTC

batshit_beaver wrote: post #52

reactordev wrote: post #46

Earlier quoted context omitted.

That might have been true pre LLMs but you can literally point an agent at the queue until it’s empty now.

You literally cannot, since ANY changes to code tend to introduce unintended (or at least not explicitly requested) new behaviors.

I’ve had mine on a Ralph loop no problem. Just review the PR..

Re: Vulnerability research is cooked

#56

Post by k_roy » Mon, Mar 30, 2026, 9:31 PM UTC

batshit_beaver wrote: post #52

reactordev wrote: post #46

Earlier quoted context omitted.

That might have been true pre LLMs but you can literally point an agent at the queue until it’s empty now.

You literally cannot, since ANY changes to code tend to introduce unintended (or at least not explicitly requested) new behaviors.

Which still means a single person with Claude can clear a queue in a day versus a month with a traditional team.

Re: Vulnerability research is cooked

#57

Post by moyix » Mon, Mar 30, 2026, 9:31 PM UTC

staticassertion wrote: post #49

moyix wrote: post #45

Earlier quoted context omitted.

This is true for a lot of things but for low-level code you can always fall back to "the intention is to not violate memory safety".

That's true, but certainly that's limiting. Still, even then, `# SAFETY:` comments seem extremely helpful. "For every `unsafe`, determine its implied or stated safety contract, then build a suite of adversarial tests to verify or break those contracts" feels like a great way to get going.

It's limiting from the PoV of a developer who wants to ensure that their own code is free of all security issues. It is not limiting from the point of view of an attacker who just needs one good memory safety vuln to win.

Re: Vulnerability research is cooked

#58

Post by lll-o-lll » Mon, Mar 30, 2026, 9:32 PM UTC

batshit_beaver wrote: post #52

reactordev wrote: post #46

Earlier quoted context omitted.

That might have been true pre LLMs but you can literally point an agent at the queue until it’s empty now.

You literally cannot, since ANY changes to code tend to introduce unintended (or at least not explicitly requested) new behaviors.

Eventual convergence? Assuming each defect fix has a 30% chance of introducing a new defect, we keep cycling until done?

Re: Vulnerability research is cooked

#59

Post by underdeserver » Mon, Mar 30, 2026, 9:32 PM UTC

tptacek wrote: post #51

joatmon-snoo wrote: post #48

Earlier quoted context omitted.

Breaking something is easier than fixing it.

People have said that for decades and it wasn't true until recently.

Specifically in software vulnerability research, you mean.

Fixing vulnerable code is usually trivial.

In the physical world breaking things is usually easier.

Re: Vulnerability research is cooked

#60

Post by samuelknight » Mon, Mar 30, 2026, 9:33 PM UTC

LLMs are expert hackers because: 1) They are expert coders, including a decently comprehensive CVE knowledge 2) They know every programming language/framework/stack 3) They know every human language

They already have super human breadth and attention. And their depth is either super human or getting there.

The state of the security industry through 2025 was expensive appsec human reviewers or primitive scanners. Now you can spend a few dollars and have an expert intelligence scrutinize a whole network.