111–120 of 181 posts
I'm suspicious of this prediction given the curl project's experiences...
I don't understand why the takeaway here is (unless I'm missing something), more or less "everything is going to get exploited all the time". If LLMs can really find a ton of vulnerabilities in my software, why would I not run them and just patch all the vulnerabilities, leading to perfectly secure software (or, at the very least, software for which LLMs can no longer find any new vulnerabilities)?
When did we enter the twilight zone where bug trackers are consistently empty? The limiting factor of bug reduction is remediation, not discovery. Even developer smoke testing usually surfaces bugs at a rate far faster than they can be fixed let alone actual QA. To be fair, the limiting factor in remediation is usually finding a reproducible test case which a vulnerability is by necessity. But, I would still bet most…
If you find a bug in a web browser, that's no big deal. I've encountered bugs in web browsers all the time.
You figure out how to make a web page that when viewed deletes all the files on the user's hard drive? That's a little different and not something that people discover very often.
Sure, you'll still probably have a long queue of ReDoS bugs, but the only people who think those are security issues are people who enjoy the ego boost if having a cve in their name.
Am I wrong in thinking that an "exploits are free" environment massively favors the defender? Given that real-world exploits usually chain 0days, the attacker has to find the whole chain while the defender only needs to fix the weakest link. The defender also gets to make the first move by just putting a "run an agent to find vulns" step in their CI pipeline. If LLMs truly make finding exploits free, almost no LLM-fi…
You aren't wrong for the most part, but this whole thing of "find me vulns" is not really accurately describing the space. Finding vulns has almost become sort of like a vibe thing even before LLMs. There would be some security patch that everyone says is critical because it fixes a vulnerability, but the vulnerability is like "under certain conditions, and given physical access to the device, an attacker can craft a…
In fairness, i think part of the reason people made a big deal was the novelty of the attack. It was something new. The defenses weren't clear yet. The attack surface wasn't clear. It was unclear if anyone was going to come up with a novel improvement to the technique. Humans love novelty.
Earlier quoted context omitted.
When did we enter the twilight zone where bug trackers are consistently empty? The limiting factor of bug reduction is remediation, not discovery. Even developer smoke testing usually surfaces bugs at a rate far faster than they can be fixed let alone actual QA. To be fair, the limiting factor in remediation is usually finding a reproducible test case which a vulnerability is by necessity. But, I would still bet most…
Bugs are not the same as (real) high severity bugs. If you find a bug in a web browser, that's no big deal. I've encountered bugs in web browsers all the time. You figure out how to make a web page that when viewed deletes all the files on the user's hard drive? That's a little different and not something that people discover very often. Sure, you'll still probably have a long queue of ReDoS bugs, but the only people…
Hardly. The linked anthropic paper is extremely underwhelming. It portrays no tectonic shifts.
> Practitioners will suffer having to learn the anatomy of the font gland or the Unicode text shaping lobe or whatever other “weird machines” are au courant
That's absurd. Do the vulnerability writers _start_ with this knowledge? Of course they don't. They work backwards. Anyone can do this. It just takes time in a category of development that most open source authors don't like to be occupied by.
> You can’t design a better problem for an LLM agent than exploitation research.
Did you read the anthropic article you linked? It found absolutely nothing and then immediately devolved into a search for 'strcat.' That's it. Again, literally _anyone with the time_ could just do this.
> a frontier LLM already encodes supernatural amounts of correlation across vast bodies of source code.
'grep strcat' is "supernatural?"
This starts sprawling very quickly after this. The AI revolution is not real. The cargo cult is headed for a new winter. I only see articles proclaiming the sky is just about to fall any day now, yet, I see no real world evidence any such thing is happening or likely to happen.
Earlier quoted context omitted.
You can have an AI agent refactor and improve code quality.
But, have you any code that has been vetted and verified to see if this approach works? This whole Agentic code quality claim is an assertion, but where is the literal proof?
Earlier quoted context omitted.
You can have an AI agent refactor and improve code quality.
It’s agents all the way down - until you have liability. At some point, it’s going to be someone’s neck on the line, and saying “the agents know” isn’t going to satisfy customers (or in a worst case, courts).
Earlier quoted context omitted.
Assuming you can catch every new bug it introduces. Both assumptions being unlikely. You also end up with a code base you let an AI agent trample until it is satisfied; ballooned in complexity and redudant brittle code.
You can have an AI agent refactor and improve code quality.
I interned for the author at 18. I assumed security testing worked like this: 1. Static analysis catches nearly all bugs with near-total code coverage 2. Private tooling extends that coverage further with better static analysis and dynamic analysis, and that edge is what makes contractors valuable 3. Humans focus on design flaws and weird hardware bugs like cryptographic side-channels from electromagnetic emanations…
Catching all bugs with static analysis would involve solving the halting problem, so it's never going to happen.