141–150 of 181 posts
I don't understand why the takeaway here is (unless I'm missing something), more or less "everything is going to get exploited all the time". If LLMs can really find a ton of vulnerabilities in my software, why would I not run them and just patch all the vulnerabilities, leading to perfectly secure software (or, at the very least, software for which LLMs can no longer find any new vulnerabilities)?
Earlier quoted context omitted.
It’s agents all the way down - until you have liability. At some point, it’s going to be someone’s neck on the line, and saying “the agents know” isn’t going to satisfy customers (or in a worst case, courts).
Sure it can. It's not like humans aren't already deflecting liability or moving it to insurance agencies.
They attempt to, sure, but it rarely works. Now, with AI, maybe it might, but that's sort of a worse outcome for the specific human involved - "If you're just an intermediary between the AI and me, WTF do I need you for?"
> or moving it to insurance agencies.
They aren't "moving" it to insurance companies, they are amortising the cost of the liability at a small extra cost.
That's a big difference.
Am I wrong in thinking that an "exploits are free" environment massively favors the defender? Given that real-world exploits usually chain 0days, the attacker has to find the whole chain while the defender only needs to fix the weakest link. The defender also gets to make the first move by just putting a "run an agent to find vulns" step in their CI pipeline. If LLMs truly make finding exploits free, almost no LLM-fi…
Considering the new 'meta' that LLMs encode knowledge about existing software but not new ones, I would expect a side effect that newly written software will be inherently less exploitable by LLMs, even if from an actual security perspective, they're worse in design.
I don't understand why the takeaway here is (unless I'm missing something), more or less "everything is going to get exploited all the time". If LLMs can really find a ton of vulnerabilities in my software, why would I not run them and just patch all the vulnerabilities, leading to perfectly secure software (or, at the very least, software for which LLMs can no longer find any new vulnerabilities)?
[flagged]
Earlier quoted context omitted.
That might have been true pre LLMs but you can literally point an agent at the queue until it’s empty now.
Shouldn't be down voted for saying this. There are active repo's this is happening in. "BuT ThE LlM iS pRoBaBlY iNtRoDuCiNg MoRe BuGs ThAn It FiXeS" This is an absurd take.
I use it all day. I consider it a near-miracle. Yet I correct it multiple times daily.
I don't understand why the takeaway here is (unless I'm missing something), more or less "everything is going to get exploited all the time". If LLMs can really find a ton of vulnerabilities in my software, why would I not run them and just patch all the vulnerabilities, leading to perfectly secure software (or, at the very least, software for which LLMs can no longer find any new vulnerabilities)?
Earlier quoted context omitted.
That might be one outcome, especially for large, expertly-staffed vendors who are already on top of this stuff. My real interest in what happens to the field for vulnerability researchers.
Perhaps a meta evolution, they become experts at writing harnesses and prompts for discovering and patching vulnerabilities in existing code and software. My main interest is, now that we have LLMs, will the software industry move to adopting techniques like formal verification and other perhaps more lax approaches that massively increase the quality of software.
Harnesses, maybe, but prompts?
There's still this belief amongst AI coders that they can command a premium for development because they can write a prompt better than Bob from HR, or Sally from Accounting.
When all you're writing are prompts, your value is less than it was before., because the number of people who can write the prompt is substantially more than the number of people who could program.
Earlier quoted context omitted.
Catching all bugs with static analysis is actually really easy, as long as you don't mind false positives.
Conventional static analysis tools come nowhere close to catching all bugs, even accounting for the false positives.
If everything is reported as a bug, there will be 0 false negatives but a lot of false positives
Earlier quoted context omitted.
It’s agents all the way down - until you have liability. At some point, it’s going to be someone’s neck on the line, and saying “the agents know” isn’t going to satisfy customers (or in a worst case, courts).
Sure it can. It's not like humans aren't already deflecting liability or moving it to insurance agencies.
Usually thats after the premiums become too high for most people to pay.
Earlier quoted context omitted.
You literally cannot, since ANY changes to code tend to introduce unintended (or at least not explicitly requested) new behaviors.
Eventual convergence? Assuming each defect fix has a 30% chance of introducing a new defect, we keep cycling until done?
What if it's a 200% chance, and every fix introduces multiple defects?