solution is simple: commits get shared privately. distros can build on these private patches. after some point those can be made public (ie commited at upstream)
AI is breaking two vulnerability cultures
161–170 of 188 posts
Re: AI is breaking two vulnerability cultures
#162Earlier quoted context omitted.
> based on the unquestioned premise that delaying disclosure for the operational convenience of system administrators is a good thing. There are reasons to question that premise! Care to mention these reasons? With "convenience of system administrators", I'm guessing you mean that there's a patch available that sysadmins can install, ideally before the vulnerability is disclosed? What else are sysadmins supposed to d…
Choose to take an availability hit rather than risk a breach.
With "availability hit" I'm assuming you mean to simply stop operations until patches are rolled out, so possibly for days? That would at least explain what's happening at GitHub...
Re: AI is breaking two vulnerability cultures
#163Earlier quoted context omitted.
They're saying to do that scan to every diff before release, to see if it finds anything.
The point is that even if all code commits are scanned as safe by ai, black hats can still analyse the commits and diffs to find vulnerabilites for people who havent patched yet. Scanning every commit doesnt automatically make everyone in the world patch immediately, vulns can still be found from commits and diffs and used against those who havent patched yet.
Look at GP to my comment again, the one I was clarifying: they're not talking about black hats or any other kind of hacker, they're talking about the original developers and preventing such vulnerabilities from existing in the first place.
Re: AI is breaking two vulnerability cultures
#164Perimeter defense is outdated. AI changes the attack surface from network edges to the data pipeline itself. We need to securing the prompt, the model, and the data.
Re: AI is breaking two vulnerability cultures
#165Earlier quoted context omitted.
The point is that even if all code commits are scanned as safe by ai, black hats can still analyse the commits and diffs to find vulnerabilites for people who havent patched yet. Scanning every commit doesnt automatically make everyone in the world patch immediately, vulns can still be found from commits and diffs and used against those who havent patched yet.
Look at GP to my comment again, the one I was clarifying: they're not talking about black hats or any other kind of hacker, they're talking about the original developers and preventing such vulnerabilities from existing in the first place.
Yes I am aware, however that still does not stop anybody examining your commits and diffs to find vulnerabilities.
Do you assume ai will just stop at a certain level? Or is it possible that it will keep increasing in intelligence? If the latter, then isnt it possible that even if you are auto checking all your commits, next week a more advanced ai model might be released that finds vulns in your old commits, even though they were checked by (an inferior) ai?
Blinding saying that auto checking commits will make you safe from ai based attacks and vulnerability free is just madness.
Re: AI is breaking two vulnerability cultures
#166Earlier quoted context omitted.
Is there evidence serious vulnerabilities are the result of vibe coding already? I haven’t seen any so if you have some references, please share.
Security researcher Dor Zvi and his team at the cybersecurity firm he cofounded, RedAccess, analyzed thousands of vibe-coded web applications created using the AI software development tools Lovable, Replit, Base44, and Netlify and found more than 5,000 of them that had virtually no security or authentication of any kind. Many of these web apps allowed anyone who merely finds their web URL to access the apps and their…
That’s quite different. Vibe coded apps are not normally even meant to be secure, it’s meant to be used by the creator only. Bad app security is not the same as a vulnerability. A vulnerability would be a library providing some functionality it claims is secure, but in reality it’s not.
Re: AI is breaking two vulnerability cultures
#167Earlier quoted context omitted.
> It has been over a decade since any ordinary off-the-shelf closed-source software was meaningfully obscured from serious adversaries. Probably goes without saying but the last line of defense is not deploying your software publicly and instead relying on server-client architectures to do anything. Maybe this will be more common as vulnerabilities are more easily detected and exploited. Of course its not always feas…
>Only the undeployed server code has remained private. How easy to do you this is for LLM to build decent emulator of the server in question by just observing what you send and what you get as response?
Honestly, I can't really imagine how this would work at all?
I could see how, given enough data, you'd be able to infer the intended logic of the server and reimplement something that's compatible (I've done this myself with Wireshark + USB devices in the past).
But how would could you reason about specific vulnerabilities in remote code just from a set of requests and responses?
Re: AI is breaking two vulnerability cultures
#168Obviously the solution is for Linux to move to a closed-source development model. Security researchers should report their findings to a committee that includes some big companies (IBM and Oracle seem like trustworthy choices here, but ideally we should find a way to get Microsoft included). Those companies would apply the security patches and distribute binary builds of Linux to their customers. Users fortunate enou…
[deleted]
Re: AI is breaking two vulnerability cultures
#169Earlier quoted context omitted.
> based on the unquestioned premise that delaying disclosure for the operational convenience of system administrators is a good thing. There are reasons to question that premise! Care to mention these reasons? With "convenience of system administrators", I'm guessing you mean that there's a patch available that sysadmins can install, ideally before the vulnerability is disclosed? What else are sysadmins supposed to d…
Choose to take an availability hit rather than risk a breach.
Presumably you also have positive downstream effects in mind: when "taking the availability hit" feels like more of a live choice, operators feel the pain of running insecure designs more. Do what you describe a couple times, and you'll naturally start thinking things like "dammit, we need to finally get away from shared kernels; this is insane", "maybe we should figure out a way to do this that doesn't involve running software that runs in God mode", or even "we should see what it takes to port our application to a platform that is more secure by design".
When you can't imagine or pretend that when a major vuln is disclosed (a) you've been secure up until the point of disclosure or (b) all you need to do now is apply a patch without thinking too much about what your blast radius just was, you might actually have stronger incentives to think about the design of the overall system so that when similar issues come up, you can avoid having to sweat those outages.
It's interesting that "defense-in-depth" gets cited and repeated all the time but the standard attitude about patching still seems to be "what do you mean?? isn't patching the only thing we can do?". How about designing systems so that you can more quickly and easily throw up other kinds of mitigations when you need to? What about designing systems with robust enough notions of graceful degradation that when something crops up for a certain feature you can "just" say "okay, let's turn only that part off for a couple days"? How about getting really, really good at CI/CD so you can more confidently add and deploy mitigations to your application code, or redeploy with a feature flag that lets you temporarily drop an unpatched-and-vulnerable dependency?
If you can manage to build a system without the assumption that just patching is always on the table, you might simply end up with better software, which would be pretty cool.
Re: AI is breaking two vulnerability cultures
#170Since the exploit can be mitigated by simply blacklisting the AF_ALG module, why didn’t they release an advisory to disable the problematic module (which AFAIU is hardly used), and then only later, say after a week, release the patch for it? At least then you would have the immediate ability for a mitigation without giving away exactly how to exploit the bug.