Are we already in the time, or close to the time, that well-trained LLMs are more efficient in finding security holes than all but the best developers out there, even for OS kernel code? Can someone educate me on this?
[deleted]
Kernel code removals driven by LLM-created security reports
21–30 of 130 posts
Re: Kernel code removals driven by LLM-created security reports
#22Are we already in the time, or close to the time, that well-trained LLMs are more efficient in finding security holes than all but the best developers out there, even for OS kernel code? Can someone educate me on this?
We are there. This is pretty much the reason why Mythos isn't being released publically.
The reason Mythos isn't being released publicly is to drive up Anthropic's valuation by making big promises.
Re: Kernel code removals driven by LLM-created security reports
#23Are we already in the time, or close to the time, that well-trained LLMs are more efficient in finding security holes than all but the best developers out there, even for OS kernel code? Can someone educate me on this?
My theory is, that a lot of security bugs are low hanging fruit for LLMs in the sense that it is a bit tedious but not that hard pattern matching. (Let's see the free occurs in foo(), so if I trigger bar() after foo() then I have a use after free, that should be possible if I trigger an exception in baz::init().)
Re: Kernel code removals driven by LLM-created security reports
#24Are we already in the time, or close to the time, that well-trained LLMs are more efficient in finding security holes than all but the best developers out there, even for OS kernel code? Can someone educate me on this?
My experience with these tools is that they generate absolutely enormous amounts of insidiously wrong false positives, and it actually takes a decent amount of skill to work through the 99% which is garbage with any velocity. Of course some people don't do that, and send all the reports anyway... and then scream from the hilltops about how incredible LLMs are when by sheer luck one happens to be right. Not only is th…
Your experience seems to be at least 3-6 months old. Long time kernel maintainers have recently written on this subject. They say that ~3 months ago the quality and accuracy of the reports crossed a threshold and are now legitimately useful.
Re: Kernel code removals driven by LLM-created security reports
#25Earlier quoted context omitted.
We are there. This is pretty much the reason why Mythos isn't being released publically.
The reason Mythos isn't being released publicly is to drive up Anthropic's valuation by making big promises.
https://blog.mozilla.org/en/privacy-security/ai-security-zer...
> As part of our continued collaboration with Anthropic, we had the opportunity to apply an early version of Claude Mythos Preview to Firefox. This week’s release of Firefox 150 includes fixes for 271 vulnerabilities identified during this initial evaluation.
Re: Kernel code removals driven by LLM-created security reports
#26Are we already in the time, or close to the time, that well-trained LLMs are more efficient in finding security holes than all but the best developers out there, even for OS kernel code? Can someone educate me on this?
My experience with these tools is that they generate absolutely enormous amounts of insidiously wrong false positives, and it actually takes a decent amount of skill to work through the 99% which is garbage with any velocity. Of course some people don't do that, and send all the reports anyway... and then scream from the hilltops about how incredible LLMs are when by sheer luck one happens to be right. Not only is th…
This is incorrect. Here's the curl maintainer talking about dozens of bugs found using LLMs: https://daniel.haxx.se/blog/2025/10/10/a-new-breed-of-analyz...
Re: Kernel code removals driven by LLM-created security reports
#27They can't maintain the code so they are no longer going to maintain the code.
and the code is in the training set, so you can trivially[0] ask an LLM to summon it back either from memory or just by asking it to revert the removal commit.
[0] not trivially if you want to validate if it works
Re: Kernel code removals driven by LLM-created security reports
#28[dead]
Re: Kernel code removals driven by LLM-created security reports
#29Are we already in the time, or close to the time, that well-trained LLMs are more efficient in finding security holes than all but the best developers out there, even for OS kernel code? Can someone educate me on this?
My experience with these tools is that they generate absolutely enormous amounts of insidiously wrong false positives, and it actually takes a decent amount of skill to work through the 99% which is garbage with any velocity. Of course some people don't do that, and send all the reports anyway... and then scream from the hilltops about how incredible LLMs are when by sheer luck one happens to be right. Not only is th…
I used GitHub's Copilot once and let it check one of my repositories for security issues. It found countless (like 30 or 40 or so for a single PHP file of some ~400 lines). Some even sounded reasonable enough, so I had a closer look, just to make sure. In the end none of it was an issue at all. In some cases it invented problems which would have forced to add wild workaround code around simple calls into the PHP standard library. And that was the only time I wasted my time with that. :D
Re: Kernel code removals driven by LLM-created security reports
#30Unmaintained code is a security issue in of itself, so this is of course a net benefit.
This can be accurately generalized: code is a security issue in and of itself.