Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kernel code removals driven by LLM-created security reports

lwn.net

Post reply on HN
Print view View original

31–40 of 130 posts

Re: Kernel code removals driven by LLM-created security reports

#31

Post by devmor »

traceroute66 wrote: post #5
staticassertion wrote: post #2

They can't maintain the code so they are no longer going to maintain the code.

> They can't maintain the code so they are no longer going to maintain the code. Yes, I don't see the point of maintaining technical debt just for the sake of it. The security environment in 2026 is such that legacy unmaintained code is a very real security risk for obscure zero-days to exploit to gain a foot in the door. Reading through the list I don't see it being an issue for the overwhelming majority of Linux us…

> Who, for example, still uses ISDN in 2026?

Most TV and radio stations.

Re: Kernel code removals driven by LLM-created security reports

#32

Post by catlifeonmars »

xbar wrote: post #30
mmsc wrote: post #19

Unmaintained code is a security issue in of itself, so this is of course a net benefit.

This can be accurately generalized: code is a security issue in and of itself.

This can be generalized: in and of itself.

Re: Kernel code removals driven by LLM-created security reports

#34

Post by mey »

xbar wrote: post #30
mmsc wrote: post #19

Unmaintained code is a security issue in of itself, so this is of course a net benefit.

This can be accurately generalized: code is a security issue in and of itself.

Now if only I could get the product team to fully understand that implication.

Re: Kernel code removals driven by LLM-created security reports

#35

Post by [deleted] »

binaryturtle wrote: post #29
jcalvinowens wrote: post #18

Earlier quoted context omitted.

My experience with these tools is that they generate absolutely enormous amounts of insidiously wrong false positives, and it actually takes a decent amount of skill to work through the 99% which is garbage with any velocity. Of course some people don't do that, and send all the reports anyway... and then scream from the hilltops about how incredible LLMs are when by sheer luck one happens to be right. Not only is th…

I used GitHub's Copilot once and let it check one of my repositories for security issues. It found countless (like 30 or 40 or so for a single PHP file of some ~400 lines). Some even sounded reasonable enough, so I had a closer look, just to make sure. In the end none of it was an issue at all. In some cases it invented problems which would have forced to add wild workaround code around simple calls into the PHP stan…

[deleted]

Re: Kernel code removals driven by LLM-created security reports

#36

Post by LeCompteSftware »

ferguess_k wrote: post #7

Are we already in the time, or close to the time, that well-trained LLMs are more efficient in finding security holes than all but the best developers out there, even for OS kernel code? Can someone educate me on this?

"Even for OS kernel code" is doing a lot of work. What you really mean is "legacy C code" and yes, since about 6 months ago these systems have gotten reliable enough that they are basically superhuman at identifying buffer overflows / etc. A remarkable number of these bugs are fixed by adding a (if (length > MAX_BUFFER) {return -1;}), just the classic C footguns. Even as a huge LLM skeptic I am not too too surprised that these systems might be superhuman at finding tedious tricky stuff like this.

At the same time, a lot of these bugs were in places that people weren't looking because it's not actually important. This kernel code had already been a longstanding problem in terms of low-effort bot-driven security reports and nobody had any interest in maintaining it. So this was more LLM-assisted technical management than LLM-assisted security, it finally made a situation uncomfortable enough for the team to do something about it.

Another example: Mythos found a real bug in FreeBSD that occurs when running as an NFS with a public connection. But... who on earth is doing that? I would guess 99.9% of FreeBSD NFS installations are on home LANs. More importantly, Anthropic spent $20,000 to find this bug. Just think in terms of paying a full-time FreeBSD dev for a month and that's what they find: I'd say "ok, looks like FreeBSD has a pretty secure codebase, let's fix that stupid bug, stop wasting our money, and get you on a more exciting project."

I do think anyone who has a legacy open-source C/C++ codebase owes it to their users to run it by Claude/Codex, check your pointers and arrays, make sure everything looks ok. I just wish people were able to discuss it in proper context about other native debugging tools!

Re: Kernel code removals driven by LLM-created security reports

#37

Post by bri3d »

ferguess_k wrote: post #7

Are we already in the time, or close to the time, that well-trained LLMs are more efficient in finding security holes than all but the best developers out there, even for OS kernel code? Can someone educate me on this?

"More efficient" of course has many axes (cost, energy consumption, manual labor requirement vs cost of human, time, quality, etc.). However, as a long-time reverse engineer and exploit developer who has worked in the field professionally, I would say LLMs are now useful; their utility exceeds that which was previously available. That is, LLM assisted exploit discovery and especially development is faster, more efficient, and ultimately cheaper than non-LLM assisted processes.

What commenters don't seem to understand is that especially CVE spam / bug bounty type vulnerability research has always been an exercise in sifting through useless findings and hallucinations, and LLMs, used well, are great at reducing this burden.

Previously, a lot of "baseline" / bottom tier research consisted of "run fuzzers or pentest tools against a product; if you're a bottom feeder just stuff these vulns all into the submission box, if you're more legit, tediously try to figure out which ones are reachable." LLMs with a test harness do an _amazing_ job at reducing this tedium; in the memory safety space "read across 50 files to figure out if this UAF might be reachable" or in the web space, "follow this unsanitized string variable to see if it can be accessed by the user" are tasks that LLMs with a harness are awesome. The current models are also about 50% there at "make a chain for this CVE," depending on the shape of the CVE (they usually get close given a good test harness).

It seems that the concern with the unreleased models is pretty much that this has advanced once again from where it is today (where you need smart prompting and a good harness) to the LLM giving you exploit chains in exchange for "giv 0day pl0x," and based on my experience, while this has got an element of puffery and classic capitalist goofiness to it ("the model is SO DANGEROUS only our RICHEST CUSTOMERS can have it!"), I believe this is just a small incremental step and entirely believable.

To summarize: "more efficient than all but the best" comes with too many qualifiers, but "are LLMs meaningfully useful in exercising vulnerabilities in OS kernel code," or "is it possible to accelerate vulnerability research and development with LLMs" - 100% absolutely.

And you don't have to believe one random professional (me); this opinion is fairly widespread across the community:

https://sockpuppet.org/blog/2026/03/30/vulnerability-researc...

https://lwn.net/Articles/1065620/

etc.

Re: Kernel code removals driven by LLM-created security reports

#39

Post by traceroute66 »

devmor wrote: post #31
traceroute66 wrote: post #5

Earlier quoted context omitted.

> They can't maintain the code so they are no longer going to maintain the code. Yes, I don't see the point of maintaining technical debt just for the sake of it. The security environment in 2026 is such that legacy unmaintained code is a very real security risk for obscure zero-days to exploit to gain a foot in the door. Reading through the list I don't see it being an issue for the overwhelming majority of Linux us…

> Who, for example, still uses ISDN in 2026? Most TV and radio stations.

> Most TV and radio stations.

I doubt it. And as I said, telcos have ceased new sales of ISDN and will be shutting down copper networks within 3–5 years.

Therefore if there are still TV and radio stations still using it, they will be forced to stop using it by circumstance, i.e. they will find their ISDN will cease working after the telco shuts down the kit in the exchange.

Re: Kernel code removals driven by LLM-created security reports

#40

Post by bri3d »

jcalvinowens wrote: post #18
ferguess_k wrote: post #7

Are we already in the time, or close to the time, that well-trained LLMs are more efficient in finding security holes than all but the best developers out there, even for OS kernel code? Can someone educate me on this?

My experience with these tools is that they generate absolutely enormous amounts of insidiously wrong false positives, and it actually takes a decent amount of skill to work through the 99% which is garbage with any velocity. Of course some people don't do that, and send all the reports anyway... and then scream from the hilltops about how incredible LLMs are when by sheer luck one happens to be right. Not only is th…

I strongly disagree with this take, and frankly, this reads like the state of "research" pre-LLMs where people would run fuzzers and scripted analysis tools (which by their nature DO generate enormous amounts of insidiously wrong false positives) and stuff them into bug bounty boxes, then collect a paycheck when one was correct by luck.

Modern LLMs with a reasonable prompt and some form of test harness are, in my experience, excellent at taking a big list of potential vulnerabilities and figuring out which ones might be real. They're also pretty good, depending on the class of vuln and the guardrails in the model, at developing a known-reachable vulnerability into real exploit tooling, which is also a big win. This does require the _slightest_ bit of work (ie - don't prompt the LLM with "find possible use after free issues in this code," or it will give you a lot of slop; prompt the LLM with "determine whether the memory safety issues in this file could present a security risk" and you get somewhere), but not some kind of elaborate setup or prompt hacking, just a little common sense.

Post reply on HN