Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI is breaking two vulnerability cultures

jefftk.com

Post reply on HN
Print view View original

11–20 of 188 posts

Re: AI is breaking two vulnerability cultures

#11

Post by pixl97 »

Analemma_ wrote: post #2

I'd argue it's actually breaking three vulnerability cultures. In addition to the two Jeff mentions, I think the culture of delaying upgrades and staying on stable versions for as long as possible is going to become increasingly untenable, if everything that's not latest can be trivially scanned and exploited. In the extreme I think there's a decent chance projects like Debian might have to radically overhaul or just…

We are now paying for the sins of our fathers (well and mostly ourselves).

We've just kept building more complex things with more exposure with no recognition that the day of reckoning was coming. And now we are in an untenable situation. With governments spending billions on AI with the big providers it's likely they've found many of these already.

Re: AI is breaking two vulnerability cultures

#12

Post by tetha »

Analemma_ wrote: post #2

I'd argue it's actually breaking three vulnerability cultures. In addition to the two Jeff mentions, I think the culture of delaying upgrades and staying on stable versions for as long as possible is going to become increasingly untenable, if everything that's not latest can be trivially scanned and exploited. In the extreme I think there's a decent chance projects like Debian might have to radically overhaul or just…

> In the extreme I think there's a decent chance projects like Debian might have to radically overhaul or just shut down completely - the whole philosophy of slow and steady with old code just won't work.

It may actually be the opposite.

Debians steady and professional approach on shipping security patches with very little to no functional difference actually enables us to consider and work on automated, autonomous weekly or faster patches of the entire fleet. And once that's in place and trusted, emergency rollouts are very possible and easy.

We have other projects that "move fast and break things" and ship whatever they want in whatever versions they want and those will require constant attention to ship any update for a security topic. These projects require constant human attention to work through their shenanigans to keep them up to date.

Re: AI is breaking two vulnerability cultures

#13

Post by papichulo2023 »

Maybe it is about time for Linux to get a real CD/CI and start using AI extensively.

Not just for vulnerabilities, having a nice agents|skills|etc.md definitions would encourage new devs to contribute instead of dealing with an overworked maintener repeating the same thing for n time.

Re: AI is breaking two vulnerability cultures

#14

Post by ryandrake »

layer8 wrote: post #10
Analemma_ wrote: post #2

I'd argue it's actually breaking three vulnerability cultures. In addition to the two Jeff mentions, I think the culture of delaying upgrades and staying on stable versions for as long as possible is going to become increasingly untenable, if everything that's not latest can be trivially scanned and exploited. In the extreme I think there's a decent chance projects like Debian might have to radically overhaul or just…

> there's a decent chance projects like Debian might have to radically overhaul or just shut down completely - the whole philosophy of slow and steady with old code just won't work. Debian continuously issues security updates for stable versions, ingestable with automatic updates. “Stable” doesn’t mean that vulnerabilities aren’t getting fixed. The argument that could be made is that keeping up with getting vulnerabi…

I can only dream, but this may re-popularize (among the rest of the non-Debian software industry) the general best practice of keeping a "sustaining" branch green, buildable, and with frequent releases, for security fixes.

I hate software that forces you to take new features as a condition of obtaining bug and security fixes. We need to keep old "stable" builds around for longer and maintain them better. I know, I know, it is really upsetting to developers to have to backport things to old versions--they wish that all they had to work on was the current branch. But that just causes guys like me to never upgrade because the downside of upgrading (new features) is worse than the upside (security fixes).

Re: AI is breaking two vulnerability cultures

#15

Post by cubefox »

xiaoyu2006 wrote: post #9

The quick test doesn't show a lot - by out straight asking if this is a security patch, it implies and guides AI to have output more probably to agree on this assumption. A confusion matrix is more useful. Nonetheless of course this is not a detailed ai capability testing blog.

Yeah, ideally we would need the phi coefficient (aka MCC, the binary Pearson correlation), which can be calculated from a confusion matrix of yes/no LLM classifications for all kernel diffs. (Number of true positives, true negatives, false positives, false negatives.)

Re: AI is breaking two vulnerability cultures

#16

Post by calvinmorrison »

tetha wrote: post #12
Analemma_ wrote: post #2

I'd argue it's actually breaking three vulnerability cultures. In addition to the two Jeff mentions, I think the culture of delaying upgrades and staying on stable versions for as long as possible is going to become increasingly untenable, if everything that's not latest can be trivially scanned and exploited. In the extreme I think there's a decent chance projects like Debian might have to radically overhaul or just…

> In the extreme I think there's a decent chance projects like Debian might have to radically overhaul or just shut down completely - the whole philosophy of slow and steady with old code just won't work. It may actually be the opposite. Debians steady and professional approach on shipping security patches with very little to no functional difference actually enables us to consider and work on automated, autonomous w…

Not only that but debian has for example, debsecan so you can see on any system what CVEs exist and if your packages are patched. ex from my system I ran it and got

> CVE-2026-32105 xrdp

which i see has a fix in sid but not on bookworm

Re: AI is breaking two vulnerability cultures

#17

Post by giancarlostoro »

Analemma_ wrote: post #2

I'd argue it's actually breaking three vulnerability cultures. In addition to the two Jeff mentions, I think the culture of delaying upgrades and staying on stable versions for as long as possible is going to become increasingly untenable, if everything that's not latest can be trivially scanned and exploited. In the extreme I think there's a decent chance projects like Debian might have to radically overhaul or just…

Arch Linux to become the only Linux OS left.

Re: AI is breaking two vulnerability cultures

#18

Post by jefftk »

xiaoyu2006 wrote: post #9

The quick test doesn't show a lot - by out straight asking if this is a security patch, it implies and guides AI to have output more probably to agree on this assumption. A confusion matrix is more useful. Nonetheless of course this is not a detailed ai capability testing blog.

[author]

I agree it is not much additional evidence! If someone wanted to try running the same test on a series of N commits from that list including this one I'd be very curious to see the answer!

Re: AI is breaking two vulnerability cultures

#19

Post by gritspants »

JumpCrisscross wrote: post #8
rikafurude21 wrote: post #3

This feels more like an old problem getting reframed as an AI problem. people were already diffing kernel commits and figuring out which ones were security fixes long before llms. if a patch lands publicly, the race has basically already started. also not sure shorter embargoes really help. the orgs that can patch in hours are already fine. everyone else still takes days or weeks. if anything, cheaper exploit generat…

> people were already diffing kernel commits and figuring out which ones were security fixes With skill, and usually not consistently and systematically. With AI, anyone can do this to any software. > not sure shorter embargoes really help Why 90 days versus 2 years? The author is arguing the factors that set that balance have shifted, given the frequency of simultaneous discovery. The embargo window isn’t an actual…

I'm here for white-hate culture. You should, you should know better.

Re: AI is breaking two vulnerability cultures

#20

Post by Hizonner »

JumpCrisscross wrote: post #8
rikafurude21 wrote: post #3

This feels more like an old problem getting reframed as an AI problem. people were already diffing kernel commits and figuring out which ones were security fixes long before llms. if a patch lands publicly, the race has basically already started. also not sure shorter embargoes really help. the orgs that can patch in hours are already fine. everyone else still takes days or weeks. if anything, cheaper exploit generat…

> people were already diffing kernel commits and figuring out which ones were security fixes With skill, and usually not consistently and systematically. With AI, anyone can do this to any software. > not sure shorter embargoes really help Why 90 days versus 2 years? The author is arguing the factors that set that balance have shifted, given the frequency of simultaneous discovery. The embargo window isn’t an actual…

> With skill, and usually not consistently and systematically.

How do you know? If the people who like to crow about vulnerabilities aren't doing it, it doesn't mean that the people who are actually in a position to exploit them systematically and effectively aren't doing it.

Those embargoes have always been dangerous, because they create a false sense of security. But, as you point out...

> With AI, anyone can do this to any software.

Yep. Even if it hadn't been true before, it's clear that now you just have to assume that everybody relevant will immediately recognize the security impact of any patch that gets published. That includes both bugs fixed and bugs introduced.

... and as the AI gets better, you're going to have to assume that you don't even have to publish a patch. Or source code. Within way less time than it's going to take people to admit it and adjust, any vulnerability in any software available for inspection is going to be instant public knowledge. Or at least public among anybody who matters.

Post reply on HN