Live data from Hacker News

AI is breaking two vulnerability cultures

jefftk.com

61–70 of 188 posts

Re: AI is breaking two vulnerability cultures

#61
The bugs are bugs description reads pretty insane to me personally but I know linux world has many people valueing principle of it over practical matters.

90d seems long too though.

Think ultimately the big AI houses will need to help the core internet infra guys. Running latest and greatest AI over stuff like nginx and friends makes sense for us all collectively I think

Re: AI is breaking two vulnerability cultures

#62

Earlier quoted context omitted.

Bulk rewrites of everything into Rust with AI assistance?

I am looking at the results of a mass vulnerability scan as I type this. Half of the bugs in one case are in fact (binary) parser errors for hand-written parsers. These really should not exist in any language - but in C it's particularly bad. Kaitai Struct or something similar would broadly have prevented these. Rust would help here, but less than a parser generator (because it could automate error checking insertion…

I was just working on a system best thought of as a “dinosaur”: written almost entirely in C (and a bit of PERL) and running on an appliance with BSD as the kernel.

It’s full of bugs and has had a string of RCE vulnerabilities published recently, probably because of Mythos.

Working with it day to day I get this feeling that the tech stack used results in a system that’s… clumsy and constrained.

Little things give me that impression, and I can’t quite put it in words, but it’s thirty years of experience working with dozens of languages and platforms speaking here.

Using C makes you clumsy.

It makes you trip over things other languages don’t.

It makes it obscenely difficult to do even simple things. It’s like trying to put a delicate ship into a bottle while wearing oven mitts.

Switching to a better language isn’t just about the specific capabilities of its compiler, it’s also about what it enables in the humans using it.

Re: AI is breaking two vulnerability cultures

#63

This feels more like an old problem getting reframed as an AI problem. people were already diffing kernel commits and figuring out which ones were security fixes long before llms. if a patch lands publicly, the race has basically already started. also not sure shorter embargoes really help. the orgs that can patch in hours are already fine. everyone else still takes days or weeks. if anything, cheaper exploit generat…

Reminder: the Ksplice patent expires October 1, 2028.

What's the implications to that

Re: AI is breaking two vulnerability cultures

#64
> On the other side you have "bugs are bugs" culture. This is especially common in Linux, where the argument is that if the kernel is doing something it shouldn't then someone somewhere may be able to turn it into an attack. Just fix things as quickly as possible, without drawing attention to them. Often people won't notice, with so many changes going past, and there's still time to get machines patched.

The 3rd one is what I practice when giving companies time to fix their issue. Note, I haven't reported anything to FOSS projects, but to several companies I found exploits in. I give them 5 days. If they don't respond at all in the first day, I deduct 1 day - apparently they're either incompetent or don't care. After the 5 days have passed, I make it public. So far they've all fixed the issue on the 3rd or 4th day.

If I were to report something to a FOSS project, I'd give them a bit more, say 8-9 days. Enough time for everyone to wake up, review the vuln, patch it and ship it. Enough time for all the downstream projects to also ship the patch.

90 days is ridiculous, especially for companies. If I report something on Friday 23:30 and they reply Monday 15:00 - what were they doing during the weekend? Did they forget their software is used 24/7? I had one company complain quite a bit, threatening to sue. When they realized there was no one to sue (me being anonymous with my report), they fixed it in less than a day.

Bottom line - if you're a company offering a product or service, you should have a security team 24/7.

If you're a FOSS project - either alert your users to stop using your software or disable the service yourself, if you can.

If it's an extremely important life-or-death service you can't shut off - then fix it quickly. What are you doing with life-or-death stuff when you can't react quickly enough?

Fuck the 90 days standard - it's what companies want us to do because it's easier on them. If security hasn't been your top priority, you have a few days to make it your top priority.

With AI, that makes even more sense now. Bugs won't be able to stay hidden for months. Especially bugs I've reported like IDORs or SQL injections - things everyone tries first.

(and I love Linux, but getting an "Oh noes!" from Anubis at kernel.org because I don't have cookies enabled (I do??) really makes me not want to report anything to the Linux kernel in particular. If I ever did find something, I'd just immediately post it as a HN comment or something like that)

Re: AI is breaking two vulnerability cultures

#65

Earlier quoted context omitted.

I am looking at the results of a mass vulnerability scan as I type this. Half of the bugs in one case are in fact (binary) parser errors for hand-written parsers. These really should not exist in any language - but in C it's particularly bad. Kaitai Struct or something similar would broadly have prevented these. Rust would help here, but less than a parser generator (because it could automate error checking insertion…

I was just working on a system best thought of as a “dinosaur”: written almost entirely in C (and a bit of PERL) and running on an appliance with BSD as the kernel. It’s full of bugs and has had a string of RCE vulnerabilities published recently, probably because of Mythos. Working with it day to day I get this feeling that the tech stack used results in a system that’s… clumsy and constrained. Little things give me…

I don't disagree with that, but my point is that Rust will not really solve vulnerabilities.

Re: AI is breaking two vulnerability cultures

#66
post #47

I must admit I'm rather enjoying this particular form of shit show, mostly because it was a predication I made in 2023 in the early days of LLMs. It wasn't really a problem related to LLMs but a glaring hole in the thinking of current computing which is the "frustratingly over-connected" and "over-trust" approach to everything. After reading Liu Cixin's "three body problem" and noting the Dark Forest, I applied that…

If you're into gaming, Cyberpunk 2077 essentially plays in a heavily technologized world, where all compute infrastructure is infested with rogue AI that replicates itself to any technology it can physically get in touch with. The only recourse is a new web, built from first principles, protected by (probably) benevolent AI systems. Every device, every server, is partially occupied by AIs doing their thing on it, virtually networked into a digital universe. I found that a fascinating thought.

Re: AI is breaking two vulnerability cultures

#67
We need automated patch and release cycles. So far we've relied on incredibly slow manual processes to accept reports, investigate, verify, patch, and prepare releases. Releasing a fix often takes months. This is way too slow when attackers can just churn out new exploits in hours. We need to iterate on value chain bottlenecks to lower Mean Time To Patch.

We should be able to turn around a bug report to a patched product ready for QA testing in 1 hour. Standardize/open source it, have the whole software supply chain use it (ex. Linux kernel -> distros -> products that use distros -> users). With AI there's no reason we can't do this, we're just slow.

Re: AI is breaking two vulnerability cultures

#68
post #38

Earlier quoted context omitted.

Very often you only have limited time for investigation and you have to act now . Action is almost always based on educated guesses.

You have moved from "We know" to "We have an educated guess" which is the right way to couch things. However I wanted to also point out that relying only on educated guesses can lead us into a position where we are "papering over the cracks" or "addressing the symptoms", not the "underlying cause" Yes, sometimes that's all that can be done, but, also, sometimes it can be more damaging than the cause itself (thinking…

> You have moved from "We know" to "We have an educated guess"

No. You kept blabbering about "science" when most uses of knowledge are not about science. The original topic was also definitely not "science": it was about having a reasonable opinion about whether, empirically, the rate of discovery of vulnerabilities is increasing or not.

Re: AI is breaking two vulnerability cultures

#69
post #49
post #46

We have a huge problem. The US is at war. Much of the world is at war at the cyber attack level right now. The US, the EU, most of the Middle East, Israel, Russia... Major services have been attacked and have gone down for days at a time - Ubuntu, Github, Let's Encrypt, Stryker. Entire hospital systems have had to partially shut down. Now, in the middle of this, AI has made attacks much faster to generate. Faster tha…

> before it gets better How is it going to get better?

1) Make it a law that companies have to vet their code for security holes before release, 2) Make it a law that companies have to apply operational security best practice on their software products/services, 3) Industry standard automation for improvements to patch lifecycle management, 4) Auditing for critical businesses and industries to ensure safety (both as a national security thing and general safety/reliability/privacy/etc)

Right now all that stuff is optional, so most companies don't do it, which makes more security holes and it takes longer to patch.

Re: AI is breaking two vulnerability cultures

#70
post #64

> On the other side you have "bugs are bugs" culture. This is especially common in Linux, where the argument is that if the kernel is doing something it shouldn't then someone somewhere may be able to turn it into an attack. Just fix things as quickly as possible, without drawing attention to them. Often people won't notice, with so many changes going past, and there's still time to get machines patched. The 3rd one…

> 90 days is ridiculous, especially for companies

It depends on the kind of vulnerability, but sometimes in order to fix a problem, you need to do an enormous amount of software engineering. Which needs to be done to a very high standard, because the expectation is that people will push security patches more or less immediately to production.

Of course, this only works if no one else is likely to discover the vulnerability in the meantime!

Post reply on HN